[mrjwm2]

------Edit------



Link to scraper files
Install first : metadata.common.musicvideos.mp4

Install last : metadata.musicvideos.mp4

Github: Github Repo


In MB Picard ensure you have these settings:
Options>Options>File Naming: Check Rename files when saving. Add this line in 'Name files like this' box:

(%musicbrainz_albumid%)%musicbrainz_recordingid%(%musicbrainz_workid%)%musicbrainz_artistid%


Options>Options>Metadata: Check Use standardized artist names, Convert Unicode punctuation characters to ASCII, Use release relationship, Use track relationships


Test it out with a few videos. Once you get used to Picard and I think you will like the result. Picard is a powerful tool that is easy to use once you figure it out.


------[End Edit------


I have written a scraper for music videos that is working and returns data for the following fields:

Title, Artist, Album, Runtime, Year, Studio, Genre, Plot, Track #, Album cover and Director.

These fields are populated only if information is available. From what I have tested. Director field returns <30% data. All other fields are 97%+ on returned data. I scraped 388 various titles continuously and had 4 videos that did not scrape ( I knew that prior to testing as well). 384 scraped with all fields except for Director populated at 100%. Director populated in 52 of the 384 videos. It took 14 minutes to scrape 388 videos.

Plot is filled first with Artist description, and if found, overwritten with Title(Song) description. This information is from Wiki (credit to Wiki is properly given in the OSD of plot).

I have written an additional scraper that will return Fanart for the titles as well. It needs to be run first, then the library must be exported to nfo files. At this point you scrape the mp4 files as music video and you have everything in one place. (more about this at a later date if users think it worthwhile)

The bad news:
This scraper ONLY works with files properly tagged with MusicBrainz Picard. Picard v1.3 and newer supports .mp4 .wmv .asf .ogv and .m4v

My reasoning for going this direction is to help reduce the bandwidth for the database hosts. Current music video scrapers are kind of a 'wam bam, thank you maam' approach. I mean no disrespect with this statement, it is that the results are achieved by not using predicted results. The scrapers take a users artist and song name and try to push it through. Artist and song names spelled incorrectly or not matching the database are ignored. The user then tries again and again and again and again. The users video is not scraped, and the website host gets a large bill in the mail. Everyone is unhappy!

My approach is to start out so your results are what YOU want. This is achieved only when the video is tagged correctly.
My scraper users a file name like this: (d5cb50d8-8897-4029-b9b3-32806df7e06d)c3fe7791-0a91-4f0a-a89b-b056f38d3cde(05980852-95f4-4ea2-806a-abf1666951b8)20244d07-534f-4eff-b4d4-930878889970.mp4

Looks crazy? Really... it is very smart. It is the MB(musicbrainz)ID for release, recording, work and artist. The great thing is MB Picard will write these file names for you in just a few seconds, in the current folder you have them. Your folder names can be whatever you want, does not matter. I name mine "artist name"-"song title". This way I always know what the file is. If I ever need to go back to a file name like " artist" - "title" Picard will do it correctly in seconds.

The scraper does not care what any name, title or album name is. It will return an artist name like P!nk, "Weird Al" and Sinéad O'Connor. The same for any other field. Data returned containing / " : : { ( & $ is not a problem.

I must emphasize that the data returned is only as good as how it is tagged in Picard.

If you select a release that does not have a 'real' coverart release ID assigned it will not display in Kodi. By this I mean, when you tag, start with Artist. Then select the release(album) you want. Look in the upper right corner... hover over the album cover you will see something like: https://coverartarchive.org/release/3ccb...10-250.jpg display in your taskbar. This is a good coverart! If MB displays "Coverart from Amazon or no coverart you get nothing.

Anyway, if this is something that looks like it may work for you, please let me know. I am in the process of setting up github (uggh) and will give a link shortly.

MB Picard ONLY tags .mp4 .wmv .asf .ogv and .m4v video files.

Kodi's musicvideo database does not natively support fanart. My work around is to scrape the music video as 'movie'. Export library nfo files. Rescrape same files as musicvideo and there you have it. All files needed in the original folder. It works, and not rocket science.

Notable is that currently MB allows one request per second from your URL and the scraper. I am able to work around this using different sites, and slowing the scraper down to stay within the limit. At the moment I am refining that a bit more to ensure that scraping is uninterrupted. Currently I am at ~2.1 seconds per request. (Kodi coders: adding fanart to the music video database would allow this no matter what restriction is put on requests)

Once posted, please excuse any code syntax errors, and let me know so I can fix it. I knew zero about a scraper until a week or so ago.

I know this is not for everyone. It for the user that wants to display the correct info for the video. Or the info that the users wants to see. You choose when tagging. MB has a mammoth database and you will have a hard time find a title not there.

You will not find many Concerts. I associate all my music videos with a song from a release(album). For the purpose of this scraper concerts are not musicvideos.

Anyway, Let me know in this thread if you are interested.

Thanks to Zag for allowing use of TADB. Thanks to MBeu!

mrjwm2

[mrjwm2]

Updated with file links. See above.

[zag]

Nice! Subscribed!

EDIT: very nice to see MP4 only, I ripped all the youtube music videos so this could work very well for me. Will report back once I test it.

[DarkHelmet]

Since I'm somehow stranded with adding new music videos I'm interested as well. I have a few questions:

Does the scraper read the file names or the tags in the file or both? As an example: Can I keep the file name "artist - song" and use the musicbrainz-ID tag field and/or any other tag fields like artist, title, track, album, year etc.? Can it also use embedded artwork?

To be honest I'd rather I could keep the file names "artist - song". Having Musicbrainz IDs as the file names would leave me pretty clueless when watching my files in any file browser, since I don't know the Musicbrainz ID by heart.

[mrjwm2]

Zag,
I too had many youtube videos I needed to get in the library. Using picard to tag and name them was perfect. I then needed this scraper to get them in the library. One thing to note is the album art. Make sure that when you tagged... it is a good artwork... artwork that has a release url associated. Some artwork says from amazon. The amazon stuff will not scrape. I went for accuracy with this scraper. Kodi will display info the user wants the video to be associated when they tag.


DarkHelmet:
The scraper reads file name only. The tagging process is the means to get the file name.

(d5cb50d8-8897-4029-b9b3-32806df7e06d)c3fe7791-0a91-4f0a-a89b-b056f38d3cde (05980852-95f4-4ea2-806a-abf1666951b8)20244d07-534f-4eff-b4d4-930878889970.mp4

(%musicbrainz_albumid%) %musicbrainz_recordingid%(%musicbrainz_workid%) %musicbrai​nz_artistid%

The only way to get the correct file name (or at least the easiest way) is to tag the mp4 with MB Picard. I have my video named with the goofy MB ID's, but they reside in a folder named "Artist - Song". If you ever want to go back to the 'artist-song' name for the file, MB Picard will do it in seconds.

I think it is a good thing to have the mp4 video tagged properly, as 'maybe' down the road Kodi will read/use the tag info when adding source. Kodi does in fact read the tags when you play the video (have a look at the log). I am not sure what Kodi does with this info tho.

I am making a scraper that uses the 'artist - song' format. It is basic, but most info will be returned. No guarantees on how accurate it will be. Date, runtime, Album most probably will be incorrect. Not sure how to get this scraper out there tho. User will need their own API key.

[leejk]

Just curious.... most music videos bought from iTunes are not MP4, but m4v I think. Why wouldn't it work with these?

[mrjwm2]

leejk,

You're correct. MB Picards documentation states mp4 only. When I did a search: "here" I see other video formats supported with verison 1.3 of Picard. I tested Picard with a m4v file and it worked. I don't have wmv, asf, or ogv to test.

(Edit)------I don't think Kodi is able to play m4v files from iTunes. The file has drm protection.

[DarkHelmet]

The scraper reads file name only. The tagging process is the means to get the file name.

I am making a scraper that uses the 'artist - song' format. It is basic, but most info will be returned. No guarantees on how accurate it will be. Date, runtime, Album most probably will be incorrect. Not sure how to get this scraper out there tho. User will need their own API key.

Your work is greatly appreciated. and I wanted to tag my mp4 files anyway. Kodi isn't however my only means for media playback. Therefore renaming my files to the mb id is not a suitable solution for me.

Concerning an API key, I believe the universal artist scraper and also the universal album scraper call musicbrainz too and I definitely don't have an API key. At least the kodi wiki states

"The initial search is always done on MusicBrainz."

http://kodi.wiki/view/Add-on:Universal_Artist_Scraper

http://kodi.wiki/view/Add-on:Universal_Album_Scraper

I have no idea about the insides of the scrapers but maybe it's worth checking out.

[mrjwm2]

DarkHelmet-

Thanks, and I understand that this scraper is not ideal for everyone. In any event, you should consider tagging if possible. Read, ask and practice with a few to get it right the first time. It's a bummer to go back and do it again and again. Trust me I did that.

Hang tight, I have a scraper pretty much complete that will work for you. It uses artist - song name as the base to scrape from. It may be a week or so until I get the time to complete and debug it. I must inform that it will not be accurate! It takes the artist - song name and returns whatever it finds first (well... kinda first), nevertheless, it will add it to your library.

Using the commons of the default scrapers you mentioned would work somewhat, but it's flow is not what I am looking at. Easier for me to re-write them, as I just learned this. The music-video database entries are very limited in comparison to music database.

FYI, the reason I wrote the mp4 scraper is because I did not like the inaccurate results of the default music scrapers. I already modified them and tried them for music videos. I have since modified/rewritten the default music scrapers you mentioned to work with my goofy name scheme, now all music scraped to my library match my music files... 100%. Tagging is key!

API key will be required, I will explain when I get the scraper done. It is no big deal, but I am not allowed to give mine out. The user needs to create a profile at a site, and the key will be automatically generated for them. Simple stuff.

[mrjwm2]

Zag-

Have you tried this scraper out yet? If not, I have a way to get fanart and extrafanart in the folders. But,this needs to be done first.

Pretty easy stuff, I have two scrapers to get the artwork. From two sources.

Run the fanart scraper, export the source, run a couple batch files I wrote, repeat for as many extrafanart files you want (with one easy xml edit), re-scrape as music video, done. It does take a bit of time. I do 50-75 at a time, about 15 minutes for the batch.

Let me know if interested, and I'll have to get you the files.

[scf2k]

Btw, do you know that musicbrainz provides the dumps of their database? You can download just dumps or complete server virtual machine image. The server can also sync with their live feed so you will have up to date data.

I thought that anyone interested could setup own database and spam it with requests as much as they want. This allows the plugin to do more analytics and database requests than the existing plugins can do. Though kodi XML scraper api takes into place with its limitation. But as I know this could be worked around by putting most of the logic into separate "service" addon. Which can benefit of using real programming language.

[fnac]

hi,
thanks for this effort. I've been waiting VERY long to get my 5000+ music videos in to Kodi in a nice way. Unfortunately I have to agree with earlier speakers about the file naming since I'm not to keen on renaming in the musicbrainz id format. I want to have original file name left untouched.

I have all my music videos in mp4 format and I have used Mediamonkey to tag artist, song, year, genre and in some cases ratings. Reason for this is that I want the discogs genre standard which aligns with my music collection.

So, I then have all my mp4 music video files perfectly tagged and basically just want some scraper to read that info and import to Kodi!

so my question is similar to others, if we just could have the existing meta data in the mp4 file read into Kodi some way?

cheers

[zag]

so my question is similar to others, if we just could have the existing meta data in the mp4 file read into Kodi some way?

I doubt that is going to happen, that would involve writing a tag reader for video files...

Name matching is the only way at the moment to use video scrapers.

[DarkHelmet]

Zag, Kodi can already read tags in mp4 video files perfectly if you just use a music scraper on the files (add mp4 to musicextensions in advancedsettings and let a music scraper run). It is however not possible to use these tags for the video library.

See my feature request here:

http://forum.kodi.tv/showthread.php?tid=248452

[DarkHelmet]

DarkHelmet-

Thanks, and I understand that this scraper is not ideal for everyone. In any event, you should consider tagging if possible. Read, ask and practice with a few to get it right the first time. It's a bummer to go back and do it again and again. Trust me I did that.

I will do that definitely. Only takes a load of time for my collection of music videos but I believe in the long run using embedded tags is a good way.

I'm a bitch when it comes to music and music videos. Therefore I can easily live with uncorrect results. I scrape each title individually. Hence a scraper that presents me several results and lets me pick the right one is totally fine for me.