WIP Stupid musicvideo scraper
#1
This scraper is now available in my repository @ github. Link below.

Install, scrape and enjoy. No frills... just the ability to add ANY file to your library.
----------------------------------------------------

After reading quite a few posts, and a need for myself, I have written a scraper that will add any file to your musicvideo library.

The only "must have" is the file name format:

artist - song name.extension

Note the space before and after the hyphen ( - ).

This format is used by other music video scrapers, so I used it to keep things simple.

What this scraper does?
It grabs the file name, inputs that file name into a Yahoo search, then extracts back the artist and song title which is added to the Kodi database. It is a stupid scraper. Kodi will display artist name and song name. It is also sortable within those 2 parameters.

Spelling, accuracy, special characters do not matter. Whatever you have the file named will return back to Kodi. If you have sloppy file names... that's what comes back.

If there is enough interest I will post it on Github. Let me know with a reply.

********* Download from here: stupid scraper
Reply
#2
In addition to the above post:

I mentioned in another thread that I wrote a scraper that returns all information for a musicvideo. I have it complete, but I do not like the results. My scraper has similar results to the TADB and IMVDB scrapers.

I ran ~300 videos through each of them and ~10% of the 300 where not scraped, ~20% where scraped inaccurately. These results are frustrating to fix.

It is near impossible to take an artist name and song title and get any kind of accurate results (hence my mp4 scraper).

My question is this:

Are enough users interested in adding the album name to the file name? (artist - song name - album name.extension)

If enough people are interested I can modify my scraper to include the album name to get accurate results. I expect 99%. The scraper searches AllMusic for info. AllMusic does pretty well with inconsistent spellings of the artist, song and album names.

(note: My files are tagged in Musicbrainz Picard and the artist, song and album names of my files are automatically generated by Picard (these names are consistent with Discogs, AllMusic and Musicbainz naming scheme). My file names and tags have any feat. artist stuff stripped as well. If your file names are from youtube, vevo, mtv, made up or best guess your results will be much lower. These sites generally add a lot of nonsense to the file name.)

If interested let me know with a reply.
Reply
#3
I'd be interested. Personally I scrape all my videos manually anyway to check it is the right one.
Reply
#4
DarkHelmet, which one? stupid or add album name? or both? lol

FYI; The add album name version will hopefully display "song name - album name - year" when manually selecting on a reload/refresh. I think manual mode won't be needed much tho.
Reply
#5
(2015-11-28, 00:03)mrjwm2 Wrote: DarkHelmet, which one? stupid or add album name? or both?

Yes! ;p

Okay, kidding aside. If I can choose freely I'd take the one with the album name.

Thanks in advance.

Kinda annoying somehow that there is a working scraper (for official music videos at least) here http://forum.kodi.tv/showthread.php?tid=205755 but it returns the album as the artist too. I'll look into that tomorrow but I have to admit I'm a complete noob in these things.
Reply
#6
The internet music video database does not associate albums to music videos at all. Hence scraping that site will not work in respect to the album field for a music video. Bummer.
Reply
#7
Please post to github
Reply
#8
Did you mean me, zag? I don't have an account at github. Since I'm a not a dev in any way I kinda feel out of place there.

Where should I post it? Here?

https://github.com/tphoenix/metadata.musicvideos.imvdb
Reply
#9
No the scraper in the 1st post Wink
Reply
#10
(2015-11-27, 00:32)DarkHelmet Wrote: I'd be interested. Personally I scrape all my videos manually anyway to check it is the right one.

Hi

Can you please tell me how to do this? Since no scraper is working i have to get this working soon. Smile

I'm using this now:

<musicvideo>
<title>Summer</title>
<artist>Calvin Harris</artist>
<album>Motion</album>
<genre>Electro house dance-pop</genre>
<runtime>3:44</runtime>
<plot> </plot>
<year>2014</year>
<director>me</director>
<studio>Calvin Harris</studio>
<track>1</track>
</musicvideo>

But it does not work for me. I changed to local where the scrapers goes in settings. Nothing happens Sad
What am i doing wrong?
Asus s1-at5nm10e - Netgear READY NAS 102 - Wired -
-Receiver Pioneer VSX-529-K - ASUS RT-AC58U - USB HDD connected Seagate - SONY BRAVIA KDL-55W955B - Netgear ReadyNAS 102
Reply
#11
By manually scraping I meant that I pick new videos in my musicvideo folder, open the menu and then choose music video information. However this requires a working scraper. Since there is none right now, I'm pretty much stranded and don't add new music videos right now and hope that a working scraper will show up at some point.

I take it that that was the contents of an nfo file you just posted here, right? Do the musicvideo and the nfo fil have exactly the same name except for the file type extension of course. coolsong.mp4 and coolsong.nfo for example.
Reply
#12
Aha didn't got that Smile

After a couple of changes it works. THANKS! But doing this manually is not an option for all my music vids. Smile But now i CAN if i want to and that's realy nice.

Thanks again!
Asus s1-at5nm10e - Netgear READY NAS 102 - Wired -
-Receiver Pioneer VSX-529-K - ASUS RT-AC58U - USB HDD connected Seagate - SONY BRAVIA KDL-55W955B - Netgear ReadyNAS 102
Reply
#13
Stupid scraper now at github, read first post.

I am finalizing the other scraper mentioned in my second post. Been real busy lately. Hope to have it in the next couple days. ( I have 39 regex for the getsearchresults so far :p ).

Here is an example of one:

id="([^"]*)" extConfusedcore="100"><title>([^"]*)</title><status>Official</status><text-representation>(?:<language>[^"]*)</language>(?:<script>[^"]*)</script></text-representation><artist-credit><name-credit><name>(?:[^"]*)</name>(?:<artist id="[^"]*)"><name>([^"]*)</name>(?:<sort-name>[^"]*)</sort-name><alias-list><alias locale="(?:[^"]*)" sort-name="(?:[^"]*)" type="(?:[^"]*)">(?:[^"]*)</alias><alias locale="(?:[^"]*)" sort-name="(?:[^"]*)" type="(?:[^"]*)">(?:[^"]*)</alias>(?:<alias sort-name="[^"]*)" type="(?:[^"]*)">(?:[^"]*)</alias>(?:<alias sort-name="[^"]*)" type="(?:[^"]*)">(?:[^"]*)</alias><alias locale="(?:[^"]*)" sort-name="(?:[^"]*)" type="(?:[^"]*)" primary="(?:[^"]*)">(?:[^"]*)</alias><alias locale="(?:[^"]*)" sort-name="(?:[^"]*)" type="(?:[^"]*)">(?:[^"]*)</alias></alias-list></artist></name-credit></artist-credit>(?:<release-group id="[^"]*)" type="([^"]*)">(?:<primary-type>[^"]*)</primary-type></release-group><date>([^"]*)</date>(?:<country>[^"]*)</country><release-event-list><release-event>(?:<date>[^"]*)</date>(?:<area id="[^"]*)"><name>([^"]*)</name>


These take time write, and necessary to get a match against the varying xml's. I will be satisfied when I get 95% of all releases for an album to return. Almost there.
Reply

Logout Mark Read Team Forum Stats Members Help
Stupid musicvideo scraper1