Music video scraping still sucks!
#16
(2018-10-26, 04:36)IPhantom Wrote: 1. Add the file per file name and/or local info only
Have a look at this scraper. It will scan the title into the library based on filename only... https://forum.kodi.tv/showthread.php?tid...pid2779060
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#17
@IPhantom , part of the issue might be that there is simply no data available for the track/artist combination that you want.

I wrote a small python script to connect to tadb's musicvid api and ran all your tracks through it.  Result....

Code:
Weak by AJR
no data available at all

Darkside by Alan Walker
No data for that track available

Destination Calabria by Alex Gaudino

Data returned from tadb for track Destination Calabria (UK Radio Edit) is as follows
The music video is directed by Eran "Rani" Creevy and produced by Ben Pugh for Ministry of Sound. It features female dancers in highly sexualized green marching band costumes, pretending to play various instruments in a seductive manner, with choreography by David Leighton. In some scenes, seemingly hundreds of the dancers are seen at once, but these are simply the original eight replicated many times using digital imagery. Waters herself does not appear in the video, with some of the marching band members miming her lyrics.

I Love Rock n Roll by Alex Gaudino
No data for that track available

Watch Out by Alex Gaudino
No data for that track available

Truth by Alexander
no data available at all

U Dont Know by Alison Wonderland
No data for that track available

Moi Lolita by Alizée
No data for that track available

Breezeblocks by alt.J
no data available at all

Im a Mess by Bebe Rexha
No data for that track available

Boys by Charli XCX
No data for that track available

You by Charli XCX
No data for that track available
<snip>

I only got the one hit as you can see.  As tadb is community driven, you could always add the missing info to it and then it would be available to everyone.

@docwra, maybe not the place to ask here, but would it be possible to extend the musicvid api to artist &/or track names ?  Currently the only two methods appear to be either mbid look-up (which is the way I did it above) or tadb artist id.  I did think about using the latter and for a small test it wouldn't matter, but getting the artistid first would entail twice the look-ups.  I just re-used a routine I already had to lookup mbids from musicbrainz and then did the search with that.  Based purely on  the fact that streaming radio helper uses the same code it's pretty accurate, but tadb artistid might be even more so.
Learning Linux the hard way !!
Reply
#18
(2018-10-26, 04:36)IPhantom Wrote: Thanks for looking that up.
I can see that this makes sense for movies where there's a special edition, director's cut, etc. But here it's kind of an overkill. In doubt force best match.
My point is still valid. I wish they could:

1. Add the file per file name and/or local info only
2. Scrape

It shouldn't be an either or option and a nfo file should definitively not be mandatory to start with.
Yep I agree, will have a look at this later today.
Reply
#19
(2018-10-26, 12:13)black_eagle Wrote: @docwra, maybe not the place to ask here, but would it be possible to extend the musicvid api to artist &/or track names ?  Currently the only two methods appear to be either mbid look-up (which is the way I did it above) or tadb artist id.  I did think about using the latter and for a small test it wouldn't matter, but getting the artistid first would entail twice the look-ups.  I just re-used a routine I already had to lookup mbids from musicbrainz and then did the search with that.  Based purely on  the fact that streaming radio helper uses the same code it's pretty accurate, but tadb artistid might be even more so. 

Actually this already exists as per the API instructions here

Return track details from artist/track name
php:
theaudiodb.com/api/v1/json/{APIKEY}/searchtrack.php?s={Artist_Name}&t={Single_Name}
The problem I believe is 2 fold:

1) The API method doesn't match tracks as it should (due to performance issues in the past, I removed the SQL % bit as it wasn't using the indexes).
2) The database doesn't contain some of the tracks (this is up to the users to add them). Anything can be imported from musicbrainz very quickly so this should be easy and of course it benefits all users forever into the future. Personally when I add data I use the "Add album" feature on the artist page, then go to the specific track. Add a description, genre and youtube link if available. I also make sure the album and track has an image uploaded. Sometimes I even add a music video director and production company so its complete. This could be useful way to sort music videos using nodes such as "sort by director" inside Kodi but might be a bit advanced for most users Wink
Reply
#20
OK that didn't take long. Try this test API method out for size:

php:
https://theaudiodb.com/api/v1/json/1/sea...20Calabria

Now that matches Alex Gaudino - Destination Calabria (radio edit) properly.

So far this API method is using this pre-processing.
Quote:    // Remove anything inside brackets, braces
    $trackName = preg_replace('/[\[{\(].*[\]}\)]/U' , '', $trackName);

So now a question for users, what special musicbrainz characters cause issues? Now I am doing some server side processing we have a chance to fix this once and for all.

EDIT1: Apostrophes have now been standardized over the entire site, there should be no more issues with that kind of match.  5405 rows affected. (Query took 20.1865 seconds.)
Reply
#21
Yes, that's better.  I get five hits with the same list of tracks rather than just the one I got previously.  The rest appear to not be in the database {"track":null}.

I don't seem to have any issues with special characters unless I forget to decode and encode in the appropriate places.
Learning Linux the hard way !!
Reply
#22
I haven't tried lately, but titles that "are" a number could not be scraped. Like Prince - 1999 or Phoenix - 1901 for example.
Reply
#23
(2018-10-26, 12:46)docwra Wrote: So now a question for users, what special musicbrainz characters cause issues? Now I am doing some server side processing we have a chance to fix this once and for all.

EDIT1: Apostrophes have now been standardized over the entire site, there should be no more issues with that kind of match.  5405 rows affected. (Query took 20.1865 seconds.) 
1. Panic! At The Disco - (so "!" ).
2. AC/DC - (so the "/" )
3. Ke$ha - (so the "$" )

for a complete list, check out this website...
https://ask.metafilter.com/242226/BandsA...ic-symbols
Reply
#24
(2018-11-02, 17:30)Powerhouse Wrote:
(2018-10-26, 12:46)docwra Wrote: So now a question for users, what special musicbrainz characters cause issues? Now I am doing some server side processing we have a chance to fix this once and for all.

EDIT1: Apostrophes have now been standardized over the entire site, there should be no more issues with that kind of match.  5405 rows affected. (Query took 20.1865 seconds.) 
1. Panic! At The Disco - (so "!" ).
2. AC/DC - (so the "/" )
3. Ke$ha - (so the "$" )

for a complete list, check out this website...
https://ask.metafilter.com/242226/BandsA...ic-symbols 
 None of those cause an issue for me when using TADB's api.  In fact, it's clever enough to return 'Ke$ha' if you supply it with 'Kesha' as an artist to search for, although searching for 'Ke$ha' works perfectly well.
Learning Linux the hard way !!
Reply

Logout Mark Read Team Forum Stats Members Help
Music video scraping still sucks!0