[meowmoo]

So you are saying this thing in Music area is ONLY a single request? Because I can clearly see in the LOG file and with my eyes when I click on it that on top of the screen albums appear/disappear pretty fast while searching for info.



Music Library mode with universal scrapper, that means it has nothing to do with this scrapper? And it is some kind of internal addon of KODI?

Offtopic....
Is this aeonmq?

[Merwenus]

Amber

[docwra]

Some albums are missing the THUMB picture. So I checked TheAudioDB.com maybe they are missing it so I upload them. But it says:
{Please Login to see images}

For example
http://www.theaudiodb.com/album/2159639




How can I "login" to the scrapper to allow audiodb.com to download EVERY pictures? Because with a freshly made library I have way too much missing thumbnails.

You need to register on the TADB site and then images will show. You can then also upload images to the website so everyone benefits.

[docwra]

I don't think it is the scraper doing more than 1 per sec. Probably an addon that is hammering Musicbrainz.

1 second isn't enough, we will hit the rate limiter after about 10 hits with that delay. It needs to be longer.

My tests show 2 seconds should be just about enough, but I default to 5 seconds on my personal MusicBrainz scripts to be sure.

[Merwenus]

You need to register on the TADB site and then images will show. You can then also upload images to the website so everyone benefits.

You missread something or I wasn't clear enough. I know the site, I used to upload images and fill informations. Thank you.

My problem is that in my Kodi Music Library, using Universal Scrapper, with settings to download and USE thumbnails from audiodb is broken, because it does not download a few images (mostly the ones you need to login). That is why I asked for help, so I can login to the scrapper, and download automatically these missing thumbs with my account.

[docwra]

My problem is that in my Kodi Music Library, using Universal Scrapper, with settings to download and USE thumbnails from audiodb is broken, because it does not download a few images (mostly the ones you need to login). That is why I asked for help, so I can login to the scrapper, and download automatically these missing thumbs with my account.

Ahh well the album you linked to is AC/DC so its probably the the forward slash causing the issue.

Try ACDC as the artist. That should work.

[Merwenus]

Well, it finds the album, but does not download the thubmnail (it does when I do 1by1, but it stops working when I try to do mass download)
Check timestamps, it tries to download thumbs too fast and I guess I get banned, because after this the 1by1 download does not work anymore.

Log file:

Code:

15:58:51.940 T:7272 WARNING: CRenderManager::WaitForBuffer - timeout waiting for buffer
16:17:10.864 T:1632 WARNING: Previous line repeats 22 times.
16:17:10.864 T:1632   ERROR: XFILE::CDirectory::GetDirectory - Error getting
16:17:11.068 T:7272   ERROR: Previous line repeats 13 times.
16:17:11.068 T:7272 WARNING: CRenderManager::WaitForBuffer - timeout waiting for buffer
16:17:26.800 T:3456 WARNING: Previous line repeats 52 times.
16:17:26.800 T:3456   ERROR: ADDON::CScraper::Run: Unable to parse web site
16:17:28.913 T:7272 WARNING: CRenderManager::WaitForBuffer - timeout waiting for buffer
16:18:01.466 T:6832 WARNING: Previous line repeats 100 times.
16:18:01.466 T:6832 WARNING: CScraperUrl::Get: Can't find precise charset for HTML "http://www.theaudiodb.com/api/v1/json/58424d43204d6564696120/album-mb.php?i=", using "UTF-8" as fallback
16:18:01.466 T:6832   ERROR: ADDON::CScraper::Run: Unable to parse web site
16:18:01.507 T:6832 WARNING: CScraperUrl::Get: Can't find precise charset for HTML "http://www.theaudiodb.com/api/v1/json/58424d43204d6564696120/album-mb.php?i=", using "UTF-8" as fallback
16:18:01.507 T:6832   ERROR: ADDON::CScraper::Run: Unable to parse web site
16:18:01.547 T:6832 WARNING: CScraperUrl::Get: Can't find precise charset for HTML "http://www.theaudiodb.com/api/v1/json/58424d43204d6564696120/album-mb.php?i=", using "UTF-8" as fallback
16:18:01.547 T:6832   ERROR: ADDON::CScraper::Run: Unable to parse web site
16:18:01.587 T:6832 WARNING: CScraperUrl::Get: Can't find precise charset for HTML "http://www.theaudiodb.com/api/v1/json/58424d43204d6564696120/album-mb.php?i=", using "UTF-8" as fallback
16:18:01.587 T:6832   ERROR: ADDON::CScraper::Run: Unable to parse web site
16:18:01.629 T:7272 WARNING: CRenderManager::WaitForBuffer - timeout waiting for buffer

[docwra]

Yep that log shows that the scraper is trying to request the TADB api with a NULL musicbrainz Album ID.

That probably means the lookup on the musicbrainz web service failed due to the flood limit.

[DaveBlake]

Yep that log shows that the scraper is trying to request the TADB api with a NULL musicbrainz Album ID.

That probably means the lookup on the musicbrainz web service failed due to the flood limit.

And therefore if you tagged with Musicbrainz ID tags you would not have this issue.

The skin.helper.service also does it's own Muisicbrainz lookup calls, I don't think that has any wait in it, so that is more likely to have caused the flood limit to be hit as the Universal scraper. What skin do you use @Merwenus? Do you have the skin.helper.service enabled?

[docwra]

And therefore if you tagged with Musicbrainz ID tags you would not have this issue.

Yes I would hope so But is it confirmed the scraper uses the MBID tag if present?

I remember from a while back it only looked up the name whatever the tags on the MusicBrainz server.

[meowmoo]

And therefore if you tagged with Musicbrainz ID tags you would not have this issue.

Yes I would hope so But is it confirmed the scraper uses the MBID tag if present?

I remember from a while back it only looked up the name whatever the tags on the MusicBrainz server.

I also think it did not read the tag.
When I rescrape an artist/album with universal scraper I don't get any information or art, but when I switch to theaudiodb scraper art and info shows up.
All my mp3s have a MBID

Haven't take a look at the log, so just guessing

[Merwenus]

And therefore if you tagged with Musicbrainz ID tags you would not have this issue.

The skin.helper.service also does it's own Muisicbrainz lookup calls, I don't think that has any wait in it, so that is more likely to have caused the flood limit to be hit as the Universal scraper. What skin do you use @Merwenus? Do you have the skin.helper.service enabled?

Amber skin and yes it has skin.helper.service enabled. What can I do to avoid this error?

[docwra]

You can use a musicbrainz mirror server in the scraper settings

http://musicbrainz-mirror.eu:5000/

Is the only one live at the moment but its a bit overloaded.

[cliffx]

I don't know if this had been answered throughout the thread, I haven't read it all, but if MB is presenting issues, can we just use other sources likes TheAudioDB instead?

I reinstalled kodi and I'm missing a lot of artist thumbs from the library.

[DaveBlake]

The Universal Scraper for Music Albums (the subject of this thread) is based upon identifying the album using Musicbrainz, and so access to their servers is essential.

To outline how it works: If music files are tagged with Musicbrainz Album Id (release id), then idenitifaction is obvious, otherwise it does a lookup on album and artist. If a match is found (and 1st release will be taken if there are many) then it fetches artist credits and track data, which if you have "Prefer online info" enabled could be used to sync your lib with MB. Then it uses the release group id (something we don't currently fetch from tag but could) returned by MB to access TADB, and fanart.tv for other info and art. If present it uses any allmusic url returned from MB (they have links) to access data from allmusic.com

However could there be other music scrapers, that do not use MB, just up to someone to write them (in Python now, the scraper API has been extened to allow that). I think there is a TADB sepcific one, but I don't know if that uses just album title and artist, or has a quiet MB lookup tucked inside. But TADB also runs out of server resources too.

I think we could improve things by reducing the requests Kodi makes to MB (or any other server). For example if it had been easy (i.e. automatically happens) to store links to those missing thumbnails of yours, Kodi would not be trying to find them all again. Likewise we could store the release and release-group IDs. I am working on both, slowly because the scraper "regexp parsing of xml" side of things is a mystery (@olympia can you help?)

Otherwise a donation of more server power to the global comminity would help I guess - throw more hardware at it - but demand can be a bottomless pit.

EDIT: Just to add that IMO using MB to identify albums is a good idea, album title and artist is often not enough but if that is all we have then MB is the place to find it accurately