Kodi Community Forum
Release Universal Scraper for Music Artists - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Information Providers (scrapers) (https://forum.kodi.tv/forumdisplay.php?fid=147)
+----- Forum: Music Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=303)
+----- Thread: Release Universal Scraper for Music Artists (/showthread.php?tid=132623)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42

RE: [Release] Universal Scraper for Music Artists - overbyrn - 2012-12-06

(2012-11-12, 23:08)Baswazz Wrote: I have problems to get the right artist information of Pink

Had the same problem when testing out current music scraping abilities of latest Frodo nightlies with universal scraper.

Fix is to put double quotes around the name. eg. when universal scraper prompts you to pick an entry, do a manual search and put in "P!nk". It should return the artist as the top entry. Did for me.


RE: [Release] Universal Scraper for Music Artists - mikari - 2013-01-06

Hi,

Thx for that great scraper, but i got this old issue where the scraper keeps asking for the artists name endlessly.
I saw that several times, but found no final conclusion, hint or fix.

Here is what i did:

Win7, x64, Frodo RC3, Standard Confluence Skin, Album Artist Scraper 2.4.2
  • Clean XBMC install, empty DB, initial scan of source
  • Directory structure is: source/artist/albums/tracks
  • In system->music->library universal scraper for artists and albums are set.
  • added new source and let it scan to the library
  • scraping worked fine, just a few wrong or missing artists,...so far so good.
  • afterwards i wanted to scan some (not all) albums individually via <i>nfo and refresh but that didnt work at first.
  • in order to make album scan working i had to assign the Universal Scraper to "Albums" in the Library view, nothing was set there...ok,...but now it comes....
  • when i wanted to fix the missing or wrong artists via <i>nfo ->Refresh, i always get the endless loop asking for the artists name.
  • no chance to refresh any other artists anymore, also the ones that actually worked before i assigned the album scraper to the albums node.
  • checking the debug log brings the below result (relevant excerpt, link to full debug.log below)
Code:
06:26:36 T:1592   DEBUG: ADDON::CScraper::FindArtist: Searching for 'Van Halen' using Universal Album Scraper scraper (file: 'C:\Users\mikari\AppData\Roaming\XBMC\addons\metadata.album.universal', content: 'albums', version: '1.6.3')
06:26:36 T:1592   ERROR: CScraperParser::Parse: Could not find scraper function CreateArtistSearchUrl
06:26:36 T:1592   ERROR: ADDON::CScraper::Run: Unable to parse web site
full debug.log: http://pastebin.com/3yd13UJx

So it seems the scraper tries to resolve the artist with the Album Scraper (?!) after assigning the Album Scraper to the Albums mode in library view and scraping some albums. When i check the context menu "Change Scraper" on Artists in the Library it shows the Universal Artists Scraper as it should.

Am i doing something wrong or is this a bug? Any hint is appreciated.

cheers,
Michael

Edit: the previously mentioned workaround to assign the Artists Scraper to each individual artists works, but,...well,..this does not seem to be the way it should work.
Edit2: highlighted possible key issue.

RE: [Release] Universal Scraper for Music Artists - clubwerks - 2013-01-14

I don't know if it's a scraper or last.fm problem, but it seems that one or the other doesn't handle apostrophes correctly. When I run the scraper, it doesn't get the biography for Jane's Addiction or Destiny's Child. What's the trick?

RE: [Release] Universal Scraper for Music Artists - jackbozzz - 2013-01-15

(2013-01-06, 07:47)mikari Wrote: Hi,

Thx for that great scraper, but i got this old issue where the scraper keeps asking for the artists name endlessly.
I saw that several times, but found no final conclusion, hint or fix.

Here is what i did:

Win7, x64, Frodo RC3, Standard Confluence Skin, Album Artist Scraper 2.4.2
  • Clean XBMC install, empty DB, initial scan of source
  • Directory structure is: source/artist/albums/tracks
  • In system->music->library universal scraper for artists and albums are set.
  • added new source and let it scan to the library
  • scraping worked fine, just a few wrong or missing artists,...so far so good.
  • afterwards i wanted to scan some (not all) albums individually via <i>nfo and refresh but that didnt work at first.
  • in order to make album scan working i had to assign the Universal Scraper to "Albums" in the Library view, nothing was set there...ok,...but now it comes....
  • when i wanted to fix the missing or wrong artists via <i>nfo ->Refresh, i always get the endless loop asking for the artists name.
  • no chance to refresh any other artists anymore, also the ones that actually worked before i assigned the album scraper to the albums node.
  • checking the debug log brings the below result (relevant excerpt, link to full debug.log below)
Code:
06:26:36 T:1592   DEBUG: ADDON::CScraper::FindArtist: Searching for 'Van Halen' using Universal Album Scraper scraper (file: 'C:\Users\mikari\AppData\Roaming\XBMC\addons\metadata.album.universal', content: 'albums', version: '1.6.3')
06:26:36 T:1592   ERROR: CScraperParser::Parse: Could not find scraper function CreateArtistSearchUrl
06:26:36 T:1592   ERROR: ADDON::CScraper::Run: Unable to parse web site
full debug.log: http://pastebin.com/3yd13UJx

So it seems the scraper tries to resolve the artist with the Album Scraper (?!) after assigning the Album Scraper to the Albums mode in library view and scraping some albums. When i check the context menu "Change Scraper" on Artists in the Library it shows the Universal Artists Scraper as it should.

Am i doing something wrong or is this a bug? Any hint is appreciated.

cheers,
Michael

Edit: the previously mentioned workaround to assign the Artists Scraper to each individual artists works, but,...well,..this does not seem to be the way it should work.
Edit2: highlighted possible key issue.

Exactly the same problem here! I have tried different settings etc. but it seems that the Album and Artist scraper are mixing up. When I use the Artist scraper it recognises the ones I have done manual (which is a lot of work) but than it replaces the artist pic with an album pic. Very frustating.

Anyone a solution?

RE: [Release] Universal Scraper for Music Artists - tarkun - 2013-01-17

Hi there,

I'm scraping my Music Library on Frodo RC3 with version 2.4.2 and while I get thumb and fanart for most artists, the scraper doesn't get BIO at all.
I've found the following lines in the debug file while trying to retrieve info for Amy Winehouse:

Code:
22:33:02 T:7700   DEBUG: scraper: GetLastFMBiographyByMBID returned <details><url function="ParseLastFMBiography" cache="lastfmartistinfo-dfe9a7c4-8cf2-47f4-9dcb-d233c2b86ec3.xml">http://ws.audioscrobbler.com/2.0/?method=artist.getinfo&amp;mbid=dfe9a7c4-8cf2-47f4-9dcb-d233c2b86ec3&amp;api_key=71e468a84c1f40d4991ddccc46e40f1b&lang=en</url></details>
22:33:02 T:7700   DEBUG: CurlFile::Open(080C9A18) http://ws.audioscrobbler.com/2.0/?method=artist.getinfo&mbid=dfe9a7c4-8cf2-47f4-9dcb-d233c2b86ec3&api_key=71e468a84c1f40d4991ddccc46e40f1b&lang=en
22:33:05 T:7700 WARNING: XFILE::CCurlFile::CReadState::FillBuffer: curl failed with code 22
22:33:05 T:7700   ERROR: CCurlFile::CReadState::Open, didn't get any data from stream.
22:33:05 T:7700   ERROR: ADDON::CScraper::Run: Unable to parse web site

Setting the scraper to query for biography in TheAudioDB.com resolved the problem.
Unfortunately, I mainly listen to progressive rock which is not covered in TheAudioDB.

Any suggestion?

Thanks in advance
Omorzo

RE: [Release] Universal Scraper for Music Artists - tymmy - 2013-01-17

It seams the scraper has stopped getting thumbs from last FM. When I look at the choose thumb (off the information screen) I will see that in this case there are 7 remotes thumbs. If i go to lastfm and click on pictures they mirror the remote thumbs. If I refresh in xbmc and look again only current thumb is there and no remote thumbs. I dont think I am getting any info really from last FM.
RC3 - Frodo

Thanks
Tymmy

RE: [Release] Universal Scraper for Music Artists - roach9 - 2013-01-20

Having same issue, looks like add-on broken.

RE: [Release] Universal Scraper for Music Artists - olympia - 2013-01-20

XBMC's last.fm API account has been suspended by last.fm for unknown reasons. Since this gets clarified, last.fm options has been removed from the scraper.

RE: [Release] Universal Scraper for Music Artists - vinistois - 2013-01-22

this scraper seems to have fallen apart.

Artist info is all broken. I get no artist thumb unless I go through manually and select universal Artist Scraper for each artist. Selecting it as the "artist folder" scraper does nothing. There is zero artist bio, and most matches are wrong. It ends up creating a bunch of artists I don't even have.

My music folder is super organized. Artist / Album (year) / Tracks

All my music is pre-tagged with Musicbrainz Picard.

Also, it seems to be the only scraper that will install on Frodo.

So.... this pretty much breaks xbmc for music, as far as usability goes.

Edit: this is the exact issue:

http://forum.xbmc.org/showthread.php?tid=138454&pid=1280375#pid1280375

RE: [Release] Universal Scraper for Music Artists - n1md4 - 2013-01-22

Hi. I hope not to incite the wrath of Olympia Wink but Universal Scraper newbie here. Your scraper is fantastic, and is an asset to XBMC.

With Last.fm suspending the API account, I thought I'd try this instead. With default settings, around 50% of my music collection is successfully scraped. I'm unsure why some artists fail to scrape.

e.g. Bobbie Gentry

1) I don't have a sub-folder within my music collection for this artist, instead it was an artist from a compilation; there are other artists from similar compilations that have scraped successfully, so it's not the omission of the sub-director that creates a fail case.

2) Site manual search results
i) http://www.theaudiodb.com/search.php : Search Results for 'bobbie gentry'
ii) http://fanart.tv/ : No results found, please try a different search term or input the id in the box above
iii) http://www.allmusic.com/artist/bobbie-gentry-mn0000065397
iv) http://musicbrainz.org/artist/a5106366-6368-4a02-a580-73a3c02abb2f

But, despite the various successful search results there is no returned data.

3) I have read previously a manual search is recommended, but I can't ascertain where to complete this search.



There are a large number of other artists, which may likely return similar results, and likewise fail to scrape. I'm all for giving back to the community too, so if there's manual inputting to be done; to any of the above sites, I'd be more than happy to.

Any assistance here would be greatly appreciated.

RE: [Release] Universal Scraper for Music Artists - olympia - 2013-01-23

Hello n1md4,

Thank you for the nicely raised issue and the willingness for giving back to the community.

Looking at specifically 'Bobbie Gentry':
- In general, the scraper always does the initial search on musicbrainz, then jumping onto the other sites either using the musicbrainz id or the link provided by musicbrainz
- I do think it is clear, that scraping artist bio is currently only possible from theaudiodb.com
- The artist doesn't exist on theaudiodb.com, so scraping from there is not possible
- You will also not be able to get images from fanart.tv as the artist doesn't exist there either
- So the only source for some info can be musicbrainz and allmusic (but no bio from allmusic)
- Information from musicbrainz is limited, but you can scrape discogs and lifespan info just right now
- In case you want additional info from allmusic (except bio) you need to add the allmusic link on musicbrainz
- In case you want the best scraping experience and to serve the community the best as well, then you should add the artist to theaudiodb and fanart.tv as these two sites provide the best seach hits, quality of information and reliability.

RE: [Release] Universal Scraper for Music Artists - Syncopation - 2013-01-23

hey olympia,

hanks so much for this wonderful scraper. Do you think adding info like in post #146 to the first post of this thread would be a good idea? As a user I have to say I think alike n1md4. If info is missing what good is it to solve this locally on my machine if I could as well update an existing database entry. But it is very hard to tell which info is coming from where. So if we all as a community want to improve the overall experience and database quality, adding this to the initial post would help a lot and might lower the entry barrier for users to contribute to databases.

Which info is coming form where and if missing where to add it.

Just my 2 cent Smile

RE: [Release] Universal Scraper for Music Artists - curtis-r - 2013-01-23

@olympia,

Thanks for the clear explanation of the issue. Is there any plan to try to get the last.fm xbmc account reactivated? Is there an end-user solution whereby using an individual's account?

RE: [Release] Universal Scraper for Music Artists - olympia - 2013-01-23

(2013-01-23, 22:04)Syncopation Wrote: hey olympia,

hanks so much for this wonderful scraper. Do you think adding info like in post #146 to the first post of this thread would be a good idea? As a user I have to say I think alike n1md4. If info is missing what good is it to solve this locally on my machine if I could as well update an existing database entry. But it is very hard to tell which info is coming from where. So if we all as a community want to improve the overall experience and database quality, adding this to the initial post would help a lot and might lower the entry barrier for users to contribute to databases.

Which info is coming form where and if missing where to add it.

Just my 2 cent Smile

You are totally right. The best would be if someone was "wikifying" all these, but I don't have the time.

RE: [Release] Universal Scraper for Music Artists - olympia - 2013-01-23

(1970-01-01, 02:02)curtis-rI8972586' Wrote: @olympia,

Thanks for the clear explanation of the issue. Is there any plan to try to get the last.fm xbmc account reactivated? Is there an end-user solution whereby using an individual's account?

http://www.last.fm/group/Last.fm+Web+Services/forum/21604/_/2199932

last.fm give seems to give a shit on this or they don't even read their API forums. No other way to get in contact with them.
Some push from your side guys might help, but honestly I don't think so.

It's time to start filling up theaudiodb.com... Adding an artist to this site is very easy and quick. You just need to enter the musicbrainz ID and the site will automatically import a lot of info, then only the gaps need to be filled in.