Release Universal Scraper for Music Albums - Printable Version +- Kodi Community Forum (https://forum.kodi.tv) +-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33) +--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27) +---- Forum: Information Providers (scrapers) (https://forum.kodi.tv/forumdisplay.php?fid=147) +----- Forum: Music Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=303) +----- Thread: Release Universal Scraper for Music Albums (/showthread.php?tid=133547) |
RE: [Release] Universal Scraper for Music Albums - Martijn - 2012-06-21 (2012-06-21, 13:26)Midnight Tboy Wrote: sorry....just to answer my own question I recommend: MusicBrainz Picard TagScanner RE: [Release] Universal Scraper for Music Albums - Midnight Tboy - 2012-06-21 Thanks - trying out Picard right now - seems to mess some folders up, like trying to seperate soundtrack albums into about half a dozen different albums, but will play til get it right Will try TagScanner out next RE: [Release] Universal Scraper for Music Albums - Martijn - 2012-06-21 (2012-06-21, 14:02)Midnight Tboy Wrote: Thanks - trying out Picard right now - seems to mess some folders up, like trying to seperate soundtrack albums into about half a dozen different albums, but will play til get it right Drag one album in. Select the files and click on "cluster". This will make sure it searches for an album that contains all of them RE: [Release] Universal Scraper for Music Albums - Midnight Tboy - 2012-06-21 thanks - though when I select Cluster, it seems to do nothing. example pain album 10 Things I Hate About You - OST all files are stupidly called 10 Things I Hate About You - OST - t01 10 Things I Hate About You - OST - t02, etc with no id3 tag info in. I drag them all in, select Cluster but they still stay under "Unmatched Files", as if the button does nothing Then when I "scan" It splits them into 8 albums, 6 of them being in the correct album. Fun and games Not looking forward to sorting this for hundreds of albums lol.....argh anyways, sorry for going off topic RE: [Release] Universal Scraper for Music Albums - McB83 - 2012-06-22 I have a problem with scraping the album rating from allmusic.com. Every album gets a rating of 2 stars, however the album rating on allmusic.com is higher. For example this album has 3.5 stars, and this one has 4 stars. RE: [Release] Universal Scraper for Music Albums - Plecebo - 2012-06-23 (2012-06-10, 16:24)night199uk Wrote: hey olympia. I had a similar issue with other services that I run and the solution was to setup a local musicbrainz mirror in my network. That way you can make several thousand queries a minute without hitting any rate limit. It would be great if there was an easy way to set musicbrainz mirror settings, host and port are the usual options, in the Universal Scraper options? RE: [Release] Universal Scraper for Music Albums - Martijn - 2012-06-24 (2012-06-21, 15:11)Midnight Tboy Wrote: I drag them all in, select Cluster but they still stay under "Unmatched Files", as if the button does nothing You should hit "search" and not "scan". Scan will use the audio fingerprint to match the audio files RE: [Release] Universal Scraper for Music Albums - Zippolighter - 2012-06-25 Hi, I see a problem with the scraper. I try to scrape Maroon 5 - Overexposed to the db, but the title list shows only one song Nr. 2 with titel and runtime. All other titles are missing. If I try the url that use in the log, I get the complete title list, the runtime is missing on all songs. [code]05:16:11 T:4964 DEBUG: ------ Window Init (DialogProgress.xml) ------ 05:16:11 T:4816 DEBUG: Thread MUSIC_GRABBER::CMusicInfoScraper start, auto delete: 0 05:16:11 T:4816 DEBUG: ADDON::CScraper::FindAlbum: Searching for 'Maroon 5 - Overexposed' using Universal Album Scraper scraper (path: 'C:\Users\Andreas\AppData\Roaming\XBMC\addons\metadata.album.universal', content: 'albums', version: '1.1.1') 05:16:11 T:4816 DEBUG: scraper: CreateAlbumSearchUrl returned <url>http://search.musicbrainz.org/ws/2/release/?fmt=xml&query=release:"Overexposed"%20AND%20artist:"Maroon%205"</url> 05:16:11 T:4816 DEBUG: FileCurl::Open(002BCB30) http://search.musicbrainz.org/ws/2/release/?fmt=xml&query=release:"Overexposed"%20AND%20artist:"Maroon%205" 05:16:11 T:4816 INFO: XCURL:llLibCurlGlobal::easy_aquire - Created session to http://search.musicbrainz.org 05:16:12 T:4816 ERROR: PCRE: missing ). Compilation failed at offset 933 in expression 'id="([^"]*)"><title>([^<]*)</title><status>Official</status><text-representation><language>[^<]*</language><script>[^<]*</script></text-representation><artist-credit><name-credit(?:>)*(?:\sjoinphrase="[^"]*">)*(?:<name>[^<]*</name)*(?:>)*<artist\sid="[^"]*"><name>([^<]*)</name>(?:<sort-name>[^<]*</sort-name>)*(?:<disambiguation>[^<]*</disambiguation>)*(?:<alias-list>.*?</alias-list>)*</artist></name-credit>(?:<name-credit(?:>)*(?:\sjoinphrase="[^"]*">)*(?:<name>[^<]*</name)*(?:>)*<artist\sid="[^"]*"><name>[^<]*</name>(?:<sort-name>[^<]*</sort-name>)*(?:<disambiguation>[^<]*</disambiguation>)*(?:<alias-list>.*?</alias-list>)*</artist></name-credit>)*</artist-credit><release-group(?:\stype="Album"\sid="[^"]*">(?:<primary-type>[^<]*</primary-type>)*(?:<secondary-type-list><secondary-type>[^<]*</secondary-type></secondary-type-list>)*</release-group><date>(\d{4})[^<]*</date>(?:<country>)*([^<]*)?.*?<track-list\scount="(\d+)' 05:16:12 T:4816 DEBUG: scraper: GetAlbumSearchResults returned <results><entity><year>XW-2012-T#17</year><artist>Maroon 5</artist><title>Overexposed</title><url cache="mb-c870093e-5d68-4842-a663-291701678229-album.xml">http://musicbrainz.org/ws/2/release/c870093e-5d68-4842-a663-291701678229?inc=recordings+release-groups+artists+labels+ratings</url></entity><entity><year>GB-2012-T#13</year><artist>Maroon 5</artist><title>Overexposed</title><url cache="mb-f9aaa857-7567-4dc1-afca-086601e1db95-album.xml">http://musicbrainz.org/ws/2/release/f9aaa857-7567-4dc1-afca-086601e1db95?inc=recordings+release-groups+artists+labels+ratings</url></entity></results> 05:16:12 T:4816 DEBUG: Thread MUSIC_GRABBER::CMusicInfoScraper 4816 terminating 05:16:12 T:4692 DEBUG: Thread MUSIC_GRABBER::CMusicInfoScraper start, auto delete: 0 05:16:12 T:4692 DEBUG: ADDON::CScraper::GetAlbumDetails: Reading 'http://musicbrainz.org/ws/2/release/c870093e-5d68-4842-a663-291701678229?inc=recordings+release-groups+artists+labels+ratings' using Universal Album Scraper scraper (file: 'C:\Users\Andreas\AppData\Roaming\XBMC\addons\metadata.album.universal', content: 'albums', version: '1.1.1') 05:16:12 T:4692 DEBUG: FileCurl::Open(002BCB30) http://musicbrainz.org/ws/2/release/c870093e-5d68-4842-a663-291701678229?inc=recordings+release-groups+artists+labels+ratings 05:16:13 T:4692 DEBUG: scraper: GetAlbumDetails returned <details><chain function="GetMBAlbumTitleByMBID">c870093e-5d68-4842-a663-291701678229</chain><chain function="GetMBAlbumArtistByMBID">c870093e-5d68-4842-a663-291701678229</chain><chain function="GetMBAlbumLabelByMBID">c870093e-5d68-4842-a663-291701678229</chain><chain function="GetMBAlbumDateByMBID">c870093e-5d68-4842-a663-291701678229</chain><chain function="GetMBAlbumTracksByMBID">c870093e-5d68-4842-a663-291701678229</chain><chain function="GetLastFMAlbumReviewByAlbumAndArtist">Overexposed::Maroon 5::Overexposed::Maroon 5</chain><chain function="GetMBAlbumRatingByMBID">c870093e-5d68-4842-a663-291701678229</chain><url function="GetAMGAlbumStyles" cache="mb-ed3c55b6-e49d-4d4e-a338-62b823b1e2d4-rg.xml">http://musicbrainz.org/ws/2/release-group/ed3c55b6-e49d-4d4e-a338-62b823b1e2d4?inc=url-rels</url><url function="GetAMGAlbumMoods" cache="mb-ed3c55b6-e49d-4d4e-a338-62b823b1e2d4-rg.xml">http://musicbrainz.org/ws/2/release-group/ed3c55b6-e49d-4d4e-a338-62b823b1e2d4?inc=url-rels</url><url function="GetAMGAlbumThemes" cache="mb-ed3c55b6-e49d-4d4e-a338-62b823b1e2d4-rg.xml">http://musicbrainz.org/ws/2/release-group/ed3c55b6-e49d-4d4e-a338-62b823b1e2d4?inc=url-rels</url><chain function="GetFanartTvAlbumThumbsByMBID">ed3c55b6-e49d-4d4e-a338-62b823b1e2d4</chain><url function="GetAMGAlbumThumbs" cache="mb-ed3c55b6-e49d-4d4e-a338-62b823b1e2d4-rg.xml">http://musicbrainz.org/ws/2/release-group/ed3c55b6-e49d-4d4e-a338-62b823b1e2d4?inc=url-rels</url><chain function="GetLastFMAlbumThumbs">Overexposed::Maroon 5::Overexposed::Maroon 5</chain></details> 05:16:13 T:4692 DEBUG: scraper: GetMBAlbumTitleByMBID returned <details><url function="ParseMBAlbumTitle" cache="mb-c870093e-5d68-4842-a663-291701678229-album.xml">http://musicbrainz.org/ws/2/release/c870093e-5d68-4842-a663-291701678229?inc=recordings+release-groups+artists+labels+ratings</url></details> 05:16:13 T:4692 DEBUG: scraper: ParseMBAlbumTitle returned <details><title>Overexposed</title></details> 05:16:13 T:4692 DEBUG: scraper: GetMBAlbumArtistByMBID returned <details><url function="ParseMBAlbumArtist" cache="mb-c870093e-5d68-4842-a663-291701678229-album.xml">http://musicbrainz.org/ws/2/release/c870093e-5d68-4842-a663-291701678229?inc=recordings+release-groups+artists+labels+ratings</url></details> 05:16:13 T:4692 DEBUG: scraper: ParseMBAlbumArtist returned <details><artist>Maroon 5</artist></details> 05:16:13 T:4692 DEBUG: scraper: GetMBAlbumLabelByMBID returned <details><url function="ParseMBAlbumLabel" cache="mb-c870093e-5d68-4842-a663-291701678229-album.xml">http://musicbrainz.org/ws/2/release/c870093e-5d68-4842-a663-291701678229?inc=recordings+release-groups+artists+labels+ratings</url></details> 05:16:13 T:4692 ERROR: ADDON::CScraper::Run: Unable to parse web site 05:16:13 T:4692 DEBUG: scraper: GetMBAlbumDateByMBID returned <details><url function="ParseMBAlbumDate" cache="mb-c870093e-5d68-4842-a663-291701678229-album.xml">http://musicbrainz.org/ws/2/release/c870093e-5d68-4842-a663-291701678229?inc=recordings+release-groups+artists+labels+ratings</url></details> 05:16:13 T:4692 DEBUG: scraper: ParseMBAlbumDate returned <details><releasedate>2012-06-25</releasedate><year>2012</year></details> 05:16:13 T:4692 DEBUG: scraper: GetMBAlbumTracksByMBID returned <details><url function="ParseMBAlbumTracks" cache="mb-c870093e-5d68-4842-a663-291701678229-album.xml">http://musicbrainz.org/ws/2/release/c870093e-5d68-4842-a663-291701678229?inc=recordings+release-groups+artists+labels+ratings</url></details> 05:16:13 T:4692 DEBUG: scraper: ParseMBAlbumTracks returned <details><track><position>2</position><title>Payphone</title><duration>231</duration></track></details> 05:16:13 T:4692 DEBUG: scraper: GetLastFMAlbumReviewByAlbumAndArtist returned <details><url function="ParseLastFMAlbumReview" cache="lastfmalbum-Maroon 5-Overexposed.xml">http://ws.audioscrobbler.com/2.0/?method=album.getinfo&artist=Maroon%205&album=Overexposed&api_key=71e468a84c1f40d4991ddccc46e40f1b&lang=en</url></details> 05:16:13 T:4692 DEBUG: FileCurl::Open(002BCB30) http://ws.audioscrobbler.com/2.0/?method=album.getinfo&artist=Maroon%205&album=Overexposed&api_key=71e468a84c1f40d4991ddccc46e40f1b&lang=en 05:16:13 T:4692 ERROR: ADDON::CScraper::Run: Unable to parse web site 05:16:13 T:4692 DEBUG: scraper: GetMBAlbumRatingByMBID returned <details><url function="ParseMBAlbumRating" cache="mb-c870093e-5d68-4842-a663-291701678229-album.xml">http://musicbrainz.org/ws/2/release/c870093e-5d68-4842-a663-291701678229?inc=recordings+release-groups+artists+labels+ratings</url></details> 05:16:13 T:4692 DEBUG: scraper: ParseMBAlbumRating returned <details><rating>5</rating></details> 05:16:13 T:4692 DEBUG: FileCurl::Open(002BCB30) http://musicbrainz.org/ws/2/release-group/ed3c55b6-e49d-4d4e-a338-62b823b1e2d4?inc=url-rels 05:16:14 T:4692 DEBUG: scraper: GetAMGAlbumStyles returned <details><chain function="GetAMGAlbumStylesByAMGID">mw0002358095</chain></details> 05:16:14 T:4692 DEBUG: scraper: GetAMGAlbumStylesByAMGID returned <details><url cache="am-mw0002358095-album.html" function="ParseAMGAlbumStyles">http://www.allmusic.com/album/mw0002358095</url></details> 05:16:14 T:4692 DEBUG: FileCurl::Open(002BCB30) http://www.allmusic.com/album/mw0002358095 05:16:16 T:4692 DEBUG: scraper: ParseAMGAlbumStyles returned <details><style>Adult Alternative Pop/Rock</style><style>Alternative/Indie Rock</style><style>Contemporary Pop/Rock</style><style>Pop</style></details> 05:16:16 T:4692 DEBUG: scraper: GetAMGAlbumMoods returned <details><chain function="GetAMGAlbumMoodsByAMGID">mw0002358095</chain></details> 05:16:16 T:4692 DEBUG: scraper: GetAMGAlbumMoodsByAMGID returned <details><url cache="am-mw0002358095-album.html" function="ParseAMGAlbumMoods">http://www.allmusic.com/album/mw0002358095</url></details> Thanks Zippolighter RE: [Release] Universal Scraper for Music Albums - Zippolighter - 2012-06-25 I found another issue, I scrape Die Toten Hosen - Ballast Der Republik. The scraper add a 2Disc Variante to the db, that was fine. But of CD 2 the titles 3 and 8 are missing. Both are available on Musicbrainz Webside Zippolighter RE: [Release] Universal Scraper for Music Albums - solidsatras - 2012-06-25 @Zippolighter (2012-06-24, 15:06)Martijn Wrote: How to post Debug Log?: Use XBMC log uploader RE: [Release] Universal Scraper for Music Albums - olympia - 2012-06-25 (2012-06-25, 07:43)solidsatras Wrote: @Zippolighter Strange that he still does this given his 326 posts.... RE: [Release] Universal Scraper for Music Albums - olympia - 2012-06-25 (2012-06-25, 05:27)Zippolighter Wrote: Hi, I see a problem with the scraper.Fixed in v1.1.2 (2012-06-25, 07:07)Zippolighter Wrote: I found another issue,Fixed in v1.1.3 (however please note, that track duration won't be scraped for such tracks) RE: [Release] Universal Scraper for Music Albums - McB83 - 2012-06-25 (2012-06-22, 23:06)McB83 Wrote: I have a problem with scraping the album rating from allmusic.com. Every album gets a rating of 2 stars, however the album rating on allmusic.com is higher. For example this album has 3.5 stars, and this one has 4 stars. I've inspected the debug logs and the rating is scraped correctly from allmusic.com. However when I export my music library I notice the following. Albums scraped with the deprecated allmusic.com scraper have a rating between 1-10. Albums scraped with the new universal scraper have a rating between 1-5. So my assumption is that XBMC works with a different rating then allmusic.com. The scraped rating from allmusic.com must be multiplied with 2 to get the correct rating in XBMC. I can provide debug logs if necessary. RE: [Release] Universal Scraper for Music Albums - olympia - 2012-06-25 (2012-06-25, 11:14)McB83 Wrote:(2012-06-22, 23:06)McB83 Wrote: I have a problem with scraping the album rating from allmusic.com. Every album gets a rating of 2 stars, however the album rating on allmusic.com is higher. For example this album has 3.5 stars, and this one has 4 stars.So my assumption is that XBMC works with a different rating then allmusic.com. The scraped rating from allmusic.com must be multiplied with 2 to get the correct rating in XBMC. I can provide debug logs if necessary. ...or allmusic.com changed their rating system during the site re-design. In any case, something is broken in XBMC yes: as you've noticed in the log yourself the scraper scrapes the [1-5] rating correctly, but XBMC even divide this by 2 (e.g a scraped 4 gets 2 at the end). RE: [Release] Universal Scraper for Music Albums - Zippolighter - 2012-06-25 Thanks olympia for the very fast solution Zippolighter |