+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Information Providers (scrapers) (https://forum.kodi.tv/forumdisplay.php?fid=147)
+----- Forum: Music Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=303)
+----- Thread: Release Universal Scraper for Music Artists (/showthread.php?tid=132623)
Re: RE: [Release] Universal Scraper for Music Artists - robertomar - 2013-02-19
(2013-02-19, 17:54)olympia Wrote:(2013-02-19, 17:47)robertomar Wrote: Artist slideshow script still downloads complete music artist bio from last.fmOdds are high it scrapes the site, rather than using the API.
(2013-02-19, 17:47)robertomar Wrote: theaudiodb.com is quite incomplete in my opinion.May I ask you a question? How many artists did you add already?
(2013-02-19, 17:47)robertomar Wrote: ...last.fm is the best music database on line...in fact it is very very far from this
Thanks for the quick reply. I think you mean artist slideshow script doesn't use such API, and it's not possible to get the same info for the scraper using it. If it's so, would it be possible to use the artist bio info downloaded from artist slideshow to complete the rest of info and images downloaded from the scraper anyway? Or, would be there any method to make Xbmc to use that info rather than scraper info into the library?
Recently, I have upgraded to Frodo and I have had to rescan and rescrape all my music library, about 450 artists and near to 100 without bio now, the other 350 with english bio from theaudiodb.com, and only some spanish artists with spanish bio (I'm spanish), while previously in Eden I got 90 % bios in spanish using last.fm scraper option. Also a few artists like Inkeys, Neuman, Habib Koite & Bamada, etc doesn't match anything. On the other hand, artist slideshow script downloads practically everything in spanish from last.fm including those rare groups that doesn't match from theaudiodb.com, and downloads fanarts for artists which the scraper hadn't found. This is why I think last.fm is more complete and it motivates my question.
Thanks in advance
RE: [Release] Universal Scraper for Music Artists - olympia - 2013-02-19
(2013-02-19, 19:13)cw-kid Wrote: Also if for example an Artist has a missing fanart and I find one on Google and download it, where should it be saved? so multiple XBMC clients get the new artwork? Using the option Get Fanart you can use the browse button to add in your own images, but I am unsure what really happens and what is the proper way to handle it.
Why don't you add it to fanart.tv or theaudiodb.com so others can benefit from your finding as well? Then you can re-scrape on any clients.
RE: [Release] Universal Scraper for Music Artists - olympia - 2013-02-19
(2013-02-19, 20:00)robertomar Wrote: Thanks for the quick reply. I think you mean artist slideshow script doesn't use such API, and it's not possible to get the same info for the scraper using it. If it's so, would it be possible to use the artist bio info downloaded from artist slideshow to complete the rest of info and images downloaded from the scraper anyway? Or, would be there any method to make Xbmc to use that info rather than scraper info into the library?
Yes, I mean instead of the official API last.fm provides to share information, the artist slideshow script probably reach out the website itself to scrape the info. This is possible with a scraper as well, but this is something I will never do for a site like last.fm.[/quote]
(2013-02-19, 20:00)robertomar Wrote: Recently, I have upgraded to Frodo and I have had to rescan and rescrape all my music library, about 450 artists and near to 100 without bio now, the other 350 with english bio from theaudiodb.com, and only some spanish artists with spanish bio (I'm spanish), while previously in Eden I got 90 % bios in spanish using last.fm scraper option. Also a few artists like Inkeys, Neuman, Habib Koite & Bamada, etc doesn't match anything. On the other hand, artist slideshow script downloads practically everything in spanish from last.fm including those rare groups that doesn't match from theaudiodb.com, and downloads fanarts for artists which the scraper hadn't found. This is why I think last.fm is more complete and it motivates my question.
LOL, theaudiodb.com is a community maintained database. I was asking you you many artists or bio did you personally add to the database to make it better? Complaining without contribution is...
... and adding 100 bio doesn't take a lot of time. 450 would be indeed more time, but I live with the assumption here that this is your hobby as for all of us, so spending some time on it shouldn't be a blocker while you are giving back to the community as well instead of just taking everything for free.
RE: [Release] Universal Scraper for Music Artists - cw-kid - 2013-02-19
(2013-02-19, 22:00)olympia Wrote:(2013-02-19, 19:13)cw-kid Wrote: Also if for example an Artist has a missing fanart and I find one on Google and download it, where should it be saved? so multiple XBMC clients get the new artwork? Using the option Get Fanart you can use the browse button to add in your own images, but I am unsure what really happens and what is the proper way to handle it.
Why don't you add it to fanart.tv or theaudiodb.com so others can benefit from your finding as well? Then you can re-scrape on any clients.
Good idea I am happy to do that and help contribute.
Was also just trying to learn more about where these images get stored on my new system.
Thanks
RE: [Release] Universal Scraper for Music Artists - cw-kid - 2013-02-19
Looking on Fanart.tv for the artist Armand Van Helden there is a background image available
http://fanart.tv/artist/b09b5127-c62e-4bb2-b790-1e4aa18749ed/helden-van-armand/
Looking on Theaudiodb.com there is no fanart available
http://theaudiodb.com/artist/Armand-van-Helden
Looking in the Universal scraper settings I have both of these sites enabled along wit HTBackdrops, Last.fm and allmusic. Yet when I refresh this artist I get no background image only a thumb image.
Thanks
I just contributed a background image for the Armand Van Helden artist on theaudiodb.com website, I refreshed the artist again in my XBMC and now its showing the image I just contributed. But I am wondering why didn't it use the already available image at the fanart.tv site?
Many thanks for your time and this excellent scraper.
RE: [Release] Universal Scraper for Music Artists - cw-kid - 2013-02-19
What to do in this situation?
Neither fanart.tv or theaudiodb.com has a background image for artist Aswad. In fact there are no search results for this artist on theaudiodb.com they only exist on fanart.tv, I have a 1280x720p background image of Aswad but I cannot contribute to fanart.tv it says it has to be 1920 x 1080 size. (You can upload 1280 x 720 images to theaudiodb.com)
So this kinda leads back to my question about using the browse button option in XBMC and adding my own images and where to save them for multiple XBMC PCs to be able to use them. Once I have added the image into XBMC, where in the system do they get stored? and can I then delete the image from my desktop etc? Or do I need to permanently keep the image on my server in the artists album folder(s)?
Thanks
RE: [Release] Universal Scraper for Music Artists - cw-kid - 2013-02-20
I used the browse button and added the background image against the Aswad artist. However looking on a second XBMC PC the background image is not showing on that PC like it is the first one.
Thanks
EDIT:
I found my Aswad background image on the first PC, it has been cached in to the folder:
C:\Users\Stuart\AppData\Roaming\XBMC\userdata\Thumbnails\e
So if you have multiple XBMC PCs, you have to upload the image to each of them ? Using the browse button in the Artist Information area ?
RE: [Release] Universal Scraper for Music Artists - User 99988 - 2013-02-20
(2013-02-18, 13:52)soopersonic Wrote: I'm seeing this issue too, randomly and for always different artists. The follow-ups of post #76 suggest that this is because of XBMC's own scraping speed. I've checked the advancedsettings.xml wiki page but couldn't see any way to slow XBMC down a little.
I can't think of any way to fix this without manually going through the artists and pressing "i" and selecting the first search result (as, in my case, the first one is always the right one). This post by olympia seems to second that.
Edit: I just thought about setting a bandwidth limit in the system menu. Maybe this slows down the whole scraping process enough to not overload the scraped site(s)? I will try this as soon as I can - unless someone tells me I'm completely on the wrong track
Just a quick heads-up for everything who might be interested: Setting the bandwidth limitation did not really help. To me it seems the problem with curl errors for certain artists arises from many previously made API-queries. In my situation, I have an audiobook with many subfolders that all trigger an unsuccessful (and thus very fast) query. After some tests during the lasts days it got apparent that many times the curl errors started after this audiobook.
The solution I used so I didn't have to do it all manually: Add the artist folders step by step. So I first added A-F then clicked "Scan item to library" for my source. After that I added my artists to the source folder in batches of 4-6 starting letters at a time and clicked "update library" each time. This way I didn't have any curl errors and all artists got added perfectly. To be on the safe side, wait a few minutes between the library updates.
RE: [Release] Universal Scraper for Music Artists - ronie - 2013-02-20
(2013-02-19, 22:12)olympia Wrote:(2013-02-19, 20:00)robertomar Wrote: Thanks for the quick reply. I think you mean artist slideshow script doesn't use such API, and it's not possible to get the same info for the scraper using it. If it's so, would it be possible to use the artist bio info downloaded from artist slideshow to complete the rest of info and images downloaded from the scraper anyway? Or, would be there any method to make Xbmc to use that info rather than scraper info into the library?
Yes, I mean instead of the official API last.fm provides to share information, the artist slideshow script probably reach out the website itself to scrape the info. This is possible with a scraper as well, but this is something I will never do for a site like last.fm.
the artist slideshow addon uses the same api as the universal scraper
and thus can't provide a full bios anymore as well.
scraping the website to bypass the api limitation is bad imo.
RE: [Release] Universal Scraper for Music Artists - olympia - 2013-02-20
(2013-02-19, 23:45)cw-kid Wrote: What to do in this situation?
Neither fanart.tv or theaudiodb.com has a background image for artist Aswad. In fact there are no search results for this artist on theaudiodb.com they only exist on fanart.tv, I have a 1280x720p background image of Aswad but I cannot contribute to fanart.tv it says it has to be 1920 x 1080 size. (You can upload 1280 x 720 images to theaudiodb.com)
Thanks
In this case, please add the artist to theaudiodb.com. There is an add botton at the bottom of the page.
RE: [Release] Universal Scraper for Music Artists - coopsta133 - 2013-02-21
I guess Olympia is ignoring the issue of scanning artist using album scraper randomly. hmmm.
The problem went away btw by uninstalling XBMC and reinstalling it. But that's hardly a fix.
RE: [Release] Universal Scraper for Music Artists - floepie - 2013-02-21
Thanks for the scraper. Just a quick couple questions!
1. Is this the default scraper when one first DL's XMBC?
2. I was scratching my head wondering why I was never getting artist art until I realized that "artists.metadata.allmusic.com" was selected in the settings. I changed this to Universal Scraper (in addition to the albums setting). After having changed this setting, in the library music view, I right-clicked on 'artists' and selcted 'change scraper'. Selected Universal Scraper after selecting default settings for all its toggles. I was asked to re-scan using the plug-in. After having scanned for some time, I accidentally terminated the scan, so only about a third of the library had successfully acquired artist art.
How the heck do I re-start this plugin so that it continues to check the other two thirds of my collection? I've tried going in and using the "change scraper". I've tried clean library. I even removed my entire library and re-added it fresh. But now, my entire collection is ignored by the Universal Scraper plugin. How can I force this plugin to recheck the entire collection add artist art?
RE: [Release] Universal Scraper for Music Artists - n1md4 - 2013-02-23
(2013-02-21, 17:35)floepie Wrote: Thanks for the scraper. Just a quick couple questions!
1. Is this the default scraper when one first DL's XMBC?
It's an addon, it might be added by default.
(2013-02-21, 17:35)floepie Wrote: How the heck do I re-start this plugin so that it continues to check the other two thirds of my collection? I've tried going in and using the "change scraper". I've tried clean library. I even removed my entire library and re-added it fresh. But now, my entire collection is ignored by the Universal Scraper plugin. How can I force this plugin to recheck the entire collection add artist art?
Are you sure the artists are available on theaudiodb.com? Assuming you set that.
RE: [Release] Universal Scraper for Music Artists - floepie - 2013-02-23
(2013-02-23, 01:43)n1md4 Wrote: Are you sure the artists are available on theaudiodb.com? Assuming you set that.
Yeah, they are. Only the first third of the artists acquired art before I accidentally terminated the scan. I can't for the life of me figure out how to restart it. So, I simply removed my entire music collection from the library and re-added and rescanned the collection back into the library. Now, I have absolutely no artist art. How the heck do you start this plugin?? There's no UI that I can see besides 'change scraper' when I right-click on the "artists" menu entry. Thanks for any pointers!
RE: [Release] Universal Scraper for Music Artists - PatrickVogeli - 2013-02-23
Hi there,
just a very quick question: am I right that, currently, this addon only scrapes biography info from TheAudioDB??
I know that allmusic stopped working because of copyright issues, but what about last.fm? Has it stopped working too?
EDIT: I forgot, in the addon you can put allmusic as the source to scrape Styles and Moods info, by I don't get those neither. I suppose allmusic is blocking everything?
EDIT 2: ok, got it. Last+fm did some changes in the api, that's because you don't use it anymore. It's depressing to see how much XBMC has improved in 2 years and how the scraper thing has gone downwards because the sites not permiting the scraping anymore.