Kodi Community Forum
Release Universal Scraper for Music Albums - Printable Version

Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Information Providers (scrapers) (https://forum.kodi.tv/forumdisplay.php?fid=147)
+----- Forum: Music Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=303)
----- Thread: Release Universal Scraper for Music Albums (/showthread.php?tid=133547)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34


RE: [Release] Universal Scraper for Music Albums - olympia - 2018-01-13

Great work Zag; cool! Smile

Re ratings the only thing is that you scrape it once, then it remains as is. Also new albums will not get the boost, unless you scrape/ sync regularly.

Edit: I just fixed many things with allmusic scraping in allmusic scraper library update v3.2.0. This scraper was badly broken for almost all things, but the rating. Seems like no one is actually using it... Smile


RE: [Release] Universal Scraper for Music Albums - docwra - 2018-01-17

@olympia Just in case you were possibly going to update the music scrapers soon here is a JSON list of new artwork Wink

Artist
"strArtistThumb": "http://www.theaudiodb.com/images/media/artist/thumb/uxrqxy1347913147.jpg",
"strArtistLogo": "http://www.theaudiodb.com/images/media/artist/logo/urspuv1434553994.png",
"strArtistClearart": "http://media.theaudiodb.com/images/media/artist/clearart/ruyuwv1510827568.png",
"strArtistWideThumb": "http://media.theaudiodb.com/images/media/artist/widethumb/sxqspt1516190718.jpg",
"strArtistFanart": "http://media.theaudiodb.com/images/media/artist/fanart/spvryu1347980801.jpg",
"strArtistFanart2": "http://media.theaudiodb.com/images/media/artist/fanart/uupyxx1342640221.jpg",
"strArtistFanart3": "http://media.theaudiodb.com/images/media/artist/fanart/qstpsp1342640238.jpg",
"strArtistBanner": "http://www.theaudiodb.com/images/media/artist/banner/xuypqw1386331010.jpg",

Album
"strAlbumThumb": "http://www.theaudiodb.com/images/media/album/thumb/xy-4ea02a5208fd0.jpg",
"strAlbumThumbBack": "http://www.theaudiodb.com/images/media/album/thumbback/ttrwtu1429177433.jpg",
"strAlbumCDart": "http://www.theaudiodb.com/images/media/album/cdart/qrwyvp1429179558.png",
"strAlbumSpine": "http://www.theaudiodb.com/images/media/album/spine/vusxtx1430131294.jpg",

And just for the future if @DaveBlake one-day implements track artwork:

Track
"strTrackThumb": "http://www.theaudiodb.com/images/media/track/thumb/xqwrqt1340354430.jpg",
"strMusicVidScreen1": "http://www.theaudiodb.com/images/media/track/mvidscreen/xuvsus1364213274.jpg",
"strMusicVidScreen2": "http://www.theaudiodb.com/images/media/track/mvidscreen/vrxrup1364213280.jpg",
"strMusicVidScreen3": "http://www.theaudiodb.com/images/media/track/mvidscreen/rrptsw1364213287.jpg",



RE: [Release] Universal Scraper for Music Albums - olympia - 2018-01-17

That's cool, however that's not possible as of yet; core needs to be enabled first to receive the feed of new artwork from a scraper addon. That has been done for video, but not for music yet. DaveBlake is looking at this as well.
I only added artist logo to Universal Artist scraper in Leia for Dave to test.

Btw. Is widethumb equal to landscape?


RE: [Release] Universal Scraper for Music Albums - DaveBlake - 2018-01-17

(2018-01-17, 15:01)docwra Wrote: And just for the future if @DaveBlake one-day implements track artwork:
Zag, song artwork is already supported via JSON and skins can access. Smile
But Kodi does not scrape songs, and that is not something I have any plans to add core scraper support for.

EDIT: But that does not mean that someone could not write an addon that fetches song art. Do that many tracks on an album really have different art?


RE: [Release] Universal Scraper for Music Albums - docwra - 2018-01-17

(2018-01-17, 15:11)olympia Wrote: That's cool, however that's not possible as of yet; core needs to be enabled first to receive the feed of new artwork from a scraper addon. That has been done for video, but not for music yet. DaveBlake is looking at this as well.
I only added artist logo to Universal Artist scraper in Leia for Dave to test.

Btw. Is widethumb equal to landscape?
Yes should be exactly the same as "MovieThumb" on fanart.tv

So 1000 x 562
Image

Same aspect ratio as fanart.

Should make for some really nice Artist browse views in skins eventually.


RE: [Release] Universal Scraper for Music Albums - olympia - 2018-01-17

ok, just to agree on a common terminology. Kodi legacy is to call those things as landscape Smile


RE: [Release] Universal Scraper for Music Albums - Epg1 - 2018-01-19

(2018-01-11, 19:51)DaveBlake Wrote:
(2018-01-11, 16:28)Epg1 Wrote: What can I do, except doing a *.nfo file, to solve this problem?
 It is a good example of why using individual files per tack (not cuesheets) and fully tagging with Musicbrainz IDs is a good idea.

The cuesheet standard does not supply a means to specify all Musicbrainz IDs, and Kodi does not process partial Musicbrainz tags (just having the Musicbrainz album ID in the single file is not helping you).  Musicbrainz integration was simply not designed to work with single file and cuesheets. So the alternative to an album.NFO file is to split to individual tracks and tag.  
I see your point, but I've some album which would be better not to split in tracks, i.e. Pink Floyd albums and others.

I've tried to change scraper in Kodi: TheAudioDB scraper recognize the tracklist and, consequently, it is fully shown in the information window in all the case the Universal Album Scraper fails (just a few cases, Universal Album Scaper recognize almost all my *.cue + *.flac | *.ape albums showing proper tracklists). I don't know how theAudioDB  scraper works, but it does the magic trick Smile.

Just a question about Univesal Album Scaper: what is parsed by the scaper? The Cuesheet file, the tags in the single *.flac (or *.ape) or both?
Parsing CATALOG or ISRC (official keywords in cuesheet) in the cuesheet would help?

Anyway, thank you a lot for giving us the Universal Album scraper. Music

Epg1.


RE: [Release] Universal Scraper for Music Albums - olympia - 2018-01-19

The reason why theaudiodb scraper works for you is that there are no editions (versions, etc) on theaudiodb.com.
theaudiodb.com is on releasegroup level, not on release level, and _accidentally_ the tracklist listed on theaudiodb.com for your album releasegroups are the ones you want. So this is really only working out better in your case by accident.

...but honestly, what's wrong with my suggested approach to create an album.nfo file in the given album folder with only the musicbrainz link to the wanted album release inside? Universal Album scraper will get the correct track list then.


RE: [Release] Universal Scraper for Music Albums - Epg1 - 2018-01-19

(2018-01-19, 11:49)olympia Wrote: The reason why theaudiodb scraper works for you is that there are no editions (versions, etc) on theaudiodb.com.
theaudiodb.com is on releasegroup level, not on release level, and _accidentally_ the tracklist listed on theaudiodb.com for your album releasegroups are the ones you want. So this is really only working out better in your case by accident.

...but honestly, what's wrong with my suggested approach to create an album.nfo file in the given album folder with only the musicbrainz link to the wanted album release inside? Universal Album scraper will get the correct track list then.
Oh, there's nothing wrong in the album.nfo approach, and I will follow it for sure Blush

Mine was only a suggestion on a possible improvement in the scraper logic (ability to choose beetween different versions of a same album in the cuesheet+one large taggable file case), but I don't know how demanding is to do at scraper code level.

All in all, I'm really satisfied how Universal Album scraper works.

Thank you for your support Blush

Epg1.


RE: [Release] Universal Scraper for Music Albums - olympia - 2018-01-19

Well, as DaveBlake pointed out, in case you have the mb album release id in the one large taggable file, then it should be a solution as well.


RE: [Release] Universal Scraper for Music Albums - DaveBlake - 2018-01-20

(2018-01-19, 11:31)Epg1 Wrote: I see your point, but I've some album which would be better not to split in tracks, i.e. Pink Floyd albums and others.
Since the player plays gapless, splitting Pink Floyd albums into tracks really isn't an issue. Why not try it and see?

Actually I am at a loss to see what advantages single file per album gives. I guess it could be a way to add silent gaps between tracks, except the player does not process INDEX 00 cue sheet statements so you would not get gaps when played using Kodi.
(2018-01-19, 11:31)Epg1 Wrote: I've tried to change scraper in Kodi: TheAudioDB scraper recognize the tracklist and, consequently, it is fully shown in the information window in all the case the Universal Album Scraper fails (just a few cases, Universal Album Scaper recognize almost all my *.cue + *.flac | *.ape albums showing proper tracklists). I don't know how theAudioDB  scraper works, but it does the magic trick Smile
Display of a scraped tracklist is being deprecated in v18, the album info dialog will instead show the list of songs from that album that you have in your library. Some skins alreday do this.

As Olympia said the TADB scraper looks up release group (not release) by album name and artist, it does not know about the various releases that can have different track listings and just gives the one list of tracks that it has. This may or may not match the release that you have. No magic, just luck that it is what you want.
(2018-01-19, 11:31)Epg1 Wrote: Just a question about Universal Album Scaper: what is parsed by the scaper? The Cuesheet file, the tags in the single *.flac (or *.ape) or both?
Neither.
Unlike video the music scrapers do not scrape files, they fetch additional info for library items - artists or albums. The Universal Album Scraper is given the Musicbrainz release id, if it is known for the album from music file tags or NFO file, and accurately fetches the album info based on that. If Musicbrainz release id is not known then it first looks up Musicbrainz release id based on album name and artist(s), takes the first matching answer (that may or may not be the release you have), and then fetches the album info.

The library entries are created by scanning the limited standard data from the cuesheet file. The existence of a cue sheet takes preference over any tags embedded in the single file itself.
 
(2018-01-19, 11:31)Epg1 Wrote: Parsing CATALOG or ISRC (official keywords in cuesheet) in the cuesheet would help? 
I have not seen anywhere thats says those are official cuesheet keywords, and no the music scanner (nothing to do with scraper) does not process those cue sheet entries. They are not held in the library, and are not keys to any of the online sources AFAIK. So no, it would not help at all.

You have clear choices: use an album.nfo file, or split into separate music files tagged with mbids (my prefered solution).


RE: [Release] Universal Scraper for Music Albums - Epg1 - 2018-01-23

(2018-01-20, 10:32)DaveBlake Wrote: You have clear choices: use an album.nfo file, or split into separate music files tagged with mbids (my prefered solution). 
 I've just tried to follow your advice putting Album.nfo file in the folder in which are the cuesheet and the *.ape file, but it doesn't seem to work.

Here are the details: in a folder named "Gang Of Four - Entertainment!" there are two files: the cuesheet ("Gang Of Four - Entertainment!.cue") and an ape file (Gang Of Four - Entertainment!.ape").

The cuesheet contains 15 tracks:
Code:

REM GENRE Punk
REM DISCID CC0BE00F
REM COMMENT "ExactAudioCopy v0.95b4"
CATALOG 01010003163>0
PERFORMER "Gang of Four"
TITLE "Entertainment!"
FILE "Gang of Four - Entertainment!.ape" WAVE
  TRACK 01 AUDIO
    TITLE "Ether"
    PERFORMER "Gang of Four"
    INDEX 00 00:00:00
    INDEX 01 00:00:33
  TRACK 02 AUDIO
    TITLE "Natural's Not in It"
    PERFORMER "Gang of Four"
    INDEX 00 03:51:50
    INDEX 01 03:51:53
  TRACK 03 AUDIO
    TITLE "Not Great Men"
    PERFORMER "Gang of Four"
    INDEX 00 06:58:33
    INDEX 01 06:58:35
  TRACK 04 AUDIO
    TITLE "Damaged Goods"
    PERFORMER "Gang of Four"
    INDEX 00 10:03:73
    INDEX 01 10:04:00
  TRACK 05 AUDIO
    TITLE "Return the Gift"
    PERFORMER "Gang of Four"
    INDEX 00 13:31:08
    INDEX 01 13:31:15
  TRACK 06 AUDIO
    TITLE "Guns Before Butter"
    PERFORMER "Gang of Four"
    INDEX 00 16:36:08
    INDEX 01 16:36:18
  TRACK 07 AUDIO
    TITLE "I Found That Essence Rare"
    PERFORMER "Gang of Four"
    INDEX 00 20:23:38
    INDEX 01 20:23:40
  TRACK 08 AUDIO
    TITLE "Glass"
    PERFORMER "Gang of Four"
    INDEX 00 23:37:23
    INDEX 01 23:37:25
  TRACK 09 AUDIO
    TITLE "Contract"
    PERFORMER "Gang of Four"
    INDEX 00 26:05:65
    INDEX 01 26:05:70
  TRACK 10 AUDIO
    TITLE "At Home He's a Tourist"
    PERFORMER "Gang of Four"
    INDEX 00 28:45:53
    INDEX 01 28:45:63
  TRACK 11 AUDIO
    TITLE "5.45"
    PERFORMER "Gang of Four"
    INDEX 00 32:16:35
    INDEX 01 32:16:53
  TRACK 12 AUDIO
    TITLE "Anthrax"
    PERFORMER "Gang of Four"
    INDEX 00 35:59:63
    INDEX 01 35:59:73
  TRACK 13 AUDIO
    TITLE "Outside the Trains Don't Run on Time"
    PERFORMER "Gang of Four"
    INDEX 00 40:20:58
    INDEX 01 40:24:13
  TRACK 14 AUDIO
    TITLE "He'd Send in the Army"
    PERFORMER "Gang of Four"
    INDEX 00 43:51:15
    INDEX 01 43:51:23
  TRACK 15 AUDIO
    TITLE "It's Her Factory"
    PERFORMER "Gang of Four"
    INDEX 00 47:31:63
    INDEX 01 47:31:65
So, according with MusicBrainz, the release version may be this one: MusicBrainz Release

The Album.nfo content is:

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<album>
<musicBrainzAlbumID>9c6edb25-3614-4f2c-ae90-54a8d0b39c8c</musicBrainzAlbumID>
</album>

I did a music library clean, a library rescan and a rescrape of this album but nothing has changed in the album info tracklist: I can see in the info window only 12 tracks rather than 15, so I think that musicBrainzAlbumID from the Album.nfo file was not elaborated at all.

I've also tried:
1) unticking in Kodi settings-->Media-->Music  the "Prefer online information" setting;
2) inserting in Album.nfo file only the musicbrainz relase URL,
but there were no changes in tracklist info, so I'm pretty sure there is something I got wrong.

Any further advice?

Thank you, Epg1.


RE: [Release] Universal Scraper for Music Albums - olympia - 2018-01-24

Have only the musicbrainz url to the release in an album.nfo. Not sure if there is case sensitivity, but create an album not Album.nfo. Then submit a debug log if it doesn't work.


RE: Universal Scraper for Music Albums - docwra - 2018-01-24

http://kodi.wiki/view/NFO_files


RE: [Release] Universal Scraper for Music Albums - Boulder - 2018-02-25

(2018-01-12, 16:46)olympia Wrote: I could actually add an option to scrape the AMG user rating instead of the "official". Question is how much this AMG thing is actually futureproof. I (and I think we all) much more prefer, community driven, open databases.
However, for ratings it's always "tricky" as those popular websites both for video and music have much much more voter than the open databases.
 Did this selection for "User rating" ever end up in the scraper? I just restarted adding Allmusic links to Musicbrainz for the albums that don't have a rating, so I'm interested in testing the update of ratings when I'm done. I briefly did a couple when we discussed this but didn't have the time to finish all the albums.


This forum uses Lukasz Tkacz MyBB addons.