Classical music (composer, year)
#16
(2017-02-20, 21:17)beedaddy Wrote: Just as a side note: I'm in the process of adding the ALBUMARTISTS tag. In some cases, this does not help. In my despair I then removed the MUSICBRAINZ tags and then it worked. This surprised me (since I set the provider for album information to "Local information only").

It sounds like you are confusing scanning (of tags embedded into your music files) with scraping (of additional information about artists and albums most of which can not be included in tags e.g. artist biography, or album review).

The ALBUMARTISTS tag is about helping Kodi deduce the individual album artist names from the ALBUMARTIST tag, to match up with musicbrainz album artist ID tags. Look closely at the cases where you say this "does not help". Do you have a different number of musicbrainz ids to artist names? Is it actually an ARTIST and ARTISTS tag issue? Since the tagging from Picard should have been OK, so I have to conclude that you probably edited the tagging in some odd way. Removing mbids could mean that subsequent scraping will be less accurate, and having tagged with Picard it seems a shame, but it is up to you.

Or it could be that you had jumbled names and mbids in the library from older tagging, and that update library alone working through the changed tags is not unscambling it. I don't know as you don't mention what is wrong exactly. Probably worth removing the source(s), cleaning the db (from settings) and adding the source(s) again so it all scans into a clean libaray. This will however lose your last played, and playcount data.

(2017-02-20, 21:48)graham-h Wrote: There could be two explanations: some of the files got their size or date changed when you tagged them and some did not (so Kodi didn't re-scan them) or else the MB tags overrode the ALBUMARTISTS. Tags are presumably 'local information' whether 'you' wrote them or Picard wrote them.
No, very wrong.
Quote:The whole business of how Kodi merges on-line information with tags is a mystery. I haven't been able to see any difference between 'Prefer online information' on or off. Certainly, Kodi scrapes less data from MB than Picard does.
You are also confusing scanning with scraping.
The music library is populated by scanning tag data embedded in the music files. It takes all the tracks with the same album tag and creates an album entry in the library, likewise with artists. But some details about the albums can be determined from tags too, e.g. year or genre, by taking the common song files values. Details about artists e.g. date of birth or formation, biog etc. and other things about albums can not be derrived from tags, but can be scraped from online sources or from local NFO files.

If you set the scraper settings as provider for album information to "Local information only" then that means don't look online when scraping additional album information, just look at album.NFO file alongside the music files for that album. The other scraper settings are selecting what online source to use for certain data.

For the additional artist and album data that Kodi can scrape see http://kodi.wiki/view/Music_tagging#Data_Kodi_Scrapes

Note that for albums genre, year, (record) label and compilation could also have already be determined from tags. 'Prefer online information' enabled means that the online value will replace that was derrived from tags. Of course if you use Picard to tag, and then later scrape additional information from Musicbrainz then you will get the same year, (record) label and compilation. For genre do you want the ones you choose to tag with, or do you want then overwritten by the wisdom of crowds from an online source (I don't think MB give genre values, but TADB do)?

If you did not tag with Picard (or something else that adds Musicbrainz id tags), or have deleted those tags, then the scraper just uses album title and artist name to try identify the artist and album. This is fallable, the scraper can return the wrong "Beethoven" "Symphony No. 5" release, it can even get the wrong "Greatest Hits" or other albums where names alone do not identify release. If you have 'Prefer online information' enabled and it finds the wrong album, then it will replace the artists associated with that album with the ones it found.

Having had scarping with 'Prefer online information' enabled make a total mess of my classical music, I would say that it is not a safe option to use unless you have mbid tags in all your music files. But for some people (with only popular musical tastes) that made a mess of their genre tags (and prefer the mess made the crowd), or did not tag year, I guess it may have a use.

Tag songs files accurately, include mbid tags to help separate artists with the same name, and if you also want to later scrape additional information about artists and albums. Scan them into the Kodi music library, checking that the tagging has created accurate results. Then, if you like, scrape additional artist and album information and artwork using the online sources that you choose (but 'Prefer online information' disabled, as frankly it is just a liability not a help, and you did all that good tagging).
Reply
#17
I set Prefer online information and Fetch additional information to false. And Default provider for album/artist information are set to Local information only. This is because I'm not interested in additional online information, I just want to get the local data sorted cleanly.

Perhaps this describes the issue more clearly:
With these settings I still got a composer Johann Sebastian Bach; Trevor Pinnock (which is the ALBUMARTIST tag) even though I have ALBUMARTISTS set to Johann Sebastian Bach / Trevor Pinnock. Without the MUSICBRAINZ-tags, it shows the composer as Johann Sebastian Bach, just as it should be.
Reply
#18
(2017-02-21, 15:21)beedaddy Wrote: I set Prefer online information and Fetch additional information to false. And Default provider for album/artist information are set to Local information only. This is because I'm not interested in additional online information, I just want to get the local data sorted cleanly.
With "Fetch additional information" off scraping doews not happen unless you do "Query info for all" from a context menu, or "Refresh" from the Info Dialog. But setting the provider is belt and braces.

Quote:Perhaps this describes the issue more clearly:
With these settings I still got a composer Johann Sebastian Bach; Trevor Pinnock (which is the ALBUMARTIST tag) even though I have ALBUMARTISTS set to Johann Sebastian Bach / Trevor Pinnock. Without the MUSICBRAINZ-tags, it shows the composer as Johann Sebastian Bach, just as it should be.

When you say composer you mean composer, not just artists node? Can you send the MBID or link for this album, so I can see if there is anything obvious in the way Picard may have tagged.
Reply
#19
(2017-02-21, 15:45)DaveBlake Wrote: When you say composer you mean composer, not just artists node?
Yes.
(2017-02-21, 15:45)DaveBlake Wrote: Can you send the MBID or link for this album, so I can see if there is anything obvious in the way Picard may have tagged.
This is MUSICBRAINZ_ALBUMID, right? One example are piano sonatas from Beethoven, MUSICBRAINZ_ALBUMID is 538948d3-3605-4688-ada8-cf5dd1691d6c.
And this is the link.
Reply
#20
(2017-02-21, 14:44)DaveBlake Wrote: You are also confusing scanning with scraping.

Not really. I know exactly what scanning is and what scraping is. What wasn't clear is which data is scanned, which is scraped and under what settings the scanned data will be overwritten. It's such a pity you are having to spend time explaining this in dribs and drabs when your valuable time would be better spent hacking at C++! However, it is much appreciated and I will update my, now 5-page, document on music tagging with Kodi which may find its way onto this board or the wiki in due course.

Quote:If you set the scraper settings as provider for album information to "Local information only" then that means don't look online when scraping additional album information, just look at album.NFO file alongside the music files for that album. The other scraper settings are selecting what online source to use for certain data.

For the additional artist and album data that Kodi can scrape see http://kodi.wiki/view/Music_tagging#Data_Kodi_Scrapes

That implies Kodi never scrapes Albumartist (ie artistCredits) or details of tracks (track number, title, track length). But in Album.cpp we see some code making heroic efforts to match Albumartists with MB data. Could this be beedaddy's problem - a mismatch between what he has in Albumartists and what is in the database from MB?

If I export to album.nfo after scraping, I get album title and artist and a track list. You explained elsewhere that .nfo contains only scraped data. Hence confusion about whether these fields are in danger of being overwritten. If Provider is set to Local information only, does the switch Prefer online information affect whether data in a .nfo file overrides data from file tags? (just asking)

Quote:If you did not tag with Picard (or something else that adds Musicbrainz id tags), or have deleted those tags, then the scraper just uses album title and artist name to try identify the artist and album. This is fallable..

Yes - trying find a classical album can be a headache. So I turn off scraping for albums (Local information only), but leave scraping on for artists. But I don't get any fanart. Is that expected behaviour?

Please keep up the good work!
Reply
#21
Again, I try to give some more information. Smile

With my settings as described above, I did one more test. The Album is the "Goldberg Variations" from J. S. Bach. Performer is Trevor Pinnock. So, ALBUMARTIST is "Johann Sebastian Bach; Trevor Pinnock" and ALBUMARTISTS is "Johann Sebastian Bach / Trevor Pinnock". opusinfo gives me the following information for the first track:
Code:
ACOUSTID_ID=71fc80a9-fe69-4aca-939e-8e5cd3d6b946
        ALBUM=Goldberg Variations
        ALBUMARTIST=Johann Sebastian Bach; Trevor Pinnock
        ALBUMARTISTS=Johann Sebastian Bach / Trevor Pinnock
        ALBUMARTISTSORT=Bach, Johann Sebastian; Pinnock, Trevor
        ARTIST=Johann Sebastian Bach
        ARTISTS=Johann Sebastian Bach
        ARTISTSORT=Bach, Johann Sebastian
        ASIN=B0000057CK
        BARCODE=028941513021
        BPM=0
        CATALOGNUMBER=415 130-2
        COMPILATION=0
        COMPOSER=Johann Sebastian Bach
        DATE=1980-01-01
        DESCRIPTION=Archiv Produktion
        DISC=1
        DISCC=1
        DISCNUMBER=1
        DISCTOTAL=1
        ENCODER_OPTIONS=--bitrate 160
        GENRE=Klassik                                                                                        
        LABEL=Archiv Produktione                                                                              
        MEDIA=CD                                                                                              
        MUSICBRAINZ_ALBUMARTISTID=24f1766e-9635-4d58-a4d4-9413f9f98a4c                                        
        MUSICBRAINZ_ALBUMID=b912c6ad-2dc3-457c-b43e-068d288cc56f                                              
        MUSICBRAINZ_ARTISTID=24f1766e-9635-4d58-a4d4-9413f9f98a4c                                            
        MUSICBRAINZ_RELEASEGROUPID=cbb263ba-3a35-3c66-9c3d-9a1241050261                                      
        MUSICBRAINZ_RELEASETRACKID=0f6e9235-e9d9-36a5-b773-6352f550a6d7                                      
        MUSICBRAINZ_TRACKID=f760aaa9-a428-458a-b1c0-5920f4377fa5                                              
        ORGANIZATION=Polydor                                                                                  
        ORIGINALDATE=0000                                                                                    
        PERFORMER=Trevor Pinnock                                                                              
        PUBLISHER=Archiv Produktione                                                                          
        RELEASESTATUS=official                                                                                
        RELEASETYPE=album                                                                                    
        REPLAYGAIN_ALBUM_GAIN=0.00 dB                                                                        
        REPLAYGAIN_ALBUM_PEAK=0.000000                                                                        
        SCRIPT=Latn                                                                                          
        TITLE=Aria                                                                                            
        TOTALDISCS=1                                                                                          
        TOTALTRACKS=32                                                                                        
        TRACK=1                                                                                              
        TRACKC=32                                                                                            
        TRACKNUMBER=1                                                                                        
        TRACKTOTAL=32
        YEAR=1980
        METADATA_BLOCK_PICTURE=3|image/gif||0x0x0|<102844 bytes of image data>
I would expect that the composer would be shown as "Johann Sebastian Bach". But instead, it is shown as "Johann Sebastian Bach; Trevor Pinnock". Oh and before importing the files, the music database was empty. Now, the artist table has the following entries:
Code:
sqlite> SELECT * FROM artist;
1|[Missing Tag]|Artist Tag Missing|||||||||||||
2|Johann Sebastian Bach; Trevor Pinnock|24f1766e-9635-4d58-a4d4-9413f9f98a4c|||||||||||||
3|Trevor Pinnock||||||||||||||
Line 2 has the correct artist id, but the wrong name. This puzzles me.
Reply
#22
I think you have indeed exposed a bug which when fixed should resolve your original issue of different albums being interpreted differently despite having similar tags.
Reply
#23
The more relevant database entry might be:
Code:
sqlite> SELECT * FROM songartistview;
idSong|idArtist|idRole|strRole|strArtist|strMusicBrainzArtistID|iOrder
1|2|1|Artist|Johann Sebastian Bach; Trevor Pinnock|24f1766e-9635-4d58-a4d4-9413f9f98a4c|0
1|2|2|Composer|Johann Sebastian Bach; Trevor Pinnock|24f1766e-9635-4d58-a4d4-9413f9f98a4c|0
1|3|3|Performer|Trevor Pinnock||0
[...]
Reply
#24
(2017-02-22, 11:52)graham-h Wrote: I think you have indeed exposed a bug which when fixed should resolve your original issue of different albums being interpreted differently despite having similar tags.
Wrong, it is just inconsistent tagging.

(2017-02-22, 10:22)beedaddy Wrote: Again, I try to give some more information. Smile

With my settings as described above, I did one more test. The Album is the "Goldberg Variations" from J. S. Bach. Performer is Trevor Pinnock. So, ALBUMARTIST is "Johann Sebastian Bach; Trevor Pinnock" and ALBUMARTISTS is "Johann Sebastian Bach / Trevor Pinnock". opusinfo gives me the following information for the first track:
Code:
...
ALBUMARTIST=Johann Sebastian Bach; Trevor Pinnock
ALBUMARTISTS=Johann Sebastian Bach / Trevor Pinnock
ARTIST=Johann Sebastian Bach
ARTISTS=Johann Sebastian Bach
COMPOSER=Johann Sebastian Bach
MUSICBRAINZ_ALBUMARTISTID=24f1766e-9635-4d58-a4d4-9413f9f98a4c                                        
MUSICBRAINZ_ARTISTID=24f1766e-9635-4d58-a4d4-9413f9f98a4c
I would expect that the composer would be shown as "Johann Sebastian Bach". But instead, it is shown as "Johann Sebastian Bach; Trevor Pinnock". Oh and before importing the files, the music database was empty. Now, the artist table has the following entries:
Code:
sqlite> SELECT * FROM artist;
1|[Missing Tag]|Artist Tag Missing|||||||||||||
2|Johann Sebastian Bach; Trevor Pinnock|24f1766e-9635-4d58-a4d4-9413f9f98a4c|||||||||||||
3|Trevor Pinnock||||||||||||||
Line 2 has the correct artist id, but the wrong name. This puzzles me.
More information is always good. Seeing the tagging, and I have edited your quote just to show the significant tags, it is clear to me what the issue is. The db entries confirm it.

Kodi is only given one MUSICBRAINZ_ALBUMARTISTID tag value (that for JS Bach), so it takes the names in the ALBUMARTIST tag as that single artist and ingnoree ALBUMARTISTS. The album artist is added first, then when the (song) artist is processed it sees it already has an artist entry with that mbid and leaves it. It also collects that as the mbid for the composer, hence you see "Johann Sebastian Bach; Trevor Pinnock"

The fix - add MUSICBRAINZ_ALBUMARTISTID tag for Trevor Pinnock. Not sure why this was not there if tagging using Picard, but the MUSICBRAINZ_ALBUMARTISTID = 24f1766e-9635-4d58-a4d4-9413f9f98a4c / 66d97594-15fd-49b9-8359-0c014dbe8719

If mbid tags are present then Kodi assumes that they are correct and complete - generally they are going to be applied by software, no hand edited, so this is reasonable. It then processes the other tags given the mbids, so 1 artist mbid tag means Kodi will only look for 1 artist name.

I made improvements in v17, but I am always interested to see what issues users get, and consider if Kodi can be enhanced to work around them. But in artist name parsing is hard to catch everything. The log when you scanned this item may have shown some name-mbid mismatch warmings, but just did its best interpretation.
Reply
#25
(2017-02-21, 22:06)graham-h Wrote: That implies Kodi never scrapes Albumartist (ie artistCredits) or details of tracks (track number, title, track length). But in Album.cpp we see some code making heroic efforts to match Albumartists with MB data. Could this be beedaddy's problem - a mismatch between what he has in Albumartists and what is in the database from MB?
Now I risk confusing you...
Historically Kodi does also scarpe some track information for an album, but it is just treated as display text. It gets stuffed in a separate table, and is only displayed on the album info dialog (on some skins). It is not used in any other way, nor visible anywhere else in the library. The Song data, derrived from tags, is not changed. This album track info scraping is being dropped in v18.

It uses the albumartist(s) when scraping, if you do not have album mbid so isn't going to change that. If album mbid is present, then the album artists should also have mbids and the scraped results should match. Yet if for some reason you have changed the albumartists then "prefer online info" with change them to the scraped values.

The "heroic efforts" you see in the album.cpp code is all about tag processing during scanning, and nothing to do with scraping.

As in my post above, Beedaddy's issue is simply mismatching tagging - 1 mbid and 2 artist names.
Ironically on this ocassion, if he had scraped with "Prefer online info" then it may have sorted things out, but with most tagging mishaps scraping would make matters worse.

Quote:If I export to album.nfo after scraping, I get album title and artist and a track list. You explained elsewhere that .nfo contains only scraped data. Hence confusion about whether these fields are in danger of being overwritten. If Provider is set to Local information only, does the switch Prefer online information affect whether data in a .nfo file overrides data from file tags? (just asking)
Ah yes it does export the silly track list that it scraped. As I explained, this lives in an isolated table and is a left over from the distant past, it does not overwrite the song data in any way.

NFO, yes, I think it does, check out MusicInfoScanner.cpp for the details.

Quote:Yes - trying find a classical album can be a headache. So I turn off scraping for albums (Local information only), but leave scraping on for artists. But I don't get any fanart. Is that expected behaviour?
Are you scraping somewhere that has fanart? The usual reason people don't get artist art is server timeout. The next is that they have to retry to fill in the gaps. But assuming that you are asking about fanart specifically, then it is a matter of asking in the right place too. Also many people use CDArt addon to do the exciting art stuff.
Reply
#26
(2017-02-22, 14:00)DaveBlake Wrote:
(2017-02-22, 11:52)graham-h Wrote: I think you have indeed exposed a bug which when fixed should resolve your original issue of different albums being interpreted differently despite having similar tags.
Wrong, it is just inconsistent tagging.
IMVHO, if there is a tag that says Composer = Johann Sebastian Bach, that should take precedence over any other infomation.
Quote:It also collects that as the mbid for the composer, hence you see "Johann Sebastian Bach; Trevor Pinnock"
There's clearly something wrong here. From the user's perspective, Kodi is rejecting an explicit composer name in a tag in favour of one derived from an artist tag which may not even include the composer. Under the hood, how does Kodi 'collect' the mbid for the composer if Prefer online information is off? There is no relationship between mbids and contributing artists in the file tags AFAICT.

Quote:If mbid tags are present then Kodi assumes that they are correct and complete - generally they are going to be applied by software, no hand edited, so this is reasonable. It then processes the other tags given the mbids, so 1 artist mbid tag means Kodi will only look for 1 artist name.
Understood, but there is no mbid tag for Composer.
Reply
#27
(2017-02-22, 14:28)DaveBlake Wrote:
Quote:Yes - trying find a classical album can be a headache. So I turn off scraping for albums (Local information only), but leave scraping on for artists. But I don't get any fanart. Is that expected behaviour?
Are you scraping somewhere that has fanart? The usual reason people don't get artist art is server timeout. The next is that they have to retry to fill in the gaps. But assuming that you are asking about fanart specifically, then it is a matter of asking in the right place too. Also many people use CDArt addon to do the exciting art stuff.
I'm getting some information coming through, now, by manually refreshing each artist using the default sites. Not a big deal for me, but it does make the screen look prettier Smile
Reply
#28
(2017-02-22, 17:05)graham-h Wrote:
(2017-02-22, 14:28)DaveBlake Wrote:
Quote:Yes - trying find a classical album can be a headache. So I turn off scraping for albums (Local information only), but leave scraping on for artists. But I don't get any fanart. Is that expected behaviour?
Are you scraping somewhere that has fanart? The usual reason people don't get artist art is server timeout. The next is that they have to retry to fill in the gaps. But assuming that you are asking about fanart specifically, then it is a matter of asking in the right place too. Also many people use CDArt addon to do the exciting art stuff.
I'm getting some information coming through, now, by manually refreshing each artist using the default sites. Not a big deal for me, but it does make the screen look prettier Smile
Have you tried using "Query info for all" from the artists node, or any list of artists? A bit easier than visiting each artist manually.
Reply
#29
(2017-02-22, 16:52)graham-h Wrote: IMVHO, if there is a tag that says Composer = Johann Sebastian Bach, that should take precedence over any other infomation.
Yeap sure.

Quote:
Quote:It also collects that as the mbid for the composer, hence you see "Johann Sebastian Bach; Trevor Pinnock"
There's clearly something wrong here. From the user's perspective, Kodi is rejecting an explicit composer name in a tag in favour of one derived from an artist tag which may not even include the composer. Under the hood, how does Kodi 'collect' the mbid for the composer if Prefer online information is off? There is no relationship between mbids and contributing artists in the file tags AFAICT.
We also have in the tags for that song
ARTIST = Johann Sebastian Bach
MUSICBRAINZ_ARTISTID = 24f1766e-9635-4d58-a4d4-9413f9f98a4c
Kodi matches the names of the COMPOSER tag to that in the ARTIST tag and can therefore deduce the mbid for the composer.

The problem is that the tag processing has already associated that mbid with the artist name "Johann Sebastian Bach; Trevor Pinnock". In all subsequent places Kodi will have that composite as the name for Johann Sebastian Bach.

Kodi uses the first artist name given with an mbid and does not change it if other files or tags give it a different name e.g. Johann Sebastian Bach or JS Bach. It just happens that the first name in this instance was completely incorrect. Kodi is not just storing text for the song but attempting to identify artists individually, and their relationship to the music files.

I hope to add an artist alias name facility to Kodi one day. You could argue that Kodi keeps changing the name, thus always the last processed rather than the first, or that it somehow at least tells the user that they have these different names for a certain mbid. The question is how complicated do we make it? Taking the first makes as most sense as the last.

We could have Kodi simply reject any mis-matched tagging of name and mbid count, would that be better? If people didn't edit what comes out of Picard there would not be any issues.

Quote:Understood, but there is no mbid tag for Composer.
No there is no tag, but as the composer is often a tagged artist too we can get it from there by matching on name as I explained.
Reply
#30
OK, I don't think it's worth spending more time on a situation which should not have arisen in the first place. My treatease on music tagging for Kodi has now got a warning about these possible mis-matches.

Artist info is now gradually coming down the wire, thanks.
Reply

Logout Mark Read Team Forum Stats Members Help
Classical music (composer, year)0