Request changes to tag processing
#1
[Split from roadmap thread - DB]

Thanks again for taking the mantle on this, Dave!

I have a bit of spare time now to test stuff as I also have detailed tagging for classical music.
However, I'd like to come forward with a request of my own as well, which would entail a change in KODI.

My usecase:
I come from foobar2000 which so far has been the gold standard of tagging for me. I have tagged some albums with musicbrainz, but not all.
foobar2000 uses the semi-colon ';' as delimiter for fields that accept multiple values.

My problem in KODI:
The advancedsetting-parameters <itemseparator> and <artistseparators> seem to be partly reduntant and conflicting.

Explanation:
KODI recognizes the foobar2000 delimiter as a comma ','. Consequently, I have defined

Code:
    <itemseparator>, </itemseparator>    <!--  separator used for multiple artists/genres in tags. Note, this is *comma* *space*  -->

This works fine for genres, which are appropriately split. It doesn't work that well for artists called "Earth, Wind & Fire" or "Crosby, Stills, Nash & Young". In the first example, KODI creates two artists, in the second example three artists. All of them have the complete discography for the correct group assigend, though.

This doesn't change if the musicbrainz ID are present, as <itemseparator> seems to take priority.

Now, my proposal would be to exempt artists from being treated through <itemseparator> and be exclusively treated through <artistsseparator>. This current treatment seems to be redundant. The latter filter allows me to define

Code:
    <artistseparators>
        <separator> and </separator>
        <separator> And </separator>
        <separator> feat. </separator>
        <separator> Feat. </separator>
        <separator> with </separator>
        <separator>; </separator>
        <separator> ft. </separator>
        <separator> Ft. </separator>
    </artistseparators>

and all should be well (as long as I change all group names that have 'and' to '&', like 'Martha and the Muffins' => 'Martha & the Muffins... no big deal and doesn't hurt scraping).
Reply
#2
Was this completely unreasonable, unreadable or otherwise intolerable? Never got any feedback.
Reply
#3
(2017-08-01, 10:36)HeresJohnny Wrote: Was this completely unreasonable, unreadable or otherwise intolerable? Never got any feedback.
Just lost. Better for feature requests to have own thread than added to my dev plan thread, I see anything posted in the Music Support forum. So I have split into its own thread.
Reply
#4
You propose that artist names are parsed using just <artistsseparator> values and not the general <itemseparator>.

I guess this could be done, keeping backwards compatibility by adding " / " to the default <artistsseparator> values. But the change would adversely effect anyone that had already set-up <artistsseparator> for themselves. Therefore I am not enthusiastic about it. I am all for giving users the tools to configure Kodi however they like, but we do need to keep backward compatibility in mind along the way.

However I am not clear on why you want this. Comma "," is a really poor separator, it appears in too many band names, so does " and ", as you point out yourself. Is it really a problem with how genres are parsed rather than artist names?

Also are you aware of using the ARTISTS and ALBUMARTISTS tags? These work even without mbid tags, so rather than add " and " as an artist separator, add ARTISTS = "artist1 / artist2".

Can you be clearer on what tagging you have (from FooBar2000) that you want Kodi to interpret?
Reply
#5
Sorry to have let this hanging for such a long time, I guess I needed to do some self-searching. I just have some questions about tag reading at this time. Is my understanding correct that

itemseparator
- defines the character that separates multi-values in a tag field
- specifically in Kodi for artist and genre only
- for id3v2 tags only

artistseparator
- is used for further sorting the library and
- defines a character to further split a single artist tag value
- but also works on file naming level (?, unsure)

For artistseparator, the Wiki is not explicit on this.


On the topic of itemseparator, a few follow-up question.
  1. ID3v2.3 uses / as item separator. Hazarding a guess as to why, I'd say because the devs prefer supporting ID3v2.3 standard as it's more compatible (but less versatile).
  2. ID3v2.4 uses the NUL value as item separator. How would I write this value out in advancedsettings?
  3. VORBIS(FLAC) uses no item separators. Instead it repeats each value to a add a new one. Example:
    ARTIST=ELVIS COSTELLO
    ARTIST=THE ATTRACTIONS
    Software that correctly reads this will show 2 artists. Software without proper support will only show the first (I think) artist.
    Does Kodi correctly interpret VORBIS multi-values, including NOT applying any item separators?
  4. Then there's APEv2 tags. Those are a lot like Vorbis tags and use lists for multi-value fields.

Now, if none of my files have ID3v2.3 tags, I would assume that <itemseparator> will never be used and Kodi will read the tags from other tag types correctly without it. Am I right in this respect?
Once these doubts are cleared up, I will be able to improve compatibility between my foobar2000 tags and Kodi reading them. Thanks in advance!
Reply
#6
While trying out one or another thing I played with some tags and rescanned them to see their result. Looks like indeed FLAC files with multiple artists and genres are handled perfectly without itemseprator. Almost.
The exception is - EMBEDDED CUE SHEETS. Man, I really thought it was a good idea back in the day when I ripped my CD collection to single files with cue sheets. That way I'd perfectly preserve gaps and other aspects. After ripping, I've created playable albums with tags by activating "Enable embedded cue sheet on this file" in foobar2000. After that, the contents look like and are treated like separate songs in the foobar library and playlist for all intents and purposes.

Unfortunately, those kind of files behave differently when their tags are read in KODI. The crucial tag fields genre, artist, album artist are not separated properly but instead presented as a single comma-separated string:

Example:
VORBIS TAG genre:   Anime; World; Soundtrack => Split correctly into their separate genres
EMBEDDED CUE SHEET genre:    Anime; World; Soundtrack  => Listed under genre as    "Anime, World, Soundtrack"

Hopefully this can be fixed.
Reply
#7
Hello HeresJohnny

Kodi's disclaimer on Cue Sheets. I highly doubt any developer will even consider looking at cue sheet code in Kodi.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#8
(2018-03-04, 02:45)Karellen Wrote: Hello HeresJohnny

Kodi's disclaimer on Cue Sheets.

I don't understand, did you forget to post a link?

(2018-03-04, 02:45)Karellen Wrote: I highly doubt any developer will even consider looking at cue sheet code in Kodi.

No idea where that is coming from but if it's all the same to you, I'll wait for Dave's reply since he has taken the mantle on music tagging in Kodi.
Reply
#9
My friend @Karellen is right, and he knows how grumpy cuesheet questions make me. I have no interest in cuesheets, and as the only dev working on the music library it means it very unlikely anyone else will makes changes. I have tried to fix bugs involving them, even though it is a frustrating and largely futile task, but I consider them to be an out dated and limited approach to music.

Kodi support is likewise limited. If it is found to be broken the most likely action is the entire support for cuesheets will be removed. Not a threat, just the reality of a small dev team on a large project.

I am one guy, and my time is limited so I have to make choices over how best I spend that time.  I do work on things I don't personally want, and fix things that I did not break, so I am not a total selfish ass. But I have far more to do than I will ever finish.  However Kodi is open source, so if someone has the skills and interest then they are welcome to take up the work.
Reply
#10
But I will take time to answer your questions because I try my best to support users improving their tagging, and no you have not quite understood the use of separators in tags.

The itempseparator was originally introduced because some older tag formats e.g. id3 v2.3 do not support multi-value tags. Note that id3 v2.4 and Vorbis comments (used in FLAC files) do support multi-value tags, but the tag editors may or may not have made use of that.

Initially "/" was taken in artist names as a separator, I think the id3v2.3 standard even suggests that, but that gave problems with the band name "AC/DC". So then " / " was taken as the separator, but made configurable in Kodi. This single separator is applied to artist, albumartist, genre and mood tags in all tag formats. For the mood tag it is irrelevent because Kodi does not codify mood, so it may get split (for JSON API) but then gets joined back together again.

I added the artistseparator parameter in v17 in a further attempt to allow Kodi to better process the inconsistent way some users tagged their music files and accurately identify individual artists from those tags. Telling users to use multi-value tags, or item separator if the format couldn't, just was not working. Many still had things like "artistA feat. artistB" or "artistA; artistB, artistC" (in classical music) as the artist tag, and the artists node looked a mess.

So the user can add artist separator values, a set of char strings, to advancedsettings.xml, and Kodi applies those, in additional to the itemseparator, to the artist and albumartist tag for all tags formats to find individual artist names. By default this set of separators is { ";", " feat. ", " ft. " }, but you can easily extend it, or if you have artists with " ft. " in their name and no mbid tags you could change it. For example I use
Code:
<artistseparators>
<separator> feat. </separator>
<separator> Feat. </separator>
<separator> with </separator>
<separator>;</separator>
<separator> ft. </separator>
<separator> Ft. </separator>
</artistseparators>

So to summerize
(2018-03-04, 00:59)HeresJohnny Wrote: itemseparator
- defines the character that separates multi-values in a tag field
- specifically in Kodi for artist and genre only
- for id3v2 tags only
No.
It can be a string, not a single char, by default is 3 chars " / " space-slash-space.
Applied to artist, albumartist, genre and mood
Applied to all tag formats
(2018-03-04, 00:59)HeresJohnny Wrote: artistseparator
- is used for further sorting the library and
- defines a character to further split a single artist tag value
- but also works on file naming level (?, unsure)
No.
It is used for splitting artist and albumartist tags into individual artists
Is a set of strings
Does not apply to file naming, music library does not parse folder or file names in any way, call them what you like,
 
(2018-03-04, 00:59)HeresJohnny Wrote: On the topic of itemseparator, a few follow-up question.
  • ID3v2.3 uses / as item separator. Hazarding a guess as to why, I'd say because the devs prefer supporting ID3v2.3 standard as it's more compatible (but less versatile).
If you want just slash "/" as the item separator (and don't care about "AC/DC") then change the value use in advancedsettings.xml. Yes it was in the id3 v2.3 documentation, but the collective wisdom has been to ignore that. Using ";" would have been so much better an idea, to my knowledge no artists have that in their name, but they didn't.
(2018-03-04, 00:59)HeresJohnny Wrote:
  • ID3v2.4 uses the NUL value as item separator. How would I write this value out in advancedsettings?
There is no need. The NUL is that formats way of doing multi-value tags, and the tag reader separates those.
 
(2018-03-04, 00:59)HeresJohnny Wrote:
  • VORBIS(FLAC) uses no item separators. Instead it repeats each value to a add a new one. Example:
    ARTIST=ELVIS COSTELLO
    ARTIST=THE ATTRACTIONS
    Software that correctly reads this will show 2 artists. Software without proper support will only show the first (I think) artist.
    Does Kodi correctly interpret VORBIS multi-values, including NOT applying any item separators?
Yes Kodi correctly interprets multi-value Vorbis tags, and separators are not needed.
But you would be surprized how many people have ARTIST = "Elvis Costello / The Attractions" (one value), Kodi processes that correctly too.
 
(2018-03-04, 00:59)HeresJohnny Wrote:
  • Then there's APEv2 tags. Those are a lot like Vorbis tags and use lists for multi-value fields.
Your question? Guessing, yes Kodi correctly processes that tag format.
 
(2018-03-04, 00:59)HeresJohnny Wrote: Now, if none of my files have ID3v2.3 tags, I would assume that <itemseparator> will never be used and Kodi will read the tags from other tag types correctly without it. Am I right in this respect?
Once these doubts are cleared up, I will be able to improve compatibility between my foobar2000 tags and Kodi reading them. Thanks in advance! 
If you have  " / " in an artist or genre tag, of any format, then Kodi will split those as separate artists or genres. But with tag formats that support multi-value you do not have to use " / " to separate names or genres.

I hope that is clearer.
Reply
#11
@HeresJohnny

Yes, unfortunately I did not notice that I failed to ctrl-V the link.

@DaveBlake has provided a more detailed and respectable answer, but I provide the link for the sake of completeness... https://kodi.wiki/view/Music_tagging#Cue_Sheets
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#12
Ok, thank you very much for taking the time to explain all this. First of all, I don't have any AC/DC in my library, so first cliff avoided. Then I guess I'll bite the bullet and try to find a way to split my single FLAC files into separate files while rewriting the EXTERNAL cue sheet to point to the new files so that I will retain the ability to burn to CD should the original perish. The only little thing that makes me sad is albums like Pink Floyd's The Wall where more often than not there are no pauses between tracks and the split will necessarily cut off something in the middle.
Reply
#13
(2018-03-05, 00:47)HeresJohnny Wrote: The only little thing that makes me sad is albums like Pink Floyd's The Wall where more often than not there are no pauses between tracks and the split will necessarily cut off something in the middle.
No need for sorrow, Kodi does an excellent job of gapless playback and nothing will get cut. For example my rip of The Wall into separate FLAC files plays seemlessly.

This may help you with the splitting task https://forum.kodi.tv/showthread.php?tid...pid2677514 but I don't know as never used it. I believe you will be so much better off with individual and fully tagged FLAC files than any cuesheets.
Reply
#14
(2018-03-05, 08:56)DaveBlake Wrote: No need for sorrow, Kodi does an excellent job of gapless playback and nothing will get cut.

I can confirm that.
I have a lot of albums where the first song is an intro for the second song, and both are only "one song", and theres no cut between. I have also some live albuns, wher the sound of the crowd are no cut.  Kodi has THE BEST gapless playback. All my flac albuns are downloaded as a unique file containing all songs with a cue sheet. With Foobar2000 I split them all to separated flac tracks without any lost of quality.
Reply
#15
I've been using CUETools for centuries. It's lightning fast with CUDA support, verifies the integrity of a rip online against the AccurateRip database; tags, creates cues, logs, toc; embeds or extracts album art and washes the dishes.

Just for completeness' sake:

I do hate external cue sheets with a passion as tagging is indeed very limited. Embedded cuesheets on the other hand have absolutely no limitation. The tags inside the FLAC look like any other tag in foobar2000 and the file can be expanded with as many tags as possible. So the limitation factor mentioned in the Wiki doesn't apply - the problem is just with readback as some tag fields are formatted in a non-standard way under the hood.
Reply

Logout Mark Read Team Forum Stats Members Help
Request changes to tag processing1