Kodi 17 - Music Library Improvements - Krypton

  Thread Rating:
  • 1 Vote(s) - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Thread Closed
DaveBlake Offline
Team-Kodi Member
Posts: 2,151
Joined: Jun 2015
Reputation: 56
Location: South West England
Post: #16
(2016-06-01 23:18)bsoriano Wrote:  I am a longtime Kodi user who on and off has used Kodi for music. The reason I keep deleting my music database from Kodi is that I have a large collection (over 145,000 tracks: mp3s, FLAC and some m4a) and it has usually been really difficult (and slow!) to get everything scanned properly into Kodi, or to get meaningful metadata for all of the artists and albums in my collection (which covers a very wide range of genres). I tag all of my files in a way that works for me and does not leave empty or with "garbage" the main tags: All of my files have an artist, albumartist, name, track number, disc number, album, genre and year. All of the songs which have more than one artist performing have "Artist 1 / Artist 2 / Artist n" in the Artist tag, and all of my compilation albums are tagged with an album artist "(Multiple Artists)". However, the program I use to tag my files (and manage and listen to my music collection, J River Media Center) does not use Musicbrainz IDs.

Are you interested in tests with a collection like what I describe above? Please let me know, since I am willing to try once more to use Kodi for music with Krypton. Thanks.

Bart, yes I would be very happy to have your large, Musicbrainz id tag free, music collecton scanned into Krypton and see how it goes. Be great if you could find the time to do that and report back.

At this point I am most interested in the pre-metadata phase, just scanning into library not scraping artist bios, album reviews artwork etc. from online sources. At the moment if you had issues before then getting meaningful metadata is not going to be any better or easier an experience. The presence of mbids would improve this, but to be honest even the person that designed the Universal Scrapers does not have them in their music files, it does work without them but success does depend on your musical tastes. But right now the third-party metadata servers are suffering from overload which is making scraping an even more miss than hit experience. Switching to a mrror can help matters (discussed elsewhere), and I have some ideas for improving things in Kodi core that may help longer term, but I don't want you to be disappointed.

So I'm not sure how much you personally are going to get from this testing (other than a sense of serving the community of course). But I'm certainly keen to know how scanning goes and what the resulting artists list looks like. Longer term I am interested to hear what your issues with scraping metadata were too. There are a few threads on this dotted about, maybe I had better start one on this forum too.
find
bsoriano Offline
Donor
Posts: 708
Joined: Mar 2011
Reputation: 18
Location: Santo Domingo, Dominican Republic
Post: #17
(2016-06-02 07:34)DaveBlake Wrote:  
(2016-06-01 23:18)bsoriano Wrote:  I am a longtime Kodi user who on and off has used Kodi for music. The reason I keep deleting my music database from Kodi is that I have a large collection (over 145,000 tracks: mp3s, FLAC and some m4a) and it has usually been really difficult (and slow!) to get everything scanned properly into Kodi, or to get meaningful metadata for all of the artists and albums in my collection (which covers a very wide range of genres). I tag all of my files in a way that works for me and does not leave empty or with "garbage" the main tags: All of my files have an artist, albumartist, name, track number, disc number, album, genre and year. All of the songs which have more than one artist performing have "Artist 1 / Artist 2 / Artist n" in the Artist tag, and all of my compilation albums are tagged with an album artist "(Multiple Artists)". However, the program I use to tag my files (and manage and listen to my music collection, J River Media Center) does not use Musicbrainz IDs.

Are you interested in tests with a collection like what I describe above? Please let me know, since I am willing to try once more to use Kodi for music with Krypton. Thanks.

Bart, yes I would be very happy to have your large, Musicbrainz id tag free, music collecton scanned into Krypton and see how it goes. Be great if you could find the time to do that and report back.

At this point I am most interested in the pre-metadata phase, just scanning into library not scraping artist bios, album reviews artwork etc. from online sources. At the moment if you had issues before then getting meaningful metadata is not going to be any better or easier an experience. The presence of mbids would improve this, but to be honest even the person that designed the Universal Scrapers does not have them in their music files, it does work without them but success does depend on your musical tastes. But right now the third-party metadata servers are suffering from overload which is making scraping an even more miss than hit experience. Switching to a mrror can help matters (discussed elsewhere), and I have some ideas for improving things in Kodi core that may help longer term, but I don't want you to be disappointed.

So I'm not sure how much you personally are going to get from this testing (other than a sense of serving the community of course). But I'm certainly keen to know how scanning goes and what the resulting artists list looks like. Longer term I am interested to hear what your issues with scraping metadata were too. There are a few threads on this dotted about, maybe I had better start one on this forum too.

Thanks Dave! I will download the latest Krypton nightly tonight and start testing with my music collection. I will report here what I find with the scanning and how the artists list looks.

Regards,

Bart
find
zag Offline
Retired Team-Kodi Member
Posts: 4,006
Joined: Oct 2007
Reputation: 75
Location: UK
Post: #18
Just to be clear, disable "Fetch additional information during updates"

Also in testing you don't have to scan your entire collection, just grab 100 or so albums/releases and copy them to a test folder. then scan that as a source.
(This post was last modified: 2016-06-03 13:11 by zag.)
find
DaveBlake Offline
Team-Kodi Member
Posts: 2,151
Joined: Jun 2015
Reputation: 56
Location: South West England
Post: #19
Yeap, Zag is right disable "Fetch additional information during updates".

But if you have the time to scan the large collection then doing so would be great, if there are problems out there I want to find them Smile
find
bsoriano Offline
Donor
Posts: 708
Joined: Mar 2011
Reputation: 18
Location: Santo Domingo, Dominican Republic
Post: #20
Thanks Dave and Zag! I will restart the process when I get home today, since I had "Fetch additional information during updates" turned on when I left it running this morning. I will do both tests (small portion and complete collection) and report back on my results.

Regards,

Bart
find
deveng24 Offline
Junior Member
Posts: 34
Joined: Jun 2016
Reputation: 0
Post: #21
I just ran my library (700+ artists) with the following parameters;

Fetch additional information during updates and prefer online info disabled.
Provider info set to local only for both album and artist.

Results;
Duplicate artist
Artist not recognized (show as musicbrainz album artist id)
Artist in wrong folder
all issues seem to be caused by musicbrainz album artist id field.... removing this field solved all my issues listed above.
also, noticed that the genre tags do not separate multiple tags when separated by a semi-colon

fyi:
using tag scanner and mp3tag to edit and musicbee as player
find
zag Offline
Retired Team-Kodi Member
Posts: 4,006
Joined: Oct 2007
Reputation: 75
Location: UK
Post: #22
Are you using the latest nightly? I thought the musicbrainzID for name was fixed in this version.

A debug log will help us track it down.
find
deveng24 Offline
Junior Member
Posts: 34
Joined: Jun 2016
Reputation: 0
Post: #23
(2016-06-03 21:18)zag Wrote:  Are you using the latest nightly? I thought the musicbrainzID for name was fixed in this version.

A debug log will help us track it down.

I downloaded yesterdays nightly (windows) build .....now that i know an album the shows the error ill re-run and upload the debug log
find
scott967 Offline
Posting Freak
Posts: 2,690
Joined: Jul 2012
Reputation: 84
Post: #24
There was a bug on album artist that Dave fixed (was picking up the artist MBID, at least when there is no album artist MBID assigned in tags). My current practice for album artist MBID is this:

1. do not use album artist MBID tag unless needed.
2. If needed, always use album MBID in addition to album artist MBID.
3. Use Album Artists (note"s") hint as needed when also using album artist MBID (must be one-to-one correspondence between album artists hint and album artist MBIDs).

scott s.
.
find
bsoriano Offline
Donor
Posts: 708
Joined: Mar 2011
Reputation: 18
Location: Santo Domingo, Dominican Republic
Post: #25
Hello again,

I scanned all of my compilation albums (1500+) and for the most part, the process went very well. The resulting list of artists is 12,866 artists, which is OK, except for tagging errors I made that I have to fix, and one error with an artist "Trin-I-Tee 5:7", which Kodi scanned as two different artists, "Trin-I-Tee 5" and "7". So I guess there is an issue when the artist name contains a ":".

I will run through this process again (I deleted the source and cleaned the database) after fixing my tagging errors and after deleting all of the album.nfos I did not remember I had from using MediaElch a while ago.

More to come.

Regards,

Bart

EDIT: I fixed my tagging errors and re-ran the process. Except for the artist error described above, the only two strange situations I have found are:

1. There is one album that, although I check and re-check the tags and they are correct (all tracks have an albumartist of "(Multiple Artists)", all tracks have the same album name), Kodi takes each track as a separate album. I will put a copy of this album in a separate folder, scan that as a source and turn debug logging on and upload the log so that you can take a look at this.

2. All of the albums I scanned are compilations (even though all of them have "(Multiple Artists)" in the albumartist tag), however, Kodi shows like 3 or 4 in the Compilations node and not any of the others. Any ideas why that would be?
(This post was last modified: 2016-06-04 18:28 by bsoriano.)
find
DaveBlake Offline
Team-Kodi Member
Posts: 2,151
Joined: Jun 2015
Reputation: 56
Location: South West England
Post: #26
Cheers Bart.

Well well, an artist name with a colon in it! The default multiple artist name separators added in 17 include ":", but it is user configurable.

The defualt for artist names in addition to " / " is any of ";", ":", "|", " feat. ", " ft. " is taken as a separator. However this can be replaced by entries in advancedsettings.xml like this

Code:
<advancedsettings>
  <musiclibrary>
    <artistseparators>
      <separator> feat. </separator>
          <separator> Feat. </separator>
          <separator> with </separator>
          <separator>;</separator>
          <separator> ft. </separator>
          <separator> Ft. </separator>
    </artistseparators>
</musiclibrary>
</advancedsettings>

Which has " with " as a separator because I know none of my individual artists have that in their name while I have tracks tagged with artist1 with artist2, and no ":". These settings replace the defualt values entirely.

Rather than edit tags perhaps you would like to give that a go?

As for NFOs they do hide in unexpected places Smile
find
bsoriano Offline
Donor
Posts: 708
Joined: Mar 2011
Reputation: 18
Location: Santo Domingo, Dominican Republic
Post: #27
Thanks Dave, I will give that a try to deal with the artist name with the :

Here is the debug log for scanning that stubborn album that gets scanned as multiple albums:

https://www.dropbox.com/s/swaoxulglnymed...g.zip?dl=0

Please let me know if this is useful or you would like me to do something else. Thanks.

Regards,

Bart
find
DaveBlake Offline
Team-Kodi Member
Posts: 2,151
Joined: Jun 2015
Reputation: 56
Location: South West England
Post: #28
(2016-06-04 18:42)bsoriano Wrote:  Here is the debug log for scanning that stubborn album that gets scanned as multiple albums:

Please let me know if this is useful or you would like me to do something else. Thanks.

Thanks Bart. Unfortunately the log is not terribly revealing in this matter, but I am intreagued. Can you show me how the files for this album are tagged?
find
bsoriano Offline
Donor
Posts: 708
Joined: Mar 2011
Reputation: 18
Location: Santo Domingo, Dominican Republic
Post: #29
(2016-06-04 20:24)DaveBlake Wrote:  
(2016-06-04 18:42)bsoriano Wrote:  Here is the debug log for scanning that stubborn album that gets scanned as multiple albums:

Please let me know if this is useful or you would like me to do something else. Thanks.

Thanks Bart. Unfortunately the log is not terribly revealing in this matter, but I am intreagued. Can you show me how the files for this album are tagged?

Dave,

I am not sure how to best show you how the files are tagged, so I have put the whole album here: https://www.dropbox.com/s/mh6pe5gqjn4u7c...9.rar?dl=0

My music database is here: https://www.dropbox.com/s/dvelmyfyzen3x5...0.rar?dl=0

The weird thing is that the Album nodes shows all of these songs as separate albums, but if I look at the Songs node, it would seem that the tags are fine. I also changed the scraper to be Local Only, and deleted the source, cleaned the library, removed the music database and restart Kodi and scanned the folder again, with exactly the same results.

Sorry for the size (146MB), I hope you can figure out what is going on here, since I am loath to throw my whole collection at Kodi right now, since I am not sure with how many of my albums this situation could happen.

Thanks for your help.

Regards,

Bart
find
DaveBlake Offline
Team-Kodi Member
Posts: 2,151
Joined: Jun 2015
Reputation: 56
Location: South West England
Post: #30
Bart thanks for the files. You may want to remove them from Dropbox now I have them.

(2016-06-04 22:28)bsoriano Wrote:  The weird thing is that the Album nodes shows all of these songs as separate albums, but if I look at the Songs node, it would seem that the tags are fine. I also changed the scraper to be Local Only, and deleted the source, cleaned the library, removed the music database and restart Kodi and scanned the folder again, with exactly the same results.

The issue here is the tags. If you take a look at them using something like Mp3tag then you will be able to see what I am talking about.

For Kodi to see separate music files as an album they need to be tagged with the same album title (which they are) and the same album artists or album artist be empty. These music files have a mix of album artists.

For example Mp3tag for track 02 has
ALBUMARTIST = (Multiple Artists)
ALBUMARTIST = Kelli O'Hara, Jake Lucas, Ted Sperling & Orchestra
and also shows (note the space)
ALBUM ARTIST = (Multiple Artists)

Kodi is seeing the "Kelli O'Hara, Jake Lucas, Ted Sperling & Orchestra" (from the ID3 v2.3 tag) as a single artist, not getting "(Multiple Artists)" at all.

The files have both ID3 v1 and ID3 v2.3 tags, but ID3v1 does not have an album artist. The ID3v2.3 standard has only single value frames so I don't know where the second albumartist value with (Multiple Artists) is coming from.

This tag needs sorting out, ALBUMARTIST set to "Various Artists", or "(Multiple Artists)" if you prefer, and any extra value deleted. You need to use a tool that show both to be able to do this.

Quote:Sorry for the size (146MB), I hope you can figure out what is going on here, since I am loath to throw my whole collection at Kodi right now, since I am not sure with how many of my albums this situation could happen.

No worries. I guess the only way to know if you have more like this is to look at the tags in Mp3tag. How did you tag your music files in the first place?

Or you could just go for it, scan with Kodi and let the artist list show you Smile

One more thing I noticed, genre is "Musicals/Broadway", that willbe taken as a single genre by Kodi. If you change it to "Musicals / Broadway" then Kodi will have separate genres "Musicals" and "Broadway", if that is the result you would prefer.
find
Thread Closed