Best way to update music db 16 -> 17
#1
So in moving to 17 how should we update the music db to make sure everything is in "17 format"? Just let Kodi update the db? Dump the db and scan the whole thing all over again? Use single or multi-file library export in 16, dump the db, and then import that in 17 (or let scraper find nfo files)? Other ideas?

scott s.
.
Reply
#2
An excellent question Scott967, and if I could further this...

1. Is there any settings I should turn off or on, before I add my Music to Kodi (and maybe describe what they do). For example: The setting to not Display Artists who are Featuring, in the Artists display. Can this setting even be adjusted before adding Music?

2. Is there still the Bug where you should add all your Compilations after your regular CD's?

3. What is recommended, read from NFO files (in my case, generated from MediaElch), or use the default 'Universal' add-on's to gather the info for each Album and Artist?

I'm sure there are others I am missing, which I may add later. 8)
Reply
#3
Good questions guys. Of course the answer depends on what you want to achieve, and your music collection.

Just have Kodi update the db is the most straight forward, and does not require an internet connection. However without re-scanning the tags you will not get any additional composer, conductor, lyricist, engineer etc. (the artist roles) information. Now if you have not included these in your tagging there is no reason to re-scan, but if you have then I recommend that you do re-scan from scratch (I want you to use the new functionality).

However, rescanning will also mean that you need to rescrape all the externally supplied info either from online databases or from NFO. With external sources the time taken and success will depend on server traffic. I hear that the Musicbrainz issues have now been resolved, but for the last few months such a re-scrape has unfortunately been a problem with frequent time-out failures causing the process to take a long time.

The export/import idea is a good way to reduce server traffic and keep all you previous art etc. I hope, although it will be in v18 now, that we can make that update route more mainstream and avoid unnecessary re-scraping. But for this upgrade because such an approach avoids scanning it will not let you see composers etc.

So to summerise: if you have tagged your music files with composer etc. then let Kodi upgrade and then rescan (and rescrape). Otherwise, an export/import approach would be an efficient way to update.

Quote:1. Is there any settings I should turn off or on, before I add my Music to Kodi (and maybe describe what they do). For example: The setting to not Display Artists who are Featuring, in the Artists display. Can this setting even be adjusted before adding Music?

The "show song artists and album artists" setting (correctly named in v17, was wrongly "show artists that appear only on compilations" in v16) only effects how the artists node is displayed, not the way music is scanned or what is held in he db. It can be changed at any time.

With advanced settings selected there are 2 library settings that effect scraping of additional data from external sources, and of course the scraper settings. You need to check these are set as you require before you add music or update the library. How they are set is much a personal preference.

a) "Prefer online information" when enabled causes the library data scanned from the music files tags to be overwritten by that which is scraped from external sources. Note the music files themselves are not changed in any way, just the library. The information scraped for artists is not available in the tags, so this setting has no effect. But some of the information that is scraped for albums can initially come from the tags e.g. year, genre(s), compilation flag and thumb art, as well as the artist(s). This setting enables all that information to be replaced with the values found online or in an NFO.

So in theory if you have not tagged your music with year, genre etc. then you would enable this to get them from NFO files. I say in theory because I am not fond of this approach, and suspect there are issues with it - IMO good music file tagging is better! I guess if you have NFO for every album, or a very mainstream pop music collection, then it could work. An example of my concerns happens with this setting enabled and no NFO file exists, if the scraper mis-identifies an album (happens lots with classical music where album name and artist are not unique, and no mbid available) then it can scramble the song infomation.

b) "Fetch additional information during updates" controls if scraping of extrenal artist and album data happens every time the library is updated. Persoanlly I disable this, I prefer to scrape what I want when I want rather than have it happen automatically. Scraping externally needs an internet connection, and on a large library can be slow. I know a lot of my music isn't going to be found, so scraping is a waste of time. I guess from a general user view point automatic scraping, that makes nice art work just appear by magic, is great. But it is pretty easy to click "Query Info for all" on the context menu when you want (for the listed artists or albums - means that you can scrape just the artists on a smart playlist rather than your entire collection)

Quote:2. Is there still the Bug where you should add all your Compilations after your regular CD's?
Not sure, so going to say yes, sorry not enough hours in the day Sad

This is a work around for an artwork/NFO issue right? I remember it being good advice but not why, so someone remind me.

Kodi depends on the artist having a unique common folder for all their music i.e. arranged in a folder structure of Album Artist/Album/songs, and scarping/art go wonky if this isn't the case.

Quote:3. What is recommended, read from NFO files (in my case, generated from MediaElch), or use the default 'Universal' add-on's to gather the info for each Album and Artist?

If you can work around the current limitations with NFO files then they are a great way to locally store, independant of the internet, the artist and album information that can not be held in music file tags. Tools like MediaElch help with the creation, and given you control of what is held. So if you like control of your data, or have ecletic music tastes (hence no info avaibale online), or don't agree with the wisdom of the crowd, or just don't want to collect information online every time you upgrade then they are a good way to proceed.

But if you just can't be assed to be so fussy, want to have art and info just appear with minimum effort, have mainstream music tastes etc., are grateful for the wisdom of crowds, then let the Universal scraper do its thing.

I do have it on my list to fix the NFO weak areas, in combination with making scraping more efficient (rather than batter ther servers) - only re-fetching the data from online when we want to check for new info not just recollecting data we have carelessly discarded.

I suspect that many users will want a mixture of both NFO and scraped online, it so depends on your musical tastes

Ask away. Happy to make an FAQ out of this.
Reply
#4
Thanks so much @DaveBlake and @scott967 for this thread, it will really help when I upgrade to Kodi 17.1 (I never do the initial release, mostly because of the Skin I use Aeon MQ8 will be late).

So I will remove and re-add my Music with Kodi 17.1 and follow your advice on what to turn on and off. Again, thanks for this.

Now on to the new questions (that I new I would remember).

1. Normally, I use MusicBrainz Picard (for initial tagging), and MediaMonkey for any additional info (normally fixing Genre's and adding Lyrics). I believe the MusicBrainz Picard I use is 1.3.2, but isn't there a beta version somewhere with fixes to some of the issues you've been working on (and do you happen to have a link to it)?

2. Also, I have the MusicBrianz Picard add-on to remove the "Featured" Artists. But because of your work in Kodi, would I be able to re-tag my music and remove this add-on (so Featured Artists are included in the tags agaiin)?

3. With the above in mind, do you have any "Recommended" Picard plug-in's?

4. And finally (for now...) does Genre Support still require a fix in advanced settings (space / space, or space ; space) for multiple Genre's?
Reply
#5
(2016-09-02, 13:30)DaveBlake Wrote:
Quote:2. Is there still the Bug where you should add all your Compilations after your regular CD's?
Not sure, so going to say yes, sorry not enough hours in the day Sad

This is a work around for an artwork/NFO issue right? I remember it being good advice but not why, so someone remind me.

Kodi depends on the artist having a unique common folder for all their music i.e. arranged in a folder structure of Album Artist/Album/songs, and scarping/art go wonky if this isn't the case.

This is referring to my advice from here: http://forum.kodi.tv/showthread.php?tid=...pid2341616

I renamed my compilations folder to "zzzzzzzzzzzzz compilations" so it gets scanned/scraped as the last folder to avoid that artists, that I have in a seperate folder with local artwork, get scanned/scraped when the compilations folder is reached before the artist folder.
Reply
#6
(2016-09-03, 11:56)DarkHelmet Wrote:
(2016-09-02, 13:30)DaveBlake Wrote:
Quote:2. Is there still the Bug where you should add all your Compilations after your regular CD's?
Not sure, so going to say yes, sorry not enough hours in the day Sad

This is a work around for an artwork/NFO issue right? I remember it being good advice but not why, so someone remind me.

Kodi depends on the artist having a unique common folder for all their music i.e. arranged in a folder structure of Album Artist/Album/songs, and scarping/art go wonky if this isn't the case.

This is referring to my advice from here: http://forum.kodi.tv/showthread.php?tid=...pid2341616

I renamed my compilations folder to "zzzzzzzzzzzzz compilations" so it gets scanned/scraped as the last folder to avoid that artists, that I have in a seperate folder with local artwork, get scanned/scraped when the compilations folder is reached before the artist folder.

Another way to deal with that is to create a music source, but use two paths in the same source i.e. Music\Normal Albums\ and Music\Compilation Albums\ then it will scan the normal albums first

Ideally though, this long standing flaw should be corrected - i.e. if there is dedicated artwork in an artist folder, it should take priority over any scraped artwork.
Reply
#7
Thanks both @DarkHelmet and @AnalogKid, yes, that was the issue I was referring too.

Just want to keep these issues in the forfront so @DaveBlake wont forget these still need to be fixed. 8)

Maybe for Kodi 18...
Reply
#8
(2016-09-04, 18:35)Powerhouse Wrote: Thanks both @DarkHelmet and @AnalogKid, yes, that was the issue I was referring too.

Just want to keep these issues in the forfront so @DaveBlake wont forget these still need to be fixed. 8)

Maybe for Kodi 18...

Always happy for polite and timely reminders Smile

It is tied in with the way NFO are linked to file/folder structure, and is on my (paper based) job list.
Reply
#9
(2016-09-02, 19:58)Powerhouse Wrote: Now on to the new questions (that I knew I would remember).

1. Normally, I use MusicBrainz Picard (for initial tagging), and MediaMonkey for any additional info (normally fixing Genre's and adding Lyrics). I believe the MusicBrainz Picard I use is 1.3.2, but isn't there a beta version somewhere with fixes to some of the issues you've been working on (and do you happen to have a link to it)?

I really know nothing about a beta version of Picard, maybe someone else does?
The only thing I can think that I would like of is that Picard does not add ALBUMARTISTS (note the S) tags by default, you have to add a script to make it. There are issues with embedded images getting lost on tag editing, but you will have to look at their forum for that (and I don't think it has been fixed).

Quote:2. Also, I have the MusicBrianz Picard add-on to remove the "Featured" Artists. But because of your work in Kodi, would I be able to re-tag my music and remove this add-on (so Featured Artists are included in the tags again)?
Yes. Krypton makes a better job of deciphering multiple names and matching them to the mbid tags.

Quote:3. With the above in mind, do you have any "Recommended" Picard plug-in's?
Anyone?

Quote:4. And finally (for now...) does Genre Support still require a fix in advanced settings (space / space, or space ; space) for multiple Genre's?
Artist tag processing is a lot less fussy, and supports multiple item separators that the user can set as they want. But genre is the same as before, so to separate genres from a single tag (i.e. ID3v2.3 tags or before) you need to use the item separator. By default this is space-slash-space, but if you have tagged genre using something else then you need to change that in advancedsettings.xml
Reply
#10
Please see below link to Picard nightly builts, which you could call "beta". Unfortunately, the "embedded picture bug" has not been fixed and albumartistS still require the script. The nightly does include some nice new features though, so worth to check it out.

http://build.oxygene.sk/job/package-picard-win-daily/
Server: Asus Sabertooth Z77 | Intel Core i5 3.4 GHz | 16 GB DDR3 | 128 GB SSD, 82 TB (9 x 6 TB, 7 x 4 TB)
HTPC 1: Raspberry Pi 2 | HTPC 2: Raspberry Pi 2 | HTPC 3: Raspberry Pi
Reply

Logout Mark Read Team Forum Stats Members Help
Best way to update music db 16 -> 170