Solved First two tracks are missing from database after scan
#1
I've got a very strange problem:

The first two tracks from the first album scanned (with changes) are missing from the database. It doesn't matter if they are new files or only with metadata changes.
They are found and added on a re-scan, but only after changes have been made AND in case they are not again the first two files to be scanned (new/changed, alphabetical order).
If no changes are made, they do not reappear on further scans.

The problem is completely reproducible, but there are no errors logged (no warning, no information). No files are locked (no files opened with any other application).

The files are located on NFS file system (Gbit LAN). Folder structure is Artist/Album (Year)/01 Track 1 title....
I'm using Kodi 19.1.0 on Ubuntu 20.04.3 LTS. It is possible that this problem has occurred before on v18. Some tracks are also missing in the old database.

I have no idea what causes the problem or what to do, and I hope for help!

(The Problem was mentioned in https://forum.kodi.tv/showthread.php?tid...pid3056815, but I have run some tests since)
Kodi v19, Ubuntu 20.04.3 LTS, Music tagging with MusicBrainz Picard 2.3.1
Reply
#2
This does sound strange (as I said on the other thread).

First to rule out metadata issues, can you use a clean installation of Kodi on any device using a local (SQLite) music db, and scan just that album folder into the music library. Do it with debug enabled and make a log. Are all the tracks added to the library? If not then post both log file and MyMusic84.db file to the cloud and links here.
Reply
#3
(2021-09-09, 19:48)DaveBlake Wrote: This does sound strange (as I said on the other thread).

First to rule out metadata issues, can you use a clean installation of Kodi on any device using a local (SQLite) music db, and scan just that album folder into the music library. Do it with debug enabled and make a log. Are all the tracks added to the library? If not then post both log file and MyMusic84.db file to the cloud and links here.
Okay, I will install Kodi with local sqlite on a separate system with debug logging enabled and scan only one folder into the library - supposedly tomorrow.
Thanks for helping!
Kodi v19, Ubuntu 20.04.3 LTS, Music tagging with MusicBrainz Picard 2.3.1
Reply
#4
(2021-09-09, 21:06)ernstlx Wrote:
(2021-09-09, 19:48)DaveBlake Wrote: This does sound strange (as I said on the other thread).

First to rule out metadata issues, can you use a clean installation of Kodi on any device using a local (SQLite) music db, and scan just that album folder into the music library. Do it with debug enabled and make a log. Are all the tracks added to the library? If not then post both log file and MyMusic84.db file to the cloud and links here.
Okay, I will install Kodi with local sqlite on a separate system with debug logging enabled and scan only one folder into the library - supposedly tomorrow.
Thanks for helping!
Can't wait til tomorrow: It happened again.
https://paste.kodi.tv/okokaxigos.kodi
Where should I best put MyMusic82.db? (it's 82 on my other system too)
[DB Link hidden so only mods can see]
Kodi v19, Ubuntu 20.04.3 LTS, Music tagging with MusicBrainz Picard 2.3.1
Reply
#5
Ah ha, a SQL error while trying to save "NaN" as a float rating value (note NOT user rating but the other value of little use for songs) , hence these songs don't get added to the library. Not clear yet why these have "NaN" but not the rest, and it does NOT come from tag data AFAIK so nothing related to the files.

Could be something related to Ubuntu 20.04.3 and std::max and minHuh? @black_eagle  may need you to have a look at what Linux does.
Reply
#6
(2021-09-10, 09:56)DaveBlake Wrote: Could be something related to Ubuntu 20.04.3 and std::max and minHuh? @black_eagle  may need you to have a look at what Linux does.

Sure, although I have built my library probably over 100 times when testing various things and never encountered this.  Also, I think if it were OS specific we would probably have seen more reports of it.  Not to say it isn't OS specific, just I think it's unlikely.

Also, doesn't 'rating' come from scraping, rather than the tags?  I don't see any scraping in that logfile so I'm at a loss at the moment to understand how that float value can be 'not a number'.  Willing to try and reproduce it if I can.
Learning Linux the hard way !!
Reply
#7
(2021-09-10, 20:06)black_eagle Wrote: Also, doesn't 'rating' come from scraping, rather than the tags?  I don't see any scraping in that logfile so I'm at a loss at the moment to understand how that float value can be 'not a number'.  Willing to try and reproduce it if I can.
In my case, its rating from tags, not scrapers. This seems to be quite unusual.
Kodi v19, Ubuntu 20.04.3 LTS, Music tagging with MusicBrainz Picard 2.3.1
Reply
#8
(2021-09-10, 20:06)black_eagle Wrote: Also, doesn't 'rating' come from scraping, rather than the tags? 
ernstlx Wrote: In my case, its rating from tags, not scrapers. This seems to be quite unusual.
Let's clear this up.
Albums and songs both have rating (a float) and user rating (an integer)

Song user rating value is initailly read from music file tags but can then be edited in various ways.
Album user rating value is initailly taken from song value of 1st scanned song but can also come from scraping NFO and then be edited in various ways.

Album rating comes from scraping.
Song rating doesn't come from anywhere (so why does it exist?). It can be set via JSON API or via Python.
In effect song rating should always be 0 when the song is added to the library.

That these first 2 tracks somehow have "NaN" implies Kodi fails to initialise a record somewhere, but like @black_eagle says why this has not happened regularly if that is the case.

The rating value does not come from the music file so I can't see how it could be the cause of the issue, but maybe it is worth looking at that actual file. Grasping at straws here, but can you upload one of them somewhere @ernstlx?
Reply
#9
(2021-09-11, 08:36)DaveBlake Wrote: The rating value does not come from the music file so I can't see how it could be the cause of the issue, but maybe it is worth looking at that actual file. Grasping at straws here, but can you upload one of them somewhere @ernstlx?
The example album: https://www.ernstlx.com/eigene/Volume_5.zip

But it's not file dependent. It happens to all albums. or more precisely, to the first 2 tracks of the first (new/modified) album scanned into the library. It even happens to files, tagged a long time ago, without rating enabled.
Kodi v19, Ubuntu 20.04.3 LTS, Music tagging with MusicBrainz Picard 2.3.1
Reply
#10
(2021-09-10, 20:06)black_eagle Wrote: Also, I think if it were OS specific we would probably have seen more reports of it.  Not to say it isn't OS specific....
There are tracks missing in the old database (MyMusic72.db from v18) - the OS upgrade happened some time before the Kodi upgrade.
Kodi v19, Ubuntu 20.04.3 LTS, Music tagging with MusicBrainz Picard 2.3.1
Reply
#11
Thanks for the files.
(2021-09-11, 09:27)ernstlx Wrote: But it's not file dependent. It happens to all albums. or more precisely, to the first 2 tracks of the first (new/modified) album scanned into the library. It even happens to files, tagged a long time ago, without rating enabled.
So you are saying that if you start with a fresh installation of Kodi, local (empty) music library, and add a music source to scan one (or more) music files into this library then the first 2 music files have this problem regardless of what files they are? Can you check that with some different albums, a single album or maybe 2 or 3 (in a folder added as source), make debug logs too.

That does sound like an initialisation issue in the code, but I don't get why only you have this problem. I also can't see any obvious mistakes in the code. Hum.... So hard to fix issue that I can't reproduce.
Reply
#12
Well whatever it is, it's not those files because they scan in fine here. I can't reproduce it either Sad  I dropped my db to start fresh and just added that album as a source and then scanned it in.  No issues.   For ref for @ernstlx my OS is 

xml:
Distributor ID:    Ubuntu
Description:    Ubuntu 20.04.2 LTS
Release:    20.04
Codename:    focal

Kernel is 5.4.0-40-generic.  Can't see why that might make a difference though.  I'm intrigued by the "first two tracks of an album".  Why not all of them, why not just the first?  Meh.....a puzzle indeed !!
Learning Linux the hard way !!
Reply
#13
(2021-09-11, 09:58)DaveBlake Wrote: So hard to fix issue that I can't reproduce.
I can't reproduce with the supplied album. Win10
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#14
(2021-09-11, 10:18)black_eagle Wrote: Well whatever it is, it's not those files because they scan in fine here. I can't reproduce it either Sad  I dropped my db to start fresh and just added that album as a source and then scanned it in.  No issues.   For ref for @ernstlx my OS is 

xml:
Distributor ID:    Ubuntu
Description:    Ubuntu 20.04.2 LTS
Release:    20.04
Codename:    focal

Kernel is 5.4.0-40-generic.  Can't see why that might make a difference though.  I'm intrigued by the "first two tracks of an album".  Why not all of them, why not just the first?  Meh.....a puzzle indeed !!

My kernel version is 5.4.0-84-generic.
Kodi v19, Ubuntu 20.04.3 LTS, Music tagging with MusicBrainz Picard 2.3.1
Reply
#15
(2021-09-11, 11:04)ernstlx Wrote: My kernel version is 5.4.0-84-generic.

OK, not sure how that might make a difference but I guess at this stage anything is possible so I will see if I can upgrade to the same kernel version and test again.
Learning Linux the hard way !!
Reply

Logout Mark Read Team Forum Stats Members Help
First two tracks are missing from database after scan0