v18 Duplicate library entries
#1
Music 
I have added replaygain tags to a portion of my library folders, just so I could see how Kodi handles replaygain. If all went as expected (It didn't) my plan was to work my way through all of my music library. I used Foobar2000 to scan and write the replaygain tags. Everything looks and plays fine following this operation within foobar2000. In Kodi (Leia 18.9) I have duplicate album entries for all the folders re-scanned into the library. I used the option "Do full tag scan even when files are unchanged".

So to correct this issue I was thinking of temporarily removing this section of my music library. Do "Clean Library" and then re-add the replaygain folders and re-scan those folders to the library. Will that work?

How do I move forward with replaygain tags over the whole of the library without creating duplicate entries?

EDIT - I was a bit hasty saying "all" folders have duplicate library entries. It is just some. I will do a bit more digging around to see if I can work out why some and not others are duplicated, and reply as and when I have any further info'
Reply
#2
Just adding replay gain tags to your music files will not cause duplicates in the library, neither will moving or renaming files. Foobar2000 must have changed some other tags values on just some of the music files for an album and not the rest.

If you don't want to lose song playback history (playcount, last played date and user rating) then I suggest exporting it before you drop any music sources. It can then be restored by importing once you have recreated your library.
Reply
#3
After doing a bit of digging around I noticed that after writing Replay Gain tags to that batch of music (with Foobar2000) the timestamp on the files had been updated from dd/mm/2013 hh.mm to 19/05/2021 19.mm. As the files have changed, it seems reasonable that the timestamp has been updated

(1) I have removed this batch of music from the library, (2) "Cleaned" the library, (3) re-added the music, (4) "Scanned" items back into the library. I noticed that some of the albums that previously showed up duplicated now showed up with a single library entry, but some still showed up as duplicated.

To get a better idea and to keep a better control of what may be happening I got Foobar2000 to write Replay Gain tags to a small batch of just six music albums. As now expected Foobar2000 updated the timestamp on the files from 29/05/2013 10.18 to 23/05/2021 14.11. I re-scanned these six albums to the Kodi library.  Five of them now appear as duplicated but one of them does not. Each duplicated album shows up one with the former timestamp, and the other with the new (today's) timestamp.  The non-duplicated album shows up with just the new timestamp 23/05/2021 14.11:20. In Foobar2000 the library shows up just fine with a single entry with the new (today's) timestamp.

Does the above throw any light on what is happening within Kodi's database? It seems likely that if I go ahead and write ReplayGain tags to the whole library I may have to rebuild the library from scratch.
Reply
#4
(2021-05-23, 16:58)jac55 Wrote: Does the above throw any light on what is happening within Kodi's database?
Sadly no it does not. Editing tags will of course change the file timestamp and possibly the size slightly too depending on how much metadata you have added or removed. Files having a different timestamp get re-scanned on library update, but that will not create a duplicate entry. It updates the existing entries for the artist(s), album and song.

(2021-05-23, 16:58)jac55 Wrote: It seems likely that if I go ahead and write ReplayGain tags to the whole library I may have to rebuild the library from scratch.
No, this is not normal behaviour something is going wrong for you. I have done exactly what you are trying - added replay gain tag values using Foobar2000 to all my music collection - and had no problems at all. Yes Kodi will rescan these files (and store the relay gain) keeping playback history, but it does not result in duplicates.
 
So if you are getting duplicates then something is going wrong, and I am happy to help get to the cause. A look at your music database with the duplicates may give some clues. If you can upload it on the cloud and PM me a link then I will see if I can spot anything.
Reply
#5
Thanks for offering to look at my database @DaveBlake , but I think I am on to something and would greatly appreciate your input.
When I play a track from a duplicated album and queue the same track from the other duplicate library entry, the first and second library entry have the following info' associated with them

(1) Timestamp 2013-07-21       path    /media/My Book/Shared Music/...
(2) Timestamp 2021-05-19       path    /var/media/My Book/Shared Music/...

I went to my Music "Sources" category. The only source of added music is named "WD MyBookMusic" and I have checked the path (using the context menu and "edit source" option. It is   /media/My Book/Shared Music

When I look at the Music "Files" category I see the following listing (with icons next to each list item)

(music icon)                 Music
(USB connector icon)    My Book
(HDD icon)                  WD MyBookMusic
(+ icon)                      Add music

Now the Music is all stored on a USB 3 connected Desktop Drive. I believe I have added my source WD MyBookMusic via "Browse" > Root filesystem > media > My Book > Shared Music

Now if I have rescanned into the library, albums via the "File category and drilled down to the appropriate folder structure to re-scan items, is it feasible that that gives a different library result than if I had gone via "Sources" WDMyBookMusic and drilled down.  For info' I am using LibreELEC on a Raspberry Pi 4 running KODI Leia 18.9.
It is my intention to remove the duplicated folder struture from the library, "Clean" library, re-add the folders, and re-scan making sure I rescan via the "Sources" category WD MyBookMusic rather than the "Files" category and hope the duplicates are fixed. The cause does seem somehow related to different library paths.

I have not got time to do that right now but will report later.
Reply
#6
The mutiple paths explains the duplicate library entries for sure, the scanning will see the one file as two different files in separate locations with exactly the same tag data. But I don't immmediately know why you have both  /var/media/My Book/Shared Music/.. and  /media/My Book/Shared Music/..

I have used LibreElec on RPi3 in the past but dropped that because of lack of a functioning v19 build, never had path issues with it but was connecting via NFS to NAS drives. No longer have a LE set-up to test, but it is possible that the route taken to the drive will have an impact on path stored. Once the different route paths are stored in the path table then rescanning will check both and changed files i.e. those with a different time stamp will get rescanned and added as a new entry.

I repeat if you want to keep song playback history and user ratings then remember to export it before you do the dropping and cleaning etc.
Reply
#7
Well, my library is sorted now with no duplicates, and back on track to continue adding ReplayGain tags throughout my library.

It must have been the case that when I re-scanned ReplayGain tagged music back into my library I must have inadvertently done so by navigating via the "My Book" option from the list below rather than "WD MyBookMusic " which had been used to create the library. As it was being re-scanned I particularly noticed that the path (displayed in the top right of my monitor screen) was showing  " /media/MyBook/Shared Music..." which is the path for the bulk of my library and the necessary path to keep things in order. I will need to remember this for future reference.

(music icon)                 Music
(USB connector icon)    My Book
(HDD icon)                  WD MyBookMusic
(+ icon)                      Add music

Many thanks for your help and guiding hand @DaveBlake
Reply
#8
(2021-05-24, 13:23)DaveBlake Wrote: But I don't immmediately know why you have both  /var/media/My Book/Shared Music/.. and  /media/My Book/Shared Music/..

Just tested that with LE.  It mounts an external USB drive to /var/media/<drive-name>  but /media is a symlink to /var/media.

xml:
ls -alh
total 17K    
drwxrwxr-x   13 root     root         202 Mar  1 09:24 .
drwxrwxr-x   13 root     root         202 Mar  1 09:24 ..
lrwxrwxrwx    1 root     root           8 Mar  1 09:24 bin -> /usr/bin
drwxr-xr-x   14 root     root        3.5K May 24 17:55 dev
drwxrwxr-x   21 root     root        1009 Mar  1 09:24 etc
drwxr-xr-x    3 root     root       16.0K Jan  1  1970 flash
lrwxrwxrwx    1 root     root           8 Mar  1 09:24 lib -> /usr/lib
lrwxrwxrwx    1 root     root          10 Mar  1 09:24 media -> /var/media
drwxrwxr-x    3 root     root          25 Mar  1 09:24 opt
dr-xr-xr-x  113 root     root           0 Jan  1  1970 proc
drwxr-xr-x   17 root     root         540 May 10 15:31 run
lrwxrwxrwx    1 root     root           9 Mar  1 09:24 sbin -> /usr/sbin
drwxr-xr-x   14 root     root        1.0K Mar 26 17:57 storage
dr-xr-xr-x   12 root     root           0 Jan  1  1970 sys
drwxrwxrwt    3 root     root          60 May 23 12:00 tmp
drwxrwxr-x    9 root     root         124 Mar  1 09:24 usr
drwxr-xr-x    9 root     root         220 Apr 11  2019 var


Both paths are valid and ultimately lead to the same media content but clearly using both leads to disruption.  Not something I was actually aware of until I looked.
Learning Linux the hard way !!
Reply

Logout Mark Read Team Forum Stats Members Help
Duplicate library entries0