duplicates in music library
#1
Hello all

I was currently running KODI 18.2 and all was fine. today I upgraded to 18.3. 

Normally i had 3.3 to 3.4k songs in my library organised by artist/album/track

After upgrading to 18.3 i found my self with over 5k songs. I looked at my songs in music and i see many duplicate entries with same artist/album/track/time.
first

I cleared the library and removed the source
then I loaded it up again (always in 18.3) 
now i had 4.9K songs still some with duplicates
then I deleted 18.3 and reinstalled 18.2. Cleaned the library and removed the source
I added the source again and after loading my music I have 4.2k songs
still i have duplicate entries

I checked my NAS music shared folder

I searched for each duplicate shown in my KODI music library on my NAS. In each case I did not find a duplicate file.
Furthermore I scanned the share folder for duplicate with ccleaner and it did not find duplicate files.

How can I resolve this

Thank you in advance for your response
Best regards to all
BikerDude
HTPC with 16gb ram M-Audio LT1010 sound card feeding McIntosh 7205 power amp Magneplanar 2.7r and Velodyne dd12. All connected to Sony smart tv.
All media on 2 fully redundant TrueNas VMs installed on Proxmox servers. Running Kodi. Everything works brilliantly until i screw it up
Reply
#2
Hello all

In addition to the above I checked to source in the file section of the music and there is no duplicate . hence this only occurs when building the library

Once again thank you
Best regards to all
BikerDude
HTPC with 16gb ram M-Audio LT1010 sound card feeding McIntosh 7205 power amp Magneplanar 2.7r and Velodyne dd12. All connected to Sony smart tv.
All media on 2 fully redundant TrueNas VMs installed on Proxmox servers. Running Kodi. Everything works brilliantly until i screw it up
Reply
#3
I really can't guess what has happened from what you have said. Quickest way forwards is if you can let me have a copy of your MyMusic72.db and sources.xml files. Put them on a shared location e.g. GDrive and PM me with a link, or post link here and the mods will obscure it so only team members can see it.

I spotted the Maggies in your signature. I have been a fan of magneplanar speakers since 1988, recently upgrading from my much loved SMGa to a pair of  1.7i, wow what a soundstage!
Reply
#4
(2019-07-30, 09:22)DaveBlake Wrote: I really can't guess what has happened from what you have said. Quickest way forwards is if you can let me have a copy of your MyMusic72.db and sources.xml files. Put them on a shared location e.g. GDrive and PM me with a link, or post link here and the mods will obscure it so only team members can see it.

I spotted the Maggies in your signature. I have been a fan of magneplanar speakers since 1988, recently upgrading from my much loved SMGa to a pair of  1.7i, wow what a soundstage!
Hi Dave thank you for your prompt reply and willingness to help. I copied the mymusic72.db  and put them on a drive. How can i share this with you?

FYI: i went through all the 575 artists every album (one by one) and eliminated some duplicates. Probably 50

I had my maggies since 1990 on a citation 19, and 7 years ago i migrated to the 7205 and 5 years ago added the velodyne. in each case there was a marked improvement on the soundstage, details in the music and the richness of the sound. I only use the sub for movies because i feel that a sub on the music distorts the presence. (i have a very eclectic taste in music and yes some music should benefit from a sub.

regards
Best regards to all
BikerDude
HTPC with 16gb ram M-Audio LT1010 sound card feeding McIntosh 7205 power amp Magneplanar 2.7r and Velodyne dd12. All connected to Sony smart tv.
All media on 2 fully redundant TrueNas VMs installed on Proxmox servers. Running Kodi. Everything works brilliantly until i screw it up
Reply
#5
Once you have your files on the cloud somewhere either PM me a link to that somewhere, or if you post it in is thread then I'll edit it out so it can't be seen.

The planar speakers are thought of as "light" on bass compared to those having a big wooden box, but I have always kept away from subwoofers. The tech has come on, but I would still want a home demo to prove my ears liked the concept and that is hard to get now I live rurally. Bought the 1.7i pair without demo (the first time in 40 years I bought kit without hearing it and many competitors), but it thankfully came good. I tried for over 2 years to get a demo, then had a mad moment! I don't recommend such rashness to anyone else. The right power amp makes a huge difference, not only to soundstage etc. but to that bass too, as does getting the Maggies away from a back wall (wish the house was bigger). My amp is bespoke prototype but is right up there for sound quality, I get more base out the Maggies than I have any right to.
Reply
#6
Sorry for my ignorance Dave. I never used the cloud.

I have the files 

Which cloud ? how?

The Maggies have natural base and for most music is plenty and more than adequate. The difference between between 1.7 and 2.7 is greater base and a greater presence but very clear and detailed. I'm happy that you are enjoying. I am building a media room mostly for watching movies. I will power old AR10's (rebuilt) with my old citation and build another HTPC. The i will get rid of the velodyne.
Best regards to all
BikerDude
HTPC with 16gb ram M-Audio LT1010 sound card feeding McIntosh 7205 power amp Magneplanar 2.7r and Velodyne dd12. All connected to Sony smart tv.
All media on 2 fully redundant TrueNas VMs installed on Proxmox servers. Running Kodi. Everything works brilliantly until i screw it up
Reply
#7
cloud - dropbox or google drive probably.  If you have an android phone it'll likely have a google account on it and you can use those details in a browser to log in and upload and share the files.  Alternatively, you can sign up to dropbox with an email address and get 2Gb of free storage here. There are plenty of other free providers though if you want to use something different.
Learning Linux the hard way !!
Reply
#8
If you don't have any of those "sign-up" services, then try this... https://uploadfiles.io/

Upload then post the link here. Edit your post and delete the link if you do not want it made public. Mods can still see the link.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#9
(2019-07-30, 22:26)Karellen Wrote: If you don't have any of those "sign-up" services, then try this... https://uploadfiles.io/

Upload then post the link here. Edit your post and delete the link if you do not want it made public. Mods can still see the link.
Thank you for responding and helping me with this issue. Here are the 2 files required. the first one id the db and the second is the xml. During this process i am learning how to manipulate the db. I downloaded db browser sqlite and with this i am correcting many of the entries that have been caused by me ripping cd and records. Once done woould i be able to replce the db and the xml on all my pc that are running kodi?

Edit: links obscured

Regards
Best regards to all
BikerDude
HTPC with 16gb ram M-Audio LT1010 sound card feeding McIntosh 7205 power amp Magneplanar 2.7r and Velodyne dd12. All connected to Sony smart tv.
All media on 2 fully redundant TrueNas VMs installed on Proxmox servers. Running Kodi. Everything works brilliantly until i screw it up
Reply
#10
(2019-07-31, 01:03)BikerDude Wrote: I downloaded db browser sqlite and with this i am correcting many of the entries that have been caused by me ripping cd and records. Once done would i be able to replce the db and the xml on all my pc that are running kodi?
Manually manipulating the db is a really bad idea @BikerDude Sad
It sounds like your music file tagging is a mess, that is what you are best putting your efforts into correcting rather than messing with the db. Manual db updates are easy to get wrong unless you know what you are doing, and will be wiped out by any library update (should you add more music to your collection). Fix the tagging!

But I have yet to have a look at your db, I will do that later today. I should be able to point you at the tagging issues from that. I will also lok for anything else a miss.
Reply
#11
(2019-07-31, 07:15)DaveBlake Wrote:
(2019-07-31, 01:03)BikerDude Wrote: I downloaded db browser sqlite and with this i am correcting many of the entries that have been caused by me ripping cd and records. Once done would i be able to replce the db and the xml on all my pc that are running kodi?
Manually manipulating the db is a really bad idea @BikerDude Sad
It sounds like your music file tagging is a mess, that is what you are best putting your efforts into correcting rather than messing with the db. Manual db updates are easy to get wrong unless you know what you are doing, and will be wiped out by any library update (should you add more music to your collection). Fix the tagging!

But I have yet to have a look at your db, I will do that later today. I should be able to point you at the tagging issues from that. I will also lok for anything else a miss. 
Thank you for your response and the effort you are putting into this

I feel comfortable in filling in the blanks for Albumname artistname and artistsort. and i do have the time and patience to do this. (the benefits of being on a pension) 

Example: certain songs are tagged as IBUM artist. Yet the underlying file on my NAS is sorted unders artist/album/song, as all the others artists, in a jefferson airplane or jefferson starship. how this happens i do not understand but i see i can correct these fields effortlessly and safely.

Another example is the record for AISHA DUO and the song deepstar. In the music file section it is recorded correctly once yet in the library section it shows as a duplicate.

Is there a better way to edit these records and make it so it is the same on all my pcs running kodi? If not maybe this could be a development objective for KODI.

I have migrated the playlist successfully to my NAS shared folder and now learning to migrate the thumbnails (need to understand how to point kodi to the nas to fetch the artist and album thumbnails

Once again thank you all for the assist 

I see many users using NAS boxes hence i presume that they are playing there media on multiple pcs
Best regards to all
BikerDude
HTPC with 16gb ram M-Audio LT1010 sound card feeding McIntosh 7205 power amp Magneplanar 2.7r and Velodyne dd12. All connected to Sony smart tv.
All media on 2 fully redundant TrueNas VMs installed on Proxmox servers. Running Kodi. Everything works brilliantly until i screw it up
Reply
#12
Glad that you have time and patience, so please let me encourage you to apply your efforts in the right place. You may "feel comfortable" filling in album name, artistname and artistsort in the db, but it is totally the wrong place to do it.

The arrangement and names of files and folder really don't matter to the way the music library is populated. The artists, albums and songs are all derived from the music file tags.   That is where your problem originates, and where you need to apply your time and patience to restore things.

I have not had chance to look at the data yet, but when you say "certain songs are tagged as IBUM artist" that gives me a big clue. I think that something outside of Kodi has corrupted your music file metadata. This has happened to a couple of other users, one case a faulty memory stick was involved and the other it was some tagging software that could not cope with the Musicbrainz id tags that were there. But I never got to the bottom of what really was the cause, only that the files were corrupted and since Kodi does not write to the media files it was not Kodi that caused it. Such corruption also makes sense of your statement that all was fine until you upgraded from 18.2 to 18.3

Corruption of music file metadata is unexpected and unfortunate, but if you check your file timestamps it may show that some files have been updated unexpectedly and that could be a clue too.

Let me get into the data, and then I will be able to tell you the extent of the problem. But stop editing the db!!!
Reply
#13
@BikerDude

This might be of help... https://kodi.wiki/view/Music_tagging
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#14
@BikerDude while I dig some more to identify exactly which of your music files have issues, you can start with some re-tagging that will ensure artists have their names and not just mbids.

The music files of the following albums need to have an ALBUMARTISTS (note the "s") tag to enable Kodi to match the multiple album artist mbids to individual artist names.
PHP Code:
AlbatrossFleetwood Mac and Christine PerfectZ:\Fleetwood Mac\
MessiahGeorg Friedrich HändelThe Sixteen Choir and OrchestraHarry ChristophersZ:\Georg Friedrich HändelThe Sixteen Choir and OrchestraHarry Christophers\
Mouth MusicBobby McFerrin with Herbie HancockYo-Yo MaZ:\Gioachino Rossini\
Nabucco HighlightsGiuseppe VerdiZ:\Giuseppe Verdi\Giuseppe VerdiTito GobbiElena Souliotis\
NabuccoGiuseppe VerdiZ:\VerdiPhilharmonia OrchestraAmbrosian Opera ChorusRiccardo MutiManuguerraLuchettiGhiaurovScottoObratzsova\
NabuccoGiuseppe VerdiZ:\Giuseppe Verdi\Giuseppe VerdiMaria Callas Vittorio Gui\
Symphonies Nos6Ludwig Van BeethovenZ:\Ludwig van BeethovenBerliner PhilharmonikerSir Simon Rattle\
Symphonies Nos28Ludwig Van BeethovenZ:\BeethovenThe London Classical PlayersRoger Norrington\
Symphony NoCoriolan Egmont OverturesLudwig Van BeethovenZ:\BeethovenThe London Classical PlayersRoger Norrington\
Symphony no9Ludwig Van BeethovenZ:\Ludwig van BeethovenThe Schütz Choir of LondonThe London Classical PlayersRoger Norrington
In particular the Verdi has many album artists e.g. Verdi, Philharmonia Orchestra, Ambrosian Opera Chorus, Riccardo Muti, Manuguerra, Luchetti, Ghiaurov, Scotto, Elena Obraztsova

Sadly Picard does not do this buy default (although it does add an ARTISTS tag to help the same process for artists), but you can set a script value. In Picard options>options>scripting add
$setmulti(albumartists,%_albumartists%)
and that creates the tag correctly. Otherwise you can do it manually adding separate frame values for ALBUMARTISTS or using the standrad item separator in single frame.
Reply
#15
Incorrect tagging (manual inconsitent editing of Picard output) is easier to spot than the corruption issue. Devil
So next I would like you to correct the 38 music files with mbid tags where the album artist name has been manually set to "Various Artists" and does not belong with the album musicbrainz id tag value.

 
PHP Code:
Z:\Bennie Green\Walkin' and Talkin'.mp3Eight classical albumsBennie Green
Z
:\Bill Evans\I Hear a Rhapsody.mp312 Classic Albums1956 1962Bill Evans and Jim Hall
Z
:\Billie Holiday\A Fine Romance.mp3The Essential Billie HolidayBillie Holiday
Z
:\Billie Holiday\Body and Soul.mp3Lady DayThe Master Takes and SinglesBillie Holiday
Z
:\Charlie Rouse\Meci Bon Dieu.mp3Bossa Nova BacchanalCharlie Rouse
Z
:\Charlie Rouse\Samba de Orfeu.mp3Bossa Nova BacchanalCharlie Rouse
Z
:\Charlie Rouse\Velhos Tempos.mp3Bossa Nova BacchanalCharlie Rouse
Z
:\Dizzy Reece\The Rake.mp3Mosaic Select No11Dizzy Reece
Z
:\Django Reinhardt\Tears.mp3Jazz Ballads 7Django ReinhardtDjango Reinhardt
Z
:\Donald Byrd\Funky Mama.mp3Donald Byrd Milestones of a Jazz LegendDonald Byrd
Z
:\Ella Fitzgerald\Cheek To Cheek.mp3100 Songs For a CentennialElla Fitzgerald Louis Armstrong
Z
:\Ella Fitzgerald\Cow Cow Boogie.mp3100 Songs For a CentennialElla Fitzgerald
Z
:\Ella Fitzgerald\Too Darn Hot.flacThe Cole Porter SongbookElla Fitzgerald
Z
:\Etta James\Trust in Me.mp3The Essential Etta JamesEtta James
Z
:\Fats Waller\Ain't Misbehavin'.mp3The Joint Is Jumpin', Fats Waller
Z:\Frank Sinatra\Come Fly With Me.mp3, Masterworks: The 1954-61 Albums, Frank Sinatra
Z:\Frank Sinatra\I'
m Gonna Sit Right Down and Write Myself a Letter.mp3MasterworksThe 1954-61 AlbumsFrank Sinatra
Z
:\Frank Sinatra\I Won't Dance.mp3, Masterworks: The 1954-61 Albums, Frank Sinatra
Z:\Frank Sinatra\Everybody Loves Somebody.mp3, Frank Sinatra, Frank Sinatra
Z:\Freddie Hubbard\But Beautiful.mp3, Open Sesame, Freddie Hubbard
Z:\Grant Green\'Round Midnight.mp3, Retrospective 1961-1966, Grant Green
Z:\Horace Parlan\Us Three.mp3, Us Three, Horace Parlan
Z:\Horace Silver\The Outlaw.mp3, 12 Classic Albums 1953-1962, Horace Silver
Z:\Jackie McLean\Fidel.mp3, Jackie'
s BagJackie McLean
Z
:\Jimmy Smith\Sweet Georgia Brown.mp3At Club Baby Grand Wilmington,  Delaware,  Volume 1Jimmy Smith
Z
:\Lou Donaldson\The Masquerade Is Over.mp3Complete Albums Collection 1953-1959Lou Donaldson
Z
:\Nat King Cole\Quizás,  quizás,  quizás.mp3The Very Best of Nat King ColeCapitol 1943-1965Nat King Cole
Z
:\Nat King Cole\Smile.mp3The Very Best of Nat King ColeNat King Cole
Z
:\Nat King Cole\Unforgettable.mp3The Very Best of Nat King ColeNat King Cole
Z
:\Nat King Cole\Sweet Lorraine.flac20 Golden GreatsNat King Cole
Z
:\Quincy Jones\The Midnight Sun Never Sets.mp3Jazz 'Round Midnight, Quincy Jones
Z:\Sarah Vaughan\Dreamy.mp3, Ballads, Sarah Vaughan
Z:\Sarah Vaughan\Lullaby of Birdland.mp3, Verve Jazz Masters 18, Sarah Vaughan
Z:\The Platters\Sixteen Tons.mp3, Big Artist Series, The Platters
Z:\The Rat Pack\Witchcraft.mp3, Legends - The Rat Pack Collection - 151 Classic Tracks (Deluxe Edition), The Rat Pack
Z:\Wes Montgomery\Somethin' 
Like Bags.mp3So Much Guitar!, Wes Montgomery
Z
:\Wynton Kelly\Blue Moon.mp3Piano InterpretationsWynton Kelly
Z
:\Wynton Kelly\Fine And Dandy.mp3Piano InterpretationsWynton Kelly 
Not sure why you did this or what you wanted to achieve. Anyway editing artist names and not adjusting the related mbid tag can cause a right mess, they come as a pair, so just don't do it. Running these back through Picard should sort out the problem.

If you do not remember doing that kind of manual editing previously then that could be a clue too so let me know. But these music files still need fixing
Reply

Logout Mark Read Team Forum Stats Members Help
duplicates in music library0