duplicates in music library
#31
Thank you all for your comments and suggestions. I do understand tagin issues. However i am having problem with Picard and am trying  how to properly use  it without messing all more than it is.

Im my latest attempt On my development machine

I deleted mymusic72
cleared the music library
and removed the music source
restarted KOdi

Reloaded the music library  I still got 3894 files

Using DB Browser sqlite i looked ar the new mymusic72 and viewing songs and sorted on STR title

Looked carefully at the duplicates and reviewing the records all was identical except for the the album Number under idAlbum there was a different value between the duplicate records strMusicbrainsID was identical

Looked at the same file in picards and the record only shows 1 item

Some duplicates had  one record with strMusicbrainsID the other with out

I need to learn how to use picard correctly (my next project)

However on my HTPC Music library song only show 25 songs out of 3894 but the contains all the albums and artist and looked at all the setting.media.music and they are identical as my laptop who shows all 3894 songs

Until i learn how to properly load, edit, tagg and  manage the records in music brains I will leave all alone (hopefully) 

Once again thank you
Best regards to all
BikerDude
HTPC with 16gb ram M-Audio LT1010 sound card feeding McIntosh 7205 power amp Magneplanar 2.7r and Velodyne dd12. All connected to Sony smart tv.
All media on 2 fully redundant TrueNas VMs installed on Proxmox servers. Running Kodi. Everything works brilliantly until i screw it up
Reply
#32
Hello all 

Update

I took what you all said on board. I went back to learn how to use musicbrainz (MB). 
First I learned that mbrainz is not reliable when tagging large libraries.

Therefore I started to put through MB selected artists folders (i have 694 to do).

If I take Dionne Warwick as an example, I ran it trough MB. ( I decided to start small)

MB and my explorer shows only one record for dionne warwick "(There's) Always Something There to Remind Me" and MB
 shows only one record for the album "The Windows of the World" 

I cleared the music library, I removed the music source i deleted the mymusic72 from userdata. and restarted Kodi

added my music library and after it was loaded it showed 3666 songs Dionne warwick 2 of the 3 songs on that album were duplicated.

I ran immediately a music library update and now under songs I show 3933 records

Before I embarque and run all 694 artist through MB, I would like to get your view and see if I am going through this the right way. If so I will run all 694 artist though MB and proceed with the massive task removing all the track numbers. this will mean in addition to all this, would require rebuilding all my 57 playlists.

Once this is done on the development pc can i copy and past the music db to my nas and have kody point to my nas to read the db and all the other userdata ? 

In viewing  find 2 records with different ID album all the rest of the record is identical. I add here both the db and the log file. Thank you for your help 

https://ufile.io/o8jb47sj
https://ufile.io/txjn3q5h
Best regards to all
BikerDude
HTPC with 16gb ram M-Audio LT1010 sound card feeding McIntosh 7205 power amp Magneplanar 2.7r and Velodyne dd12. All connected to Sony smart tv.
All media on 2 fully redundant TrueNas VMs installed on Proxmox servers. Running Kodi. Everything works brilliantly until i screw it up
Reply
#33
I am mystified in so many ways @BikerDude.
I really don't know why you are having such trouble with this. We must be failing to communicate something fundamental to you, but I don't get what. I will look for some clues in both he data you linked and in what you said.
 
Quote:First I learned that mbrainz is not reliable when tagging large libraries.
I would not say it was unreliable.
If you expect it to automatically look at all your music files and correctly identify the releases automatically then no, it won't do that, it needs some manual checking and selection of the correct release.
If you expect every album of your collection to be in the Musicbrainz database then again you may be disappointed (depending on how diverse or popular your tastes).
But used with just a little discrimination and oversight it is pretty dam good. What is the issue that causes you to make that comment?

I look at the db and see 74 albums that are duplicated. 
Some have one entry with mbids, one entry without. Yet the duplicated songs related to each album seem to point to the same music files, and the log shows these files only being scanned into the library one. No idea how you achieved that.
Some albums have same name but the music files have been tagged with different releases hence are split across mutiple albums entries. The songs from an album need to be tagged consistently.

There are songs on these albums (and others) that all have the same track numbers, that will cause problems, although I don't expect duplication to be the issue. But are they not tagged with correct track number??
Quote:If so I will run all 694 artist though MB and proceed with the massive task removing all the track numbers.
WhatHuh
Picard would have tagged with disc and track number in the first place, track numbers are needed so do not remove. OTOH why are so many the same?
Quote:this will mean in addition to all this, would require rebuilding all my 57 playlists.
What on earth have you been doing with track number tag?
I can not see any connection between track number tag, and playlists. If you mean how you have named the musc files, then Kodi does not care what numbers you put in that. It is the tag values that matters to Kodi, use Picard to correct that.


Before you embark on a mass tag editing exercise let's see if we can get some of these problem albums tagged correctly and scanned into the library without repeats.

Let's start with the Dionne Warwick folder, just tag those files using Picard. You only have 3 songs, are they really from the release you picked? Or did you just accept the first thing found in the Musicbrainz db? If you don't have whole identifiable releases then maybe using Musicbrainz is just not for you, or for those music files. But don't leave mbid tags mixed in there.

What I do when using Picard is this:
1) add a folder usually containing 3 to 12 subfolders of different albums (depends on how many new things I bought/ripped);
2) click cluster (and based on album name etc. it usually does quite a good job at grouping the tracks into albums)
3) Click Look Up in Browser. That goes of to the web page with what it has found as possible matches and I explore those until I am happy.
4) On the web page click the green tagger button. That transfers the results from the web browser to the right hand side of the Picard screen
5) Drag and drop from the LHS to the RHS
6) Look at the tagging is going to apply, files and tracks matched correctly?
7) Save those tags.

If I was clever I would make a video.

Now you already have mbid tags present (including mixed up release etc.), so you may need to drag RHS to LHS to unmap music files from MB db results and then follow my instructions.

The way to proceed is to work on just a few albums, and only add those as your music source (not all of the Z: drive). Do that starting from an empty library, and see if you get only one copy of each album and one of copy of each song.

The duplicated albums (pick a few to start with) are:
Code:

America, History: America's Greatest Hits
Annie Lennox, The Annie Lennox Collection
Antonello Venditti & Francesco De Gregori, Theorius campus
Big Brother & the Holding Company, Cheap Thrills
Bob Marley and The Wailers, Mellow Mood
Bud Powell, Time Waits: The Amazing Bud Powell, Volume 4
Cream, The Best of Cream
Crosby, Stills, Nash & Young, So Far
Diana Krall, The Very Best of Diana Krall
Dionne Warwick, The Windows of the World
Dire Straits, Money for Nothing
Dusty Springfield, The Very Best of Dusty Springfield
Eagles, Their Greatest Hits 1971-1975
Ella Fitzgerald, The Cole Porter Songbook
Elton John, Goodbye Yellow Brick Road
Enya, Watermark
Eric Tingstad, Nancy Rumbel & David Lanz, Woodlands
Fausto Papetti, 18a raccolta
Gipsy Kings, Gipsy Kings
Herb Alpert & The Tijuana Brass, Classics, Volume 1
Inti-Illimani, The Flight of the Condor
Inti-Illimani, Viva Chile
Isaac Hayes, Theme From Shaft
Jethro Tull, Aqualung
Joe Cocker, Civilized Man
Joe Cocker, Greatest Hits
Joe Cocker, Joe Cocker
Joe Cocker, Luxury You Can Afford
Joe Cocker, One Night of Sin
Joe Cocker, Sheffield Steel
Joe Cocker, Something To Say
Joe Cocker, Stingray
Joe Cocker, Unchain My Heart
Joni Mitchell, Blue
Led Zeppelin, Led Zeppelin
Led Zeppelin, Led Zeppelin II
Lynyrd Skynyrd, Skynyrd's Innyrds: Their Greatest Hits
Madonna, Like a Prayer
Max Greger & His Orchestra, Oscar-Melodien zum Tanzen
Michael Jones, Pianoscapes
Moby, 18
Moby, Play
Pat Benatar, Best Shots
Phil Collins, Hello, I Must Be Going!
Pink Floyd, The Wall
Rondò Veneziano, Visioni di Venezia
Sam Cooke, The Best of Sam Cooke
Spyro Gyra, Morning Dance
The 5th Dimension, Greatest Hits on Earth
The Animals, The Singles Plus
The Beach Boys, Still Cruisin'
The Beatles, 1962-1966
The Beatles, A Hard Day's Night
The Beatles, Abbey Road
The Beatles, Beatlemania! With The Beatles
The Beatles, Beatles for Sale
The Beatles, Help!
The Beatles, Let It Be
The Beatles, Magical Mystery Tour
The Beatles, Past Masters, Volume One
The Beatles, Please Please Me
The Beatles, Revolver
The Beatles, Sgt. Pepper's Lonely Hearts Club Band
The Beatles, The Beatles
The Crazy World of Arthur Brown, The Crazy World of Arthur Brown
The Everly Brothers, Cadence Classics: Their 20 Greatest Hits
The Hollies, 20 Golden Greats
The London Symphony Orchestra, Classic Rock 2: The Second Movement
The Turtles, Happy Together
The Yardbirds, The Collection
Tom Petty and the Heartbreakers, Damn the Torpedoes
Tommy James & The Shondells, The Best of Tommy James & The Shondells
Tracy Chapman, Crossroads
U2, U218 Singles
Reply
#34
@BikerDude

1. Can you confirm that you have created the settings as specified in section 4.2 and 4.3 here... https://kodi.wiki/view/Music_tagging#Tagging
I did mention it earlier, but you have not acknowledged my comment. Could be a reason for the ongoing bad tagging.

2. Instead of re-scanning your entire library after fixing one album, just scan that one album you fixed. It is certainly a lot easier to fault find a few lines than trying to dig through almost 4000 lines.
Setup 3 folders... f:\Original Bad\, f:\KodiSource, and f:\Fixed or however you choose to name them, you get the idea. Pick an album. Move it to the KodiSource. Fix it as suggested by daveblake, scan it, if it is all ok, move to the third folder.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#35
(2019-09-15, 01:26)Karellen Wrote: 1. Can you confirm that you have created the settings as specified in section 4.2 and 4.3 here... https://kodi.wiki/view/Music_tagging#Tagging
I did mention it earlier, but you have not acknowledged my comment. Could be a reason for the ongoing bad tagging.
 

4.2 is the going to be the key here.  make sure you get rid of those existing tags.  for the love of god.
Reply
#36
Hello all

Thank you very much for your input it is greatly appreciated

I did not have my setting as indicated in 4.2 and 4.3

This is exactly what i did with dionne warwick folder. I am not sure that i deleted the old tags in this case

I Did create 2 additional shares on my NAS one called development and the second called musicbrainz

what will will do now
  1.  Move all my files from my music share to the development share
  2. set MB to clear all tags as in 4.2
  3. in MB  i will chose one artist from the development share
  4. cluster 
  5. lookup or scan
  6. save records in musicbrainz share 
  7. copy from musicbrainz share to music share
  8. clear music library in kodi
  9. remove music source in kodi
  10. delete mymusic72db from userdata
  11. open kodi
  12. add music source from my music share to library (only contains one folder for now) 
  13. update music library again and see what happens
  14. if all is ok then
  15. then select new artist and repeat process 1 to 14
  • if duplicate  find out why
this is going to be a massive task. however at the end i will become a savvy person on tagging
Best regards to all
BikerDude
HTPC with 16gb ram M-Audio LT1010 sound card feeding McIntosh 7205 power amp Magneplanar 2.7r and Velodyne dd12. All connected to Sony smart tv.
All media on 2 fully redundant TrueNas VMs installed on Proxmox servers. Running Kodi. Everything works brilliantly until i screw it up
Reply
#37
Thank you for your thorough analysis  

dionne warwick was processed as you recommend all the other no. In the case of DW i was careful to correctly tag and save those 3 records. In pickard all looked fine and as it should be (according to my limited knowledge). Yet in the music db there are 2 album IDs for these tracks. I do net see where the second album ID comes from.

I will however process DW once again though MB

my objective of removing the track number is to help in sorting the songs in the playlists (I my have to re think this objective ) 

once again thank you
Best regards to all
BikerDude
HTPC with 16gb ram M-Audio LT1010 sound card feeding McIntosh 7205 power amp Magneplanar 2.7r and Velodyne dd12. All connected to Sony smart tv.
All media on 2 fully redundant TrueNas VMs installed on Proxmox servers. Running Kodi. Everything works brilliantly until i screw it up
Reply
#38
@BikerDude if there is only one album, then there shouldn't be three records.  Perhaps this is where you are going wrong ?  If Picard is showing 3 different albums on the top right, then it thinks that there are 3 different albums that those tracks are on and will select the tags accordingly.  However, if this is because the tags are wrong in the first place and there is indeed only the one album then you need to expand those albums in the top right and click and drag the track(s) that are in the wrong album into the right one.

BTW, you do not need to quote the previous reply each time.  It makes finding what you actually have to say tedious because I have to scroll for ages to get past a quoted post that I've already read.  Just saying Wink
Learning Linux the hard way !!
Reply
#39
Thank you @black_eagle 

in the DW album in question i only have 4 tracks from that album

I started the fix following my process here listed above for DW and it worked fine I have 1 album 4 tracks correctly tagged

I believe what was happening was that i did not clear previous tag information before saving

thanks to this forum thread i am fixing my library which i am discovering has many issues, so i am biting the bullet  and rebuilding all my library.

noiw i need to learn how to properly save the MB files so they save in artist + album + track on my share  

Once again Thank you
Best regards to all
BikerDude
HTPC with 16gb ram M-Audio LT1010 sound card feeding McIntosh 7205 power amp Magneplanar 2.7r and Velodyne dd12. All connected to Sony smart tv.
All media on 2 fully redundant TrueNas VMs installed on Proxmox servers. Running Kodi. Everything works brilliantly until i screw it up
Reply
#40
You can get Picard to automatically move your files for you when you save an album.  If you look under Options -> Options... -> File Naming, you put in the root path in the box at the top by either browsing to the top level of your share or typing it in.  Note, you have to tick the Move files when saving box first.

It'll give you some examples at the bottom of that window showing how files will be saved, but basically, its /root_path/artist/album/tracks   Once you're sure that you've set the root path correctly, experiment by saving just one album and checking that it appears where it should.
Learning Linux the hard way !!
Reply
#41
Thanks Mate for your help

I have been working at this since yesterday and I have completed 16 artists so far. I also learned the process for tagging each individual artists in a compoliations and saving it as the artists original album. 

at the moment all is fine with the exceptions that although KODi does recognise the artists and put them in their own folder when in artist view, it insists of adding an additional folder for various artists with the same albums and tracs. Yet in the songs view all songs are correctly identified and in viewing mymusic72 all tracks are correctly identified by artist and album with no mention of various artist. I will just keep plugging away.

but so far so good Thanks
Best regards to all
BikerDude
HTPC with 16gb ram M-Audio LT1010 sound card feeding McIntosh 7205 power amp Magneplanar 2.7r and Velodyne dd12. All connected to Sony smart tv.
All media on 2 fully redundant TrueNas VMs installed on Proxmox servers. Running Kodi. Everything works brilliantly until i screw it up
Reply
#42
(2019-09-16, 09:02)BikerDude Wrote: I also learned the process for tagging each individual artists in a compoliations and saving it as the artists original album. 
No, don't do things like that!!
Tag things with the album/release that they are - either the album is a compilation album or it isn't.

I am beginng to suspect that you just have a big collection of haphazarrd music files from all over that you have put under artist folders. You are going to have a hell of a time matching those to releases in Musicbrainz, and the inconsistent jumble that could result would explain the mess you are getting with the library.
(2019-09-16, 09:02)BikerDude Wrote: at the moment all is fine with the exceptions that although KODi does recognise the artists and put them in their own folder when in artist view, it insists of adding an additional folder for various artists with the same albums and tracs. Yet in the songs view all songs are correctly identified and in viewing mymusic72 all tracks are correctly identified by artist and album with no mention of various artist.
Each song has song artist(s) (ARTIST tag) and a album artist(s) (ALBUMARTIST tag). On a normal album they are often exactly the same, or there may be additional song artists on some songs where the main artist has done a duet and another artist has made a "guest appearence". Then some albums are various artist compilations e.g. Club Mix '96 - every song is by a different song artist, and so the album artist is "Various Artists".

The artists node (what you call the folder in artists view) by default shows both song and album artists, although you can change a setting just to see album artists (those artists that are credited with an album, not just some songs on it). If you have compliations, which you obviously do, then you will see "Various Artists" as an album artist.

 I hope that by working through those albums that I listed you will begin to see what was wrong with the tagging, and hence avoid the issues.
Reply
#43
Hi all

@DaveBlake thank you for your comment. I have not mastered how to copy a sentence of your message and quoted here. So this is how i will try to respond:
to your second point first,
I fully agree with you on this point however, i do not want to see various artists (if possible) when i look under music artist view. Therefore, I elected to to invest the time to research the artist original album for that song and allocate that song to the original album

In MB i discovered that once you load a compilation into the middle clustered and unclustered section and click on lookup, MB will add to the right column a selection of possible albums containing one or more tracks from the compilation. These albums can be from the artist who ios also the albumartist or other compilation albums where the the albumartist is various artist. If not I scan the remaining tracks in the middle section and again MB will add to the right side suggested albums. all this is fine.
If i see an album in the right column where the artist is also the albumartist, it may contain 1 or 2 songs with green bars and all the other songs and the rest are with music notes, and in various artist albums i see the remaining songs of my library with green rectangle.

I select the songs in the various artist album and i drag it to the artist album and in the artist album now that from went from having a music not to a green rectangle. I tag the artist album and save it to a different share. 

If there is not artist album in the right column I lookup the song on MB it open a webpage where it lists all albums which contained that track. If i see an album from the artist i select that album and tg it. MB will add that album to the right column and all the tracks are with music notes.

I then take the tracks contained in the various artists album that have the track with the green rectangle and drag and drop it on the same nane track in the artists album. the track in the artists album now has green rectangle. now I save the album in MB. MB saves the album in an MB share.

I then try to play the track from that share. If all is alright I copy the files to a development share under the artist folder. 

I then go to Kodi and update my music library. The artist now appears with the album and the track when I go to music artist. I then go to music songs and have a running count of the number of songs. I then look ar mymusic72 db and I see that all is fine

So Far no duplicates, no Ibum artists no cyrillic named artist.

There are still tracks that will remain as albumartist various artist. In many cases although I am saving the compilation, Kodi still recognises the artists in the music artist section. sometime it doesn't. I guess I will have to live with this.

I only use the artist, the playlist, the songs and the top100 views from kodi and i care to have the artist thumbnail. I usually play from playlists very very seldomly will i play from the artist view. My 57 playlists cover most of my moods, situations events or for driving my car or my motorcycle. 

I will only play from the artist view if i need to test my speakers (because I may be starting to have issues with them)

After a 10 hours I am up to the F at the moment and so far all is good. 

once again thank you for your input and patience
Best regards to all
BikerDude
HTPC with 16gb ram M-Audio LT1010 sound card feeding McIntosh 7205 power amp Magneplanar 2.7r and Velodyne dd12. All connected to Sony smart tv.
All media on 2 fully redundant TrueNas VMs installed on Proxmox servers. Running Kodi. Everything works brilliantly until i screw it up
Reply
#44
(2019-09-17, 11:12)BikerDude Wrote: I have not mastered how to copy a sentence of your message and quoted here.
Image
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#45
(2019-09-17, 11:12)BikerDude Wrote: i do not want to see various artists (if possible) when i look under music artist view.

You can turn off seeing artists that are featured artists and not album artists by going to Settings -> Media -> Music -> Show song and album artists and turning that OFF.  Then you will only get album artists in the artist list and any artists that are on a compilation album will not be shown.

Trying to fix it in the tags is really not the way to go, unless your tags are either really messed up so Picard is misidentifying stuff or just plain missing.
Learning Linux the hard way !!
Reply

Logout Mark Read Team Forum Stats Members Help
duplicates in music library0