[DaveBlake]

@black_eagle I think what @BikerDude wants is not to see the album artist including "various artists", just the song artists. There is no option for that. I have no idea why having a single "various artists" item amongst all the rest on the artists node should be something unwanted, it certainly isn't worth the effort of finding alternative original albums when trying to tag a collection of assorted music files (that could have come from compilations, who knows?).

It is fine to cherry pick songs from an album and not have the other music files, but trying to recognise a suitable album on Musicbrainz that covers the songs you have by an artist (which its what it sounds like you are trying to do) is a fraught endevour. I however does make sense of the things I have seen in you database during this discussion. I worry for you @BikerDude, I don't have a good feeling about how you are going about things. I hope it works out for you, I really do.

[BikerDude]

First thank you all for your assistance and guidance

After an all nighter, I finally concluded what i call phase 1 of this project. 

I now have 711 artist as opposed to 596 before. and 3001 songs in the song section as opposed to 3742 before any duplicates. Therefore I lost 714 songs.

But No duplicates, no Ibum artists, no cyrillic named artists no various artists wah whoo.  This task took less than anticipated It only took 42 hours as opposed a budgeted 60

What I discovered

• the issues started most probably with current spikes via the phone line that last april destroyed my server mother board, modem and a sound card. In addition I had multiple disk failures which destroyed my pool.
  
• In using MB I discovered that it does not load in the unclustered all the files in many cases i loaded part of the files located in a specific folder. In these cases I deleted to folder in my library and in MB
  
• After clustering not all tracks get allocated (fair enough) I scanned them and they got moved to the right column somer still remain unclustered . As in the case above i deemed that this was due to poor tagging and I deleted the files both from MB and my Library
  
• Once the files are clustered i perform a lookup and the files get moved to the right hand side
  
• MB for what ever reason every time it found a Greatest Hits folder in my library MB recognised it as a bee gees greatest hit album and listed the songs for the bee gee album. yet if i played the song listed in the bee gee album with MB "open in player" it played the correct song for the album in my library. Even in this case i elected to delete these files.
  
• I also deleted or did not save any folder MB loaded and showed 2 records for every song in an allocated album (only one instance of the song existed in my library folder.
  
• Once MB discovers "appropriate allocation of the track in a proposed album, in many cases the album proposed iis not the same as the album identified in my library. In this case i performed a lookup in browser on the song and i find the correct albumit belongs to. In this case i use the tagger function on the MB website for the correct album and MB will load that album in the right hand column. I the dragged and dropped the song from the proposed album to the correct album (i deem) and the save the new album and delete the proposed album. This was a massive and tedious undertaking. but it worked and produced results.
  
• I loaded a small number of saved folders into Kodi and kept track of the number of songs loaded and they reconciled.
  

What I learned

• How to use MB
  
• regardless of where the songs come from (USB CD Vinyl or downloaded) always run them through MB before adding them to your library. This will ensure that all tracks will be have a uniform tag and structure. 
  
• always use quality hd such as iron wolf, red or HGST (or any other enterprise grade) drives
  
• enable smart services on your drives with email notification. this will give you a heads up on the status of your drive before they become critical
  
• perform regular scrubs of your drives
  
• keep 1 (preferably 2) HD to replace defective drives
  
• always have MB save results in separate folders or drives
  
• If your solution is screwed up as mine was, I recommend a separate development folder/drive to store your work until you are finished. Once you are finished with your rebuild and happy with it move the files to your production folder/drive
  
• have a complete backup of your media, better yet in addition have a complete mirror (or fully redundant server) of your media. In this case you can still enjoy your media while your rebuild it    
  
• connect your system to an UPS
  

Next steps

• start phase 2 loading of the  missing media
  
• upgrade hd (waiting for new ones next week)
  
• backup and mirror new library
  
• Plan phase 3 rebuilding of playlists
  
• Get a life
  

Once again thank you very much for your patience help, guidance teaching and patience with me it greatly appreciated. you got me out of a nasty pickle.

[BikerDude]

hello all

The ghost is back....sig

Sorry to do this, but i believe i am either cursed, unlucky or plain stupid. 

 From the post above I had completed phase 1 and was happy as a clam because as i mentioned no duplicates, no gibberish no matter how many times i updated the library.

Since then, I went through the remainder of the library and checked if i or MB missed anything from the first pass.

There were leftovers

I started running them through MB and for the most of them I saves then once tagged. and added them to the development share. Certain of these files were salvageable most I deleted. Once this was done I had 3431 records in Music+songs. all was fine with the world.

Yesterday I started to rip the CDs  for the artist i previously deleted. I run the cds through MB and again saved them. i was upto 3832 records

I was tired and fed up with this task last night but after dinner I decided to rip 2 more cds.

The beach boys pet sound and still cruisin.

I loaded the songs in MB and MB automatically suggested 3 albums in the right column. Pert sound. Pet sound and still cruisin. Yes it did show 2 pet sounds . In one of the It had God only Knows as a green rectangle and the rest all had music notes. In the other pet sound it had all the rest of the songs with green rectangles  and God only knows with a music note. So I dragged God only knows from the first pet sound to the second pet sound. At this point the first pet sound had all the songs with music notes so I deleted it and the second had all the songs including god only knows  with green rectangles. Hence I saved the complete pet sound and the still cruisin albums to my musicbrains share.

I Updated Kodi music and it showed 3897 tracks I updated kodi once again (without adding, deleting, or playing anything), The result was I now had 4224 tracks. Sig

I looked at the beach boys folder in kodi and I saw that it had 4 albums. (one track on one album was from an other compilation) and the it had a pet sound album with god only knows, a second pet sound album with the rest of the songs and still cruisin album with no songs. I checked my drive and it only had 2 the 2 albums. I searched in explorer the the group and it showed the 2 albums. I looked  at other artists i loaded during the day and I saw the same situation with the beatles folder.

Frustrated i gave up and went to bed.

 This morning 

I opened kodi and cleared the music library. When I checked the artist view, the artist were this there. I proceeded to remove the share from the music library

• I deleted the 2 artist folder from my drive
  
• i uninstalled kodi including user settings
  
• Re installed Kodi and turned on log function
  
• added the share drive to the music library it now showed 3832 tracks in the song view
  
• I ran the update library again and now the song view shows 4193
  

 I am going through the artists view and looking at all artist with the same issue
Other artists are
america

• Andrew Gold 
  
• Andy Williams
  
• The Animals
  
• Annie Lennox
  

I ran these through MB again and they all seemed fine to me. So i do not understad why these duplicated occur. I suppose that all taggin is correct therfore why would this be happening


I ran all my music through MB as i learned, however i do not understand why this is happening just o some of them . Maybe i am too fastidious and therefore pedant with this because i love my music, maybe i should look a less complicated environment. Not all artist have this issue. 

 Once more thank you for your time allocated to me 
Here i include the log file and the mymusic72 db

[helta]

FOCUS on Musicbrainz Picard.  I mean that is really out of the scope of this forum, but your issues are all related to tagging.

For example:  When you add one album, cluster and lookup, and it offers more then one release with your cluster then Picard has (often times, inappropriately) split your songs into different releases.  In this case, lets pretend your album had 10 tracks.  9 of which are with "Release A", and one track with "Release B".  You can move the matched file from "Release B" and drag and drop it on "Release A" completing your album correctly.  

Another scenario:
If you cluster and lookup another album and it has 15 tracks, but only 14 are matched with one missing.  Right click the release and see if there is another version of the album with 15 tracks, if so choose it.

[HeresJohnny]

Just my 2 cents: The Kodi part of the problem has been addressed exhaustively (as has been the correct use of Picard). What remains seems to be a Musicbrainz application issue and should be taken up with them instead of using our lead programmer to fix your collection.

[BikerDude]

Hello all 

Sorry folks 

I performed what @helta suggested. as i mentioned i spent most of my week re tagging all my music. In the case of my folder of America. it is 1 album with all its songs and the correct and MB has tagged them and I saved them correctly. This is in my opinion not an issue of tagging. What would cause kodi to load 2 version of this album one with album cover one without. . In both cases i can play the songs.

In looking ar mymusic72 db I look at the 2 records for america and the only field that is changing is the albumID. This is not given to KOdi from a tag. what assigns this albumID in kodi i do not know

Thank for your input  and help

[black_eagle]

What would cause kodi to load 2 version of this album one with album cover one without.

Simply, a tagging issue.  Kodi constructs the albums from the data in the tags.

Imagine this - You have 4 books.  You rip all the pages out of all the books but as well as the page number, you write a code number on each page so you know which book it belongs to.  Now put the books back together.  If the code number is wrong for one page, it'll go in the wrong book and the book won't make sense.  Worse, if the code number doesn't match any of the books you will end up with five, because the page has to go somewhere.  If more pages have the wrong code, you could end up with multiple books with the same title, but only some of the pages in each of these books instead of all the pages in one book.

This is the same with Kodi. If the code and the logic was wrong, there would be a mass of threads here complaining about it.  But there aren't.

What I suggest.  Zip up an album (perhaps Pet Sounds would be useful here as you said it is not correct in your Kodi library or the America album) and put it on Dropbox/googledrive/onedrive/Some sort of web hosting where we can download it/ and let someone (me, Dave, whoever) look at the tags that you have and see if we can see where you have gone wrong.  Make sure you zip up the directory that Kodi scans with the album in it.... EG 'music/Beach Boys/Pet Sounds' <- zip the bit in italics.

But honestly, the issue is with the tags that Kodi is reading, not with Kodi itself.

I await your zip file with interest.

[DaveBlake]

Just popping by...
@BikerDude the others are right it is a tagging issue. If you can make a zip of the music files and upload as @black_eagle suggests than that would be interesting, one of the mods will hide the links it soon enough to protect copyright etc. If we could see the tags it is more likely we can point at the issue.

I really would like to know how you are creating duplicates, it seems that seeing the files is the only way to know

[BikerDude]

Hello all

Thank you for your help and for volunteering to review this file.

Even now. I ran these file through MB and as i dragged then in the unclustered column (the center one) MB immediately identified the album and presented it in the right column. I opened the album and all the songs were identified with a tick mark. I refreshed the files and saved them again. MB saved the in my musicbrainz share. I opened the folder and there they were all 12 songs. 

I deleted the old america folder and removed it from the recycling bin. I copied the new america folder from the musicbrainz share to the development share. Opened Kodi cleared the music library, removed the music source, shut down kodi.

I the restarted kodi and added the development share to the music library. Checked the results in the songs view and i had 3844 songs ran an update of the music library an now I have 4226 files. In the artist view I check the America folder and the it is One album with the album cover and all the songs and next to it an other album with the same title and the same 12 songs. I can click a song in either album and they will play. Many other artists show the same result.

Forgot to say that in the mymusic72 db there are 2 record for this album and as far as i can see the only difference between the 2 records id the IDalbum. Id 28 for one and 886 for the other. The strmusicbrainzalbumid and the the strreleasegroupmbid are blank for album id 886

I attache herewith the zip file you requested. 

@Karellen- Link removed

Once again thank you for doing this

[black_eagle]

@Karellen Any chance you can PM me that link, else I can't help any ? Cheers.

If you can't, I understand. I don't want to cause any issues.

[DaveBlake]

Well the taggng of of those music files is absolutely fine (so we were all wrong!!). The cause of your duplicates in that case is a real mystery @BikerDude, but it is something unique to you.

All I can do is poke though the data and see if I can find some clues. Some of your duplicates are tagging issues, for example Ace of Base "Greatest Hits" album you have tagged 2 tracks from one MB release and a third song from another MB release - Kodi rightly adds this as two different albums (some users want to have more than one release of an album in thei library). Same has happened with "After Bathing at Baxter's" by Jefferson Airplane, "Please Hammer Don't Hurt 'Em", "The Essential Michael Jackson" etc.... This is the kind of tagging mix that I feared would happen. It is what Black_Eagle explained with the book shredding analogy.

A file in V:\playlists\ has caused a very odd entry - no artist, no album title - I have to wonder if that is the beginning of the issue. The data scanned after it certainly look suspicious. What is in V:\playlists\

You have added an entire drive as the music source, unless you are going to have music and only your library music on that drive I suggest that you move the music into a folder e.g. V:\music, and keep that "playlists" folder (which contains .xsp files that I would expect to be in your userdata folder not in with the media at all).

Remove that music source in Kodi - goto Music>Files and click remove on context menu etc.
Remove the MyMusic72.db - delete the file
Move the music files and folders (but not the playlists!!!) under a new folder on the V drive
Restart Kodi, add that folder as your music source and let it scan everything.

You will still have duplicate albums because of tagging issues, but there should be only one America album.

Post copy of the MyMusic72.db and log file so I can check what not having V:\playlists folder content scanned into the library does.

EDIT
Can you also post a zipped copy of "Sexy Instrumental Jazz  Relaxation Saxophone Music 2013 Collection.mp3" and "Relaxing and Mellow Pop Music Smooth Jazz Saxophone Songs.mp3". I want to see what tagging they have and experiment (I also need to relax more).

[Karellen]

@BikerDude

You would be surprised at how much discussion has occurred behind the scenes in trying to figure out the cause of the duplication.

We were pretty certain that it was not a Kodi problem, but we were trying to determine what steps you were taking to make it happen. Finally we figured it out. Kodi is too smart!!

For some reason you have playlists in the form of .xsp files in your music folder, which should never be there as they belong in the skin folder. These xsp files point to certain song types (70's, 80's and 90's). Kodi is extracting the xsp file and the songs in the playlist are re-scanned as new items into the library. One library entry is a direct link to the song file, the other entry is scanned via the playlist.

Simple fix which DaveBlake explained above. Move your music out of the root of your drive and into v:\music. You can then create, for example, v:\music\compilations\, v:\music\soundtracks\, v:\music\pop etc.

Oh, and ensure the playlists are kept away from your Sources.

[DaveBlake]

While I was thinking about this @Karellen actually did some conclusive testing (good on him, receive todays gold star).

Having smart playlist files (.xsp)  in a media source folder that gets scanned into the library will result in duplicate albums.

This is an unusual occurance, smart playlist files created by the user are saved in the userdata folder, and that is generally not added as a music source. It is possible that this is the first time a user has scanned .xsp into the library. However now I know this can happen I will look at ensuring future versions of Kodi are robust enough to avoid the issue. The resulting extra albums point at the same music files as the correct albums (a diagnostic clue for any other ocassion)

So @BikerDude change the music source as I suggested in my previous post, so that your "playlists" folder with both .xsp files and poorly tagged files does not get scanned. You still have mixed tagged albums to fix, but the scanning of .xsp files was a big contributor to your duplicates.

In fact please do all I ask in the previous post, just to confirm things and enable me to give feeback on your tagging generally.

[BikerDude]

Thank you @DaveBlake  Very very much for taking the time to look at may data. I was thinking i was going nuts.

• I do understand exactly what @black_eagle is saying and to some extent i accept the point you make about the multiple different albums in which MB will allocate my music in most cased. I could accept this, but when, such as the case for the beach boys god only knows, MB insisted in putting that song in a different album instead of leaving it in Pet sounds. In this case I would move (drag and drop) the song form the suggested falbum to pet sound for completions sake and in most cases this works.
  
• I did several times the deletion of Mymusic72 and rebuild the library (this is when I get the 3844 records in the songs view. at this point I have no duplicates. I get 4226  when I do an another library update immediately after I realise that i have 3884 just for the sake of it. I will look at each of the offending artists and run them through MB again. 
  
• The V:\ is a dedicated share for my music on my NAS. All my Pcs will eventually point to the music share.  I never tried to move the file to an other share in this last phase. I will try this. This will give me something to do today
  
• The ace of base was a tagging issue caused by me that has been fixed. I found this out by reviewing all my files
  
• The playlist is the folder containing all my playlist which is now all garbage and need to be rebuilt. I put it there because i share the same playlist with other computers in the house. I have in advance settings change the location where Kodi loads the playlists. I will however move the playlist on a seperate share from the music and modify advance sharing. May I suggest that this point be included in the advance setting section of the Kodi document. 
  
• The 2 Jazz MP3 are downloaded from youtube (I too use then to relax). They are not tagged. my plan with them is to use audacity and split the songs in different artists. This is not high on my priority list. If you want I can PM you with these files.
  

Thank you for you patience, suggestions and for putting up with this pedant individual. It is greatly appreciate.

[DaveBlake]

@BikerDude I think we may have cross posted.

Since that playlist folder contains junk as well as .xsp, it is even more reason to use a folder on a shared drive for library music rather than the whole drive itself as music source. But check those recent posts from Karellen and I