[DaveBlake]

So, while I haven't read every single post in this thread, from what I did read I got the impression that idiots where welcome to join in on the testing........so here I am. I've spent the last few days playing around with version 1025.

Yes testing very welcome. What made Kodi such a great media player is the enthusiasts that contributed to it, with either design/coding skills or by testing and feeding back in a helpful and unexpectant way. Having millions of users is a huge burden, and those of us that work on Kodi would happily have fewer of them. But having more enthusiasts supporting the project development by testing things is very welcome indeed.

Second, I have to admit that most of my "playing" has actually been fixing mistakes in tags that were only noticed while analyzing my newly exported folder structure...... most errors being caused by the Performer/Musiciancredits field.

Yes good quality consistent tagging is still key to getting a good library
Kodi has more robust tag processing than it did, but good to find and fix any inconsistencies in your files.

I have been waiting for more feedback from you because at first I didn't completely understand what you were experiencing. Now I will see if I can get any issues that you have not been able to resolve yourself.

With that said, so far I only have a couple of observations:

a.) A lot of album sub folder names were appended with partial MBID's (similar to the post where Dave wrote about  artists having the same name) but this is with albums that don't have the same name and it also doesn't seem to be consistent. - My album folder under Aretha Franklin is "Aretha Franklin_918d" but my album folder under Bob Dylan is just "Bob Dylan" - no MBID appended.
b.) Not sure I can clearly describe this one, but the bottom line is that if I do multiple exports to different directories, the newly generated "trees" don't get the same number of "folder.jpg's" and fanart.jpg's...... no scanning or scraping in between exports. I'll post back if I pick up on a pattern as to when a folder gets a jpg transferred and when it doesn't.

EDIT: It appears (probably no surprise to most) that the appended folders are the result of tagging errors. My initial export generated a sub folder for Bob Marley's album "Legend" to "Legend_edfd". Checking the "album" (actually only have 2 tracks from it) with mp3tag, I found that one track did not have a MB album id while the other did. Fixed it, updated/cleaned library, and did a fresh export and the album sub folder was not appended. Then checked the next artist with an "appended folder issue" and all of it's tracks have the same Album id but I'm sure there's some kind of tag issue involved. I'll try to take note of the various causes as I go thru the affected artist.

Yes that kind of tagging error effectively creates two albums in one folder. On export Kodi spots that and separates them. But I will make a note of that experience for the wiki.

An explanation of how local artist artwork is handled in Leia, and how that works with the improved export facility is yet to be written.  Some of it is burried in this thread, but those doing testing now are running on intuition only. I know I need to write up the basics at least (and then have others improve it), meanwhile here is part of it...

The output form export can go to 3 different places:

1. Single file: A uniquely named xml file is produced that can later be imported into a music library (that has already been populated by scanning tags from music files) to modify/add the additional artist and album information and artwork. A nice and complete approach to replicating additional info in a clean install, but editing one big file (if you want to change things) can be hard for people.
   
2. Separate files per entry: This creates an <artist name>/<album name> structure in the specified destination and both art and nfo files beneath them. Useful for checking what export produces without mxing in with your music files and any existsing art or nfo. Or if you just want to edit nfo for specific artists or albums you can dump the lot somewhere, pickout the specific ones and make changes etc. then use them to manually  replace those in the "library folders".
   
3. To Library folders: This puts the art and nfo files in the locations that Kodi will subsequently scrape them from. That is the artist NFO and art is placed under <artist name> folders in the Artist Information Folder (a location that is part of Kodi settings, if you don't have one set it will as for it). The album NFO and art is placed in the album folder along with the music files
   

I think you have been doing 2), and it has usefully enabled you to identify tagging errors.

However there is nothing stopping users from giving the location of the Artist Information Folder as the destination folder when doing 2) (or having done 2. making the destination folder they used the Artist Information Folder). This results in album folders within the artist folders in the Artist Information Folder. It is not a problem, but Kodi is not looking there for album NFO or art when it scrapes.

That could be confusing for users, and I suspect it is what you have done. However it is natural to keep the additional album info alongside the music files of that album, and it is also backwards compatibile. But artists do not all fit in the music folder structure in a nice way, so a separate location for their additional info is necesary. hence a split location for library folders.

The other thing users can do is set the Artist Information Folder  to be the same as their music folder (beneath which thay have album artist folders). Then export to 3)  results in additional artists folders that don't have music beneath them. It will also create new folders for those albums artists where the artist name differes from the folder name containing the music for that artist. Similar mixing will happen to albums.

You may have experienced that too Bryan, I'm not quite sure.

I am open to suggestions for how the UI for this can be improved to make things clearer. But it is important to realise that music library export is not a backup facility (for that copy the db), it is an advanced user facility for management of additional local artist and album info and art. Beginners will be scraping online art etc., this is for those that want to go further, have exotic music collections or are control freaks for what data/art they have (no insult intended as I speak of myself).

Just following up..... eliminated a number of problem folders by one of the following:
1) Removed "illegal" characters (ie colons and such) from MusicBrainz modified album tags that I hadn't anticipated being turned into a folder name so I hadn't worried about it.
2) Changed disc number from 1/1 to just 1......really couldn't believe that would work, but it did on some albums.
3) Similar to #1, renamed a "music folder" which wasn't exactly what the tags said the name of the album was.....apparently location matters now

1) I'm not sure what tags had colons. Folder naming deals with removal of reserved characters, so not sure what effect your edited tagging was having. Can you explain?

2) Not clear on how disc number tag could have had an impact either. Can you give example?

3) No, just as before the name of the folder containing music files for an album does not matter. I think what you are experiencing is the result of setting export destination for export to separate files to be the same as your music source. The artist folder name matched, but the album folder name below it did not.

I guess it is too easy to do that unintentionally, and then draw those conclusions from the results. Not sure how to discourage it, as it is not "wrong" just creates results that can confuse. Maybe some rewording of labels?  Export to separate files is the more a diagnostic tool than for everyday use, used with an isolated destination for output rather than mixed in with music.

That left me with nine albums that I could not find anything wrong with in the tags or location folder names. They are all self-titled albums (as in Dire Straits/Dire Straits, Fleetwood Mac/Fleetwood Mac etc)
All I could do there was to remove the MB album id from the tags completely and then verify that the "export folders" weren't appended with something else .......they were not, so I'm calling it "good for now".

As for inconsistent exporting of folder.jpg's,  no pattern found yet but since I already have them with my music files I'm not too worried about it.

What exactly was the resulting folder layout for those 9 albums, sorry but I have become lost along the way about what happend. Can you clarify please.

And was that artist of album folder.jpgs?

I think it would help to export all items to separate files, with a destination folder completely away from both music sources and Artist Info Folder, and see what you get. Then come back with anything unexpected in the resulting <destination>/<artist name>/<album name> folder layout. Thank you for your enthusiasm.



Save
Save

[Bryan_W]

1) I'm not sure what tags had colons. Folder naming deals with removal of reserved characters, so not sure what effect your edited tagging was having. Can you explain?

2) Not clear on how disc number tag could have had an impact either. Can you give example?

3) No, just as before the name of the folder containing music files for an album does not matter. I think what you are experiencing is the result of setting export destination for export to separate files to be the same as your music source. The artist folder name matched, but the album folder name below it did not.

I guess it is too easy to do that unintentionally, and then draw those conclusions from the results. Not sure how to discourage it, as it is not "wrong" just creates results that can confuse. Maybe some rewording of labels?  Export to separate files is the more a diagnostic tool than for everyday use, used with an isolated destination for output rather than mixed in with music.


What exactly was the resulting folder layout for those 9 albums, sorry but I have become lost along the way about what happend. Can you clarify please.

And was that artist of album folder.jpgs?

I think it would help to export all items to separate files, with a destination folder completely away from both music sources and Artist Info Folder, and see what you get. Then come back with anything unexpected in the resulting <destination>/<artist name>/<album name> folder layout. Thank you for your enthusiasm.

Hi Dave,
Trying to be brief, but I'm sensing that I may not succeed......

First, all my tests have been to separate files and a semi isolated location. 
Using a Temptations album that had the "illegal character issue" as an example, here's my setup: 

Kodi_1025 portable installation path  -  C:\Kodi18_1025
Music Library path - D:\LocalStuff\Music\Albums\The Temptations\My Girl-The Very Best of The Temptations

Using MB Album # 4f0a9c91-4c4f-42ba-bb54-eb9878446b09  the following tag for the album name was generated: (again, I wasn't worried about the colon at the time so I let MB do wht it wanted)

My Girl: The Very Best of The Temptations

Exporting produced the following: 
Export Library path -       D:\LocalStuff\Kodi18exporttest\The Temptations\My Girl_ The Very Best of The Temptations_4f0a
After changing colon to a dash for all the tracks with mp3tag, resulting export was - 

D:\LocalStuff\Kodi18exporttest\The Temptations\My Girl-The Very Best of The Temptations.

Similar thing with a Grateful Dead album.
MB Album # 03385b21-ba70-4253-a635-2f64e893a95e
Dick's Picks, Volume 23: Baltimore Civic Center, Baltimore, MD 9/17/72

Music Library path -D:\LocalStuff\Music\Albums\Grateful Dead\Baltimore Civic Center 72 Disk 1
Export path - D:\LocalStuff\Kodi18exporttest\Grateful Dead\Dick's Picks, Volume 23_ Baltimore Civic Center, Baltimore, MD 9_17_72_0338

After mp3tag editing (figured I'd make the album name shorter as long as I was removing colon) results were  - D:\LocalStuff\Kodi18exporttest\Grateful Dead\Dick's Picks Vol 23 - Baltimore Civic Center -1972

I'm going to stop here. I want to:
1) Figure out how to best organize my test data into something you can more easily analyze ...... I archived the resulting directory(s) each time I tried an export, but since I did the "export everyone" option, there's  lot of folders/nfo files that aren't relevant...... I'm thinking I'll either take screenshots or I'll pm you with a link to some files...maybe a little of both.   
2) Try the test you suggested - I figure  since my music is on D: drive, exporting to C: drive should be far enough away ;-)
3) Get a better handle on what's happening with the folder.jpg issue.

[bilgepump]

This results in album folders within the artist folders in the Artist Information Folder. It is not a problem, but Kodi is not looking there for album NFO or art when it scrapes.

I'd mistakenly thought after I saw the album "stuff" in the artist info folder that things were changing and that kodi would be looking in the artist info folder for everything. Yes, that might be confusing without knowing that kodi isn't looking there for the album info.

But it is important to realise that music library export is not a backup facility

Do you say that because when you export the album info will go into the artist info folder, but Kodi won't "suck it back in" because that's not where it looks when it's doing its sucking for album info? .

(for that copy the db), it is an advanced user facility for management of additional local artist and album info and art. Beginners will be scraping online art etc., this is for those that want to go further, have exotic music collections or are control freaks for what data/art they have (no insult intended as I speak of myself).

Or, for using in an offline installation (me). It's given me something I did not have, all the pretty pictures of singers I've never heard of (). J/K, I'd had the "show artists who aren't on albums"(or whatever it says) setting off because there was no point. Now with my little local source of artwork in the artist info folder, I can have all the artists showing.

[DaveBlake]

I'd mistakenly thought after I saw the album "stuff" in the artist info folder that things were changing and that kodi would be looking in the artist info folder for everything.  Yes, that might be confusing without knowing that kodi isn't looking there for the album info.

Yes I need to make that clearer, we are working on it.

But it is important to realise that music library export is not a backup facility

Do you say that because when you export the album info will go into the artist info folder, but Kodi won't "suck it back in" because that's not where it looks when it's doing its sucking for album info? .  

I say that music library export is not a backup facility (and never has been) because I think a backup is a complete store of something from which you can recreate a system, and music export does not do that. The essential data of the library comes from tags within the music files, and that is not exported. Having an xml file, or lots of NFO lone will not recreate the library.

When I backup my Kodi installation I copy the db files (and custom nodes and settings files), I don't use export. That is what I would advise others to do too.

Also the principle idea of NFO files is to enable users to provide additional artist and album data themselves rather than depend on an online source to provide it. It really is a bit clunky to then create NFO files to save that downloaded data locally, best way to do that is a single xml file. If I wanted to transfer the downloaded additional data to a new system I would use export to single file and then import that file. Using separate NFO files only has value if you are wanting to edit them or manage the information some way.

(for that copy the db), it is an advanced user facility for management of additional local artist and album info and art. Beginners will be scraping online art etc., this is for those that want to go further, have exotic music collections or are control freaks for what data/art they have (no insult intended as I speak of myself).

Or, for using in an offline installation (me).  It's given me something I did not have, all the pretty pictures of singers I've never heard of ().  J/K, I'd had the "show artists who aren't on albums"(or whatever it says) setting off because there was no point.  Now with my little local source of artwork in the artist info folder, I can have all the artists showing.

I am glad you fell you have gained from this, but I am a little confused. Do you mean that you have an online system, scraped art for song artists from online sources, exported it, and can now also use it from an offline system that can only show local art?

[bilgepump]

of something from which you can recreate a system, and music export does not do that.

ah,ok, I look at any copying of something useful as a backup. In this case, it's just a backup of the artwork and information, not of the whole system.

If I wanted to transfer the downloaded additional data to a new system I would use export to single file and then import that file.

You trust your computer more than I do mine. One file, and the computer's in charge of sucking it back in? I get the creeps when the computer has too much power. I like having all the labelled folders, I can open them and see what's in there, I can read the nfo files.

Do you mean that you have an online system, scraped art for song artists from online sources, exported it, and can now also use it from an offline system that can only show local art?

Yep. Before your artist info folder, it was good for all the album artists. The "other" artists were all there, too, but no pictures (just a generic icon) or information. Now, using the artist info folder, they've got the lot.

[DaveBlake]

If I wanted to transfer the downloaded additional data to a new system I would use export to single file and then import that file.

You trust your computer more than I do mine.  One file, and the computer's in charge of sucking it back in?  I get the creeps when the computer has too much power.    I like having all the labelled folders, I can open them and see what's in there, I can read the nfo files.  

I trust the software I wrote to do the sucking
You can read the xml file too, uses the same xml tags as the NFO files. But if human intervention (even just checking) is something you want to do then I can see why you would want to use separate files.

[bilgepump]

I trust the software I wrote to do the sucking

good one. But there are all those other flickering do-dads that are in there, trying to mess things up....

But if human intervention (even just checking) is something you want to do then I can see why you would want to use separate files.

I've been copying my cassettes into the computer and scanning covers, so I might want to stick those in there. or something.

[Bryan_W]

I think it would help to export all items to separate files, with a destination folder completely away from both music sources and Artist Info Folder, and see what you get. Then come back with anything unexpected in the resulting <destination>/<artist name>/<album name> folder layout. Thank you for your enthusiasm.

SaveSave

Not exactly sure I'd refer to myself as enthusiastic regarding this particular issue since the truth is I'm not really too worried about appended folder names......... I mostly just thought it was somewhat odd and in addition to pointing it out, I thought I'd try to find the cause.....mostly just "for fun".  

Anyway, I just got around to playing some more with the albums that couldn't be fixed with tag editing.
I created a test library containing  only artists having "self titled" albums and set my export location to a completely different drive than that of my music source.
The short version of my findings are:

1) With a music source having a Library/Artist/Album folder structure (and with individual tracks all being tagged with a MusicBrainz Album ID) if the artist only has one album it doesn't matter if it's self titled or not - exporting as "separate files" will recreate the folders "correctly"/"as one would expect".  For example - I have one Blind Faith album - that being the self titled album 'Blind Faith' (Did they even ever have more than one? Nevermind, another topic)..... Anyway, It exports fine. However.......

2) With any artist that has more than one album and one is self titled, exporting as separate files will produce:
               <Export root folder>
                         <Artist1>
                               <NOT self titled album name1>
                                <Self titled album name_xxxx>........... where xxxx is the first four characters of what the tags have as the MusicBrainz Album ID
                                <NOT self titled album name2>   etc......
                          <Artist2>
                                <Artist2 Album name1> etc......

Using The Beatles as an example for this one  - with both 'Abbey Road' and 'The Beatles' as album subfolders, 'The Beatles_bdc5' is the folder created during export. Removing 'Abbey Road' from the library, and 'The Beatles' is not appended with _bcd5 

3) Including in the test library an artist that does not "officially" have a self titled album (but has more than one album subfolder) and creating a "faux self titled album" by changing one of the album subfolder names to the artists name, and an _xxxx appended export folder will be created - (regardless of what the tags say the album is called)

In my test I added Bruce Springsteen as the artist and include two album subfolders - 'Born to Run' and 'Blood Brothers'. I renamed the 'Blood Brothers' folder to 'Bruce Springsteen' and the export folders produced were 'Born to Run' and 'Bruce Springsteen_c831'

4) (As previously posted) if any self titled album does not have a MB Album ID no appending occurs..... or maybe one could say it gets appended with a <blank> MBID?

Anyway, going to move on to checking out the folder.jpg issue.