[Karellen]

Can you redo test 5 when I get next text build ready

Yep. Sure will.

[DaveBlake]

Test 4/5 was "separate files per entry, 4 was album artist and 5 was albums,album artists. They were extracted to the "destination folder". I did notice that when the album artist and album names were identical the album folder created had _nnnn appended (nnnn is from album mbid, but need to test what happens if I didn't have an album mbid). It doesn't seem like it is necessary to have the name collision test between artists and albums?
...
Update. Looking through the nfos and folders created, I do have one other album that the export folder name has the album mbid appended, and I don't see that there is any name collision at all between it and artists or songs.

The collision testing covers a number of things:

1. (First) album artist and album name repreats - get lots of these with classical music e.g. Bruckner / Haitink / Vienna Phil "Symphony No. 3", and Bruckner / Chailliy / RSO Berlin "Symphony No. 3"
   
2. Same artist and album on totally different paths - you can have same album twice in Kodi if one tagged with mbid and the other isn't (a 3rd will overwrite one or the other when scanned into lib) and they are on separate music sources. This happens with my test data, less likely in real life perhaps?
   
3. The collision testing allows for all artists and albums in the library, not just the ones being exported, so that a consistent/repeatable folder name is given. So say exporting just scraped albums, collisions with those not scraped but could be exported later is avoided.
   

Now I don't know if that explains your results, or if I have made collision avoidence too sensitive. The collision testing is done using SQL, so I can let you have the queries to examine your data with if that is any use. In my own testing this did find a few hidden tagging errors (mixed mbid tags), and duplicates I didn't know I had.

Interested to check this deeper.

I notice on the album.nfo files the <track> data is no longer exported?

Yes, the scraped track listing (with no relationship to what music you have) is being deprecated in v18. Removed from both nfo and album info dialog. Many skins already display the far more useful list of songs on the album info dialog, but I anticipate changing the dialog to do that by default. There need to be some other core implementation changes to that dialog (split db access off main thread), so it will all come in a dialog PR.

[DaveBlake]

Back to thinking aloud...

My current idea is to add song ratings, playcount and lastplayed date to the album xml (both in album.nfo and single file export) as a means to save and restore the song data that Kodi creates and changes (or the user enteres via Kodi) during use.

At the moment that data gets lost if you do a fresh install, or drop a source etc. I feel it is wise that Kodi does not mess with the media files it plays, so saving as music file tags is out. But it would be so nice to be able to tranfser the values. Someday that kind of data may become user profile dependant too, but no reason not to do something while waiting for that.

By tying it to album, rather than music file, it also means that the data can easily be transfered if music files are moved about. Again at the moment moving music files, unless you dig into the db manually, means loosing ratings etc.

Downside is what to do for singles? There is a fake album entry per artist with singles, export that??

Also probably need an export option for this song data, after all sometimes you may want to start with a clean slate, not have old play stats imported. But want to keep that dialog as simple as possible.

[docwra]

Yep that sounds fine. But another idea may be to export all the ratings to a single XML file.

Don't forget people can back up their ratings to the cloud now as well

[Edit: Fixed link - DB]

[DaveBlake]

Yep that sounds fine. But another idea may be to export all the ratings to a single XML file.

Don't forget people can 309611 (thread)back up their ratings to the cloud[url] now as well

Yet another addon doing things I am obvlious too :p
Please don't make me look at more Python, take pity on my poor overworked brain...

What identifying parameters for song are sent to the Cloud, and used to retrieve a rating?
Using track mbid, matching on match on artist, album title and song title, or what?
Using JSON API to get/set values?

[docwra]

The Add-on can match on Artist-Album name, Artist/Track name or MBID

I use it to sync my ratings over all my Kodi installations. But it can also be used to backup single instances or share ratings with others. This does require an account on www.theaudiodb.com though.

Currently it does not sync playcount.

And yep it uses the JSON api to submit the ratings back to Kodi.

[User 224999]

Sorry Dave for climbing in so late, been busy with family life as I told you before.
Like discussed before I'd like to help out where I can to migrate stuff from SHS into native Kodi code and improve the music library overall.

Let me know where you need help and what needs to be tested.
Do you have some todo list (like Trello or something) with requests, improvements and such and their status ?

[docwra]

Sorry Dave for climbing in so late, been busy with family life as I told you before.
Like discussed before I'd like to help out where I can to migrate stuff from SHS into native Kodi code and improve the music library overall.

Yes! Can't wait.

[DaveBlake]

No worries Marcel, our timing has just been off, neither having time to talk to the other when they do
But it would be good to find a practical way to liase.

Let me know where you need help and what needs to be tested.
Do you have some todo list (like Trello or something) with requests, improvements and such and their status ?

I have a todo list, but nothing so sophisticated as Trello, with just me slowly doing the work there was no need!! C++/core changes go at a slower pace than than addons I think.

I have tried to make my list public here https://forum.kodi.tv/showthread.php?tid...pid2558589 and to inform people what I am currently doing. And anyone can take a look in my repo of course.

But I am happy to collaborate, and if that means having clearer plans and using tools then sure. Are you ready to get into some C++ yourself?

How can you help? Well....
Scott and Karellen have done some recent testing for me so that is covered (for now).

I have done nothing to make the newer data (e.g. Musicbrainz release group id) available to addons. The JSON API needs updating, and whatever route addons use. I would like some input on that.

The work related to this thread includes Kodi having a separate nominated folder for artist nfo and art (not trying to mix them into the music file structure). I think SHS has similar and it would make sense to use the same folder structure. Again what is needed for that? Folder naming based on artist name needs a consistent approach, so should Kodi pass that out to the addon?

[docwra]

One thing that needs a bump is those PR's for CDART and such that Ronnie did on github. Then the python scrapers to support them.

[DaveBlake]

One thing that needs a bump is those PR's for CDART and such that Ronnie did on github. Then the python scrapers to support them.

I took on the scraper work from @ronie as he was busy, made quite a lot of headway but stalled on Python / got distracted back to core work. I have clear ideas for what is needed, but would gladly hand over to a Python expert

I am aware of the CD art PR, but it is really better to get this refactoring done then include it.

[bilgepump]

I'm also still willing to help, if I can. If you need something tested by regular clueless types, I'm it.

[DaveBlake]

Bilgepump, not quite at the "normal user" test phase yet but I will shout and wave (via forum) when I am.

[bilgepump]

Bilgepump, not quite at the "normal user" test phase yet

Yes, the fact that I have no clue what you are all talking about tipped me off.

[User 224999]

No worries Marcel, our timing has just been off, neither having time to talk to the other when they do
But it would be good to find a practical way to liase.

I have a todo list, but nothing so sophisticated as Trello, with just me slowly doing the work there was no need!! C++/core changes go at a slower pace than than addons I think. I have tried to make my list public here https://forum.kodi.tv/showthread.php?tid...pid2558589 and to inform people what I am currently doing. And anyone can take a look in my repo of course.

But I am happy to collaborate, and if that means having clearer plans and using tools then sure. Are you ready to get into some C++ yourself?

Well, maybe first start this way, I'll review the code you're working on to get some clue of what's going on and the todolist in the forum post.
If easier we can decide to create some public TODO list in Trello (or alike) later.
Yes, ready to get into some C++ but I'll start with the python scrapers, see what needs improving so I have a goal of what to contribute to the core code. I know for example the artwork and json api for music is very minimal compared to videos so I'd properly start there.

I have done nothing to make the newer data (e.g. Musicbrainz release group id) available to addons. The JSON API needs updating, and whatever route addons use. I would like some input on that.

Sounds like a great project for me ;-)

The work related to this thread includes Kodi having a separate nominated folder for artist nfo and art (not trying to mix them into the music file structure). I think SHS has similar and it would make sense to use the same folder structure. Again what is needed for that? Folder naming based on artist name needs a consistent approach, so should Kodi pass that out to the addon?

Yes, this is exactly the part in SHS that drove me (almost) mad haha.
In the end it turned out that 50% of the users just want their artwork stored in their media folders (including me) while others want a different folder structure to store their artwork/metadata.

I believe there are two use cases here:

1. Regular users, the kind of users that want ease of setup with lesser settings to make.
They probably don't care 1 second of where their artwork/metadata is, if they even want to store that.

2. Power users, the kind of users that want to be in control of each setting and think carefully about their setup, where data is stored.

If it were me, I would make sure that artwork is by default stored in the media folders, this is also very consistent with the other media types.
Besides that, create some settings in the scraper setup for the more advance power users like a custom folder where artwork should be stored and which artwork/metadata to collect.

Imo all this base logic should be in Kodi core and/or the official scrapers and the addons (like SHS and such) are either obsolete or only cover some edge cases that are not needed to target main audience. In anyway I want to get rid of the 100's of python processes running to just provide a part of the puzzle.