[DaveBlake]

Since I have your music db I'll take a closer look at your examples.

EDIT::But I have to note that not only is your music collection huge, but it is very unusual e.g. you have albums with hundreds of songs each (all track 1) , you have 18946 albums (out of 56878) with 1 song but not flagged as a single. Albums to not match their mbid values, songs don't have them. No idea if Embly contributes to any of this. Perhaps useful as a volume test but not respresentative structurally.
 

For the moment, I think some improvement is still possible even under the current approch.
e.g. Subset of music loaded via node...

Code:

<?xml version='1.0' encoding='UTF-8'?>
<node order="530" type="filter">
    <label>Funk</label>
    <content>songs</content>
    <match>one</match>
    <rule field="genre" operator="is">funk</rule>
    <rule field="genre" operator="contains">funk - </rule>
    <rule field="genre" operator="contains"> - funk</rule>
    <order direction="ascending">artist</order>
</node>

...takes about 12 seconds (on low end hardware).
I think this delay is mostly based on db performance, so nothing to improve here.

Impossible to know how long the db query is taking on your "low end hardware" without a debug log. Testing on my dev env the query against 332k of songs takes ~2s. If I swap from corrulated subquery syntax to joins it is 10% faster, which is a possible optimisation, and even more query specific clever tricks could get the result in a lightening quick 100ms, but either would need a redesign of the way queries are built from smart playlist rules, and hard to do for the possible rule combinations.
 

After this I have list with around 1937 items. Now I click on a item (song). As far as I know, the playlist will be created at this moment.This takes between 20 seconds and 3 minutes depending when I click.

Yes that is when the internal list of songs to play is populated, in this case with the 1937 songs. But I am really surprized by the 3mins and the variability, are you sure and can you provide a debug log of it doing that (turn on debugging, reproduce, post debug.log in pastebin and post link).
 

I assume the playlist-creation is running in background thread asynchron and if first thread is not finished, while I select a new song (generate new playlist), the first thread will not be cancelled.

Actually not. If the queue is taking a long time to populate then the GUI freezes while it does so and you can't select another song or move to another node etc., and there is no way to cancel. That's crap isn't it, but it is how things currently work.

However the player is asynchronous, so if you click on another song (once you can) when something is playing it will continue to play while the queue is cleared and repopulated possibly with the same songs as before.
 

If I select in the list (not playlist) another song Kodi starts over recreating the playlist.
This is imho absolutely unnecessary, cause nothing has changed. Here is room for improvement cause this is just a decition, when you trigger the create-playlist subroutine and maybe cancel previous thread.

I agree room for improvement, e.g. knowing that the list of songs on display has not changed since the queue was populated and hence not clearing it just switching song.
 

Another spot for improvement:
At the previous example I select the subset of songs via node and it took around 12 seconds.
Now, if I use the group function...
...it is quite fast (almost instantly) showing me the genres list, but:
If I select e.g. Funk as genre it takes 1 minutes. (compared to 12 seconds before)
Here is also imho something wrong cause this is not a DB performance issue. I think here is also a design problem the actual reason.

What you quoted "use the group function" was the default genre node, did you mean that? It takes you 12s to show all the genres? Are you then saying that clicking on a specific genre e.g. "funk", it takes 1min to show all the artists with albums or songs with that genre? If so that sounds very odd, again a debug log please.
 

Additional information:
When playlist creation is in progress, the CPU is not really busy. 4 cores, and peak of one core is maybe 50-60%. Combining load of 4 cores would be approx. 1 core at 100%. The rest would be idle. Not sure if there is some load-balancing and playlist-creation is a single thread, or as it looks multithread but not using the full cpu power.
I gave pop-genre also a shot, but I stopped after 35 minutes playlist creation and still pending.

Not sure about processor use TBH, but what you are doing is more memory allocation limited than processing itself. I can't comment on what was happen with "pop-genre" without a debug log.
 

When I have time, I will also investigate on those again to remove the most painful delays. Unfortunately, the code is very complex and as you mentioned, this would need lots of work and really experienced developers to fix the design issues.

Yes. People with the skills and interest are always welcome to volunteer their effort, but the learning curve to be able to contribute to core changes effectively is steep, and there is a lack of experienced developers with spare time to be able guide and mentor. Patience and persistence are needed, and lots of good will on all sides.

I have huge respect for the work of previous devs, but what they originally designed was a media player that could run on an old Xbox. This has limitations given what Kodi has grown organically into, and some design weaknesses, but on the other hand Kodi is an amazing media player.