[DaveBlake]

Split from other tread as spearate topic

I've got a huge collection speaking of a 125MB database file within mysql. It's slow, although connected with gigabit lan. I make use of 'similar tracks' for last.fm a lot having the possibility to find similar music to the one that is playing just now within my own collection. It takes quite a time with local db but with mysql it's slow. Is there a way to fasten db queries by doing some indexing? Even adding huge genres (3k files) into a playlist is not what I call performance. I would expect a speed bonus for db in comparison to filesystem. Adding 3K lines of text into a playlist is a simple copy paste. Could having the playlist be a db file or a table itself enhance this in regards to performance as it's only a query? I don't have to fear missing tracks within a playlist as I keep the db up2date.
Is there a db query for kodis musicdb that's able to measure system's performance and make it comparable? A benchmark?

Lastfm - I assume that is using some addon? I would guess it is doing something inefficient querywise, but only what is possible for a script via JSON API. But I emphasise guess, you need to discuss it with the addon author.

When you say "huge genres" you mean 3k of song files? The performance issue with any long list in playlists or nodes especially songs is well known, and it actually isn't the database that is the problem but what Kodi does with the data having fetched it. It is being looked into but does require substantial redesign work.

Benchmarking - not as I know of, although people have timed their system here and there. As you realise LAN speeds make a difference as well as processer power and disc access speed (or wherever the music files are)

[Uatschitchun]

Hi there, the thread title is misleading, as 'similar artist script' only was an example.

I was addressing the performance issues in general. Where am I able to follow the discussion around 'what kodi does with the data having fetched it'?

As stated, my library is huge I would say (150k files). I've sorted the music on file system level into folders of top-genres (Hard, Smooth, Reggae, World, Soundtrack, etc) app. 10-12 of that kind. I even tagged the genre tag of most of the files in these folders with a corresponding genre tag, to replicate this ordering with using genre view (be it kodi or ipod or whatever database centered device I'm using).

Coming from this, the genres contain huge numbers of titles. Simply hitting play on one of these top genres leads to a good deal of time until kodi starts playing,whereas a db query maybe only takes seconds...

[DaveBlake]

I was addressing the performance issues in general.
...

Coming from this, the genres contain huge numbers of titles. Simply hitting play on one of these top genres leads to a good deal of time until kodi starts playing,whereas a db query maybe only takes seconds...

Yes we know and have known for some years. Fixing the underlying issue is a major redesign that is being looked into, if it was easy it would have been done already.

Meanwhile I am looking at better ways to see smaller and more manageable amount of your collection. Afterall do you really want to play all those songs? I know there are some people that want to queue everything and pause/restart over days (if not weeks) while they listen through it. For now Kodi is not the tool for that, it was not designed to fulfill that requirement. But even if queing the entire coillection was quick, then better selectioin of smaller amounts will always be a requirement too.

I'll fix the thread title, but you do mention lastfm, and the speed of that is a script issue.

[Uatschitchun]

I was heavily involved in media center development (mymediasystem.org) a while ago, so I know what sounds simple sometimes isn't as software has history and even small changes sometimes need large redesigns!

Sometimes, yes, I like to simply hit random on one of the top genres... be it lazyness, convenience or simply the surprise :-)

And no, I'm not listening for days. It was just my thought that, as db queries are fast, some actions within kodi have to be faster...

Are you able to describe in a few words, what that 'underlying issue' is, just to enlighten me?

Btw, here's the last.fm script I was talking about:
http://brianhornsby.com/kodi_addons/similartracks

[DaveBlake]

Sometimes, yes, I like to simply hit random on one of the top genres... be it lazyness, convenience or simply the surprise :-)

Meantime you might like to try using Party Mode instead for that, which avoids manipulating the entire song collection in programme memory before playing anything. The snag is that you have to edit the Party Mode smart playlist (in the playlists, an entry created by default with no rules) to specify the genre otherwise is defaults to everything. But then pick "Party Mode" from the sideblade.

I'm not expert on using this, random play isn't something I ever use.

It was just my thought that, as db queries are fast, some actions within kodi have to be faster...

Reasonable thinking, but unfortunately not.

Are you able to describe in a few words, what that 'underlying issue' is, just to enlighten me?

Kodi fetches data from the db, and timings on that are ok, but then processes the results dataset into local memory (a nest of linked lists with sub-lists) before it can be displayed and it is doing that for a lot of songs that takes the time. I guess all the alloc/dealloc of memory etc. At very least some kind of lazy read is needed when it comes to loading this internal storage from the results dataset, or even better (in my view) would be a complete reworking of the CFileItem structure we stuff everything into.

The db often gets the blame, but it is the subsequent data processing and use of objects to get the data onto the screen that is the issue, not the RDBMS itself, nor the schema, nor the queries.

If you have the skills and interest by all means have a look at the code

[Uatschitchun]

Thx for explanation!

If you have the skills and interest by all means have a look at the code

Sadly I've got no coding skills :-(
I speak some languages, but none any computer understands :-)