Posts: 49
Joined: Aug 2017
Reputation:
0
2019-01-19, 23:31
(This post was last modified: 2019-01-20, 09:42 by quickmic.)
Firstly, thx to all devs for their big affords.
You are doing awesome work here!
I just have a general request. Please test Kodi with big databases while developing. There are some major performance issues when huge collections are used.
Playlists are damn slow, nodes are unusable in case of "group" parameters etc. Even on high end hardware.
If you need sample databases I can upload something for tests.
Posts: 17,859
Joined: Jul 2011
Reputation:
371
First, how about properly doing a thread description.
Further logs, used version and whatever else are missing.
Posts: 49
Joined: Aug 2017
Reputation:
0
I don't know how a better thread title should reflect a imho general development problem.
Version? Anything from 17 up to nightly 18 which I'm currently using. Not tested previous versions.
Logs? There are no errors, just slow in case of big databases. e.g. Trying to play a song from a couple thousands selection takes minutes.
But if you think this thread is inappropriate, just delete it and I zip it.
Posts: 49
Joined: Aug 2017
Reputation:
0
big databases means:
Audio: 332844
Music videos: 22578
Movies: 3840
TVShows: 12136
Posts: 49
Joined: Aug 2017
Reputation:
0
I think this in not a music-only issues. I assume this is a general issue, just because of the of the size of my music collection it's best traceable there.
I assume it's a problem with playlist creation. Somewhere here maybe: GUIMediaWindow.cpp in "bool CGUIMediaWindow::OnPlayAndQueueMedia" here -> "g_playlistPlayer.Add(iPlaylist, nItem);"
But I didn't dig that deep.
Currently my Musicvideos are also not really snappy, but my db for videos is 10 times less than audio.
I'm almost sure, the issue appears on every collection.
Posts: 49
Joined: Aug 2017
Reputation:
0
I'm using sqlite, but the DB-Size is not the problem. I ruled that out.
Hardware is not the problem. Tested on osmc vero 4k and Ubtuntu. Vero 4k is lower of course, but on i5 amd64 ubuntu also slow.
Anyway, the only thing I wanna report/suggest/offer.
I would upload my DB and my nodes for test cases. Everybody who will use that, will immediately spot the problem.
Posts: 49
Joined: Aug 2017
Reputation:
0
2019-01-20, 01:39
(This post was last modified: 2019-01-20, 01:48 by Karellen.)
I've uploaded my complete setup.
@Karellen- database link removed
Tracing general performance issue:
music - > widget genre -> pop -> try play a song ... it will stall
Tracing node group issue:
music - > widget genre-slow -> pop ... it will stall
Posts: 49
Joined: Aug 2017
Reputation:
0
2019-01-20, 02:08
(This post was last modified: 2019-01-20, 09:21 by quickmic.)
btw, I created that DB with emby for kodi plugin, but this is not relevant. The behaviour will not be different on native kodi db, just in case the content looks a bit different...
Posts: 4,545
Joined: Jun 2015
Reputation:
269
2019-01-20, 10:41
(This post was last modified: 2019-01-20, 11:01 by DaveBlake.)
Yes, Kodi is slow when trying to populate the linked lists that sit behind the GUI and current playlist queue with a large number of items. It is a known issue, and nothing to do with the actual databases and how they are accessed. There probably could be some db optimisation done but it is not the main contributor. Using MySQL (clent/server setup) is slower than (local) SQLite, and a slow processor and data/network access will obviously makes things worse, but there is a design limitation.
What this means in practice is that if you have a large collection of music, for example, then that looking at the songs node (all the songs), or a big playlist, or clicking play from one of those (hence filling the queue with all those songs) is going to be slow. It is often hit in the music library more than video because users have more songs (332844 in this case) than movies (3840), but the issue effects both and pictures too come to that.
To fix this issue means a total redesign of how the GUI works and memory is used to use some kind of lazy loading, not the current approach which waits until all the items are loaded into the list before displaying any of them. Likewise sorting needs moving into the db (which is quick at it) rather than being done in memory with some algoithmic mistakes.
Frankly the active dev team is very small at the moment so tackling these fundamental redesign issues is slow progress. All we can say is it is known issue and hopefully can be fixed one day.
In the meantime as a workaround make use of smaller nodes e.g. genre, year etc. , smart playlists or party mode (to randomly play songs from a playlist) to limit how many items are displayed or added to the queue at one time. FWIW my prefered work around, since I have a large music collection and like to listen with TV off, is to use a remote app to browse my music and control what I play. It also uses a SQLite db just on an Android tablet and yet UI speed is fine, which anecdotally confirms the slowness is not having lots of songs or using SQLite but what Kodi does with the data once it is fetched.
