Slow movies filter (and inaccurate counts)
#1
When in the movies library, if I select the 'Filter' option from the left-menu it takes anywhere from 15 to 30 seconds to appear (or even more on bad days). Looking at the debug log it appears to be down to multiple calls to "GetPeopleNav" which can be a little slow (presumably loading the actor/director counts on that dialog). Looking at the mysql log it appears to be executing some fairly overkill queries (see the mysql log below) which retrieve a bunch of data when it really only needs the counts. The queries themselves execute fairly quickly, but I'm guessing streaming all the results back to the pi is what is taking so long. Not only is it retrieving a bunch of data it doesn't appear to need, but the actual count it displays does not seem to be accurate - the filter dialog shows 48k actors, yet the actors table in the database only has 32k records (see http://tiny.cc/vibvby)

complete debug log: http://pastebin.com/guFSMLsn
mysql slow queries: http://pastebin.com/9Y9k5xKL

using Kodi 16.1-RC2 Git:4c58ea8
Openelec 6.95.1 7.0 Linux 4.4.6
Raspberry Pi3
Configured to use mysql 5.6 networked server over a 100Mb wired LAN

Any help is appreciated. Thanks!
Reply
#2
Yes, the Filter takes so long to appear because it queries the database for the Genre, Actor, Director and Studio totals - these are quite complex joins returning a vast amount of data that can be very slow:

Image

These totals are apparently pointless, simply window dressing, as when you click on a Genre, Actor, Director or Studio the relevant information is queried again from the database.

The Filter would appear much more quickly if these totals were not being calculated - the totals do not appear necessary for the operation of the Filter.

The queries executed by the Filter dialog are available here (including MySQL query log): http://pastebin.com/bijYVU61

I'll add this as an Estuary feature request, to drop the totals.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#3
Moving to OS Independent, as not specifically a Raspberry Pi issue.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#4
@redec I added an Estuary feature request, here on the assumption this is basically a skin implementation issue (which may be incorrect - it could be a core issue).

Anyway, after further testing I realised the Filter dialog is "live" in that as you twiddle the various parameters (In Progress, Year etc.) the Genre/Actor/Director/Studio totals update dynamically, which makes dropping them unlikely. The dynamic nature of the dialog makes the Filter dialog rather unwieldy on low end gear, even RPi3 - it probably works well only on higher-end x86 gear.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#5
The totals themselves aren't the problem really - the problem is that the client is querying *all* the data, and then counting it on the client side. It could instead have the database do the totaling and just send back the count...the database can do this count far more efficiently than the client can and then it doesn't have to send mb's of data back to the client - just the single number instead.

Edit: oh sorry, I now see that you already mentioned this approach in your feature request. Thx.
Reply
#6
Alternatively, I'm not familiar with the kodi internal architecture, but those numbers seem like something that could be loaded asynchronously...that can obviously get very tricky tho if there isn't already good infrastructure/platform support for async loading like that...
Reply

Logout Mark Read Team Forum Stats Members Help
Slow movies filter (and inaccurate counts)0