+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: General Support (https://forum.kodi.tv/forumdisplay.php?fid=111)
+---- Forum: OS independent / Other (https://forum.kodi.tv/forumdisplay.php?fid=228)
+---- Thread: Slow Recently Added Episodes (/showthread.php?tid=378386)
Pages:
1
2
Slow Recently Added Episodes - RTam1990 - 2024-08-01
This has been bothering me for a while. Is there any way to speed up the performance of the Recently added episodes list/widget on the home page? I'm using Matrix 19.4 but I believe it was happening on previous versions as well. I'm not able to migrate to a newer version of Kodi right now.
It's taking about 3 or 4 minutes to update.
My hardware is an SBC Orange Pi 5 with 16GB which is not working hard and has plenty of free memory and disk space.
I'm running Kodi in a docker on my NAS with a MariaDB backend also running in a docker. I have two Kodi clients connected by WiFi to the LAN. One is connecting at about 175Mbps on 5G while the 2nd on is connecting at about 292Mbps on 5G. These speeds vary.
I believe last time I tried looking into this a while ago, I checked the MariaDB slow log and nothing showed up. I tried manually executing what I thought was the SQL query being run and it returned very quickly in my SQL client...under 100 or 200 milliseconds, I believe, with no errors in the MariaDB or the Kodi logs.
I did a quick search and didn't really find anything related to this.
Any suggestions on where to look or what to do to improve this would be greatly appreciated.
RE: Slow Recently Added Episodes - pkscout - 2024-08-01
When you say "update" are you talking about the amount of time to render that section, or the time between a change in the library and when it shows up in that list?
For the former, it would be helpful to know which skin you're using. No skin should take 4 minutes to render a section like that, but at least it would help us know how to help you. It would also be good to enable debugging and then upload a complete log that shows the problem happening.
If it's the latter, the widgets are updated on an internal timer that is core to Kodi. There is no way to change that. Some things (like updating your library) will trigger an update of sections like that as well. One caveat is that if you have multiple Kodi instances sharing the same library, a library update on one won't trigger any update of the widgets on the others.
RE: Slow Recently Added Episodes - RTam1990 - 2024-08-02
It's the amount of time to render the section, Recently added episodes.
I'm using the default Estuary skin.
I turned on debugging. Log is at https://paste.kodi.tv/ugahihihoh.kodi
I didn't see anything odd or that stood out in my quick look at the log. I turned on debug and then restarted Kodi. The log has from the start of Kodi to just after the list was completely rendered.
Thank you
RE: Slow Recently Added Episodes - DaVu - 2024-08-02
Just to mention it...you are using an outdated and unsupported version of Kodi. Outdated and unsupported because it's Kodi 19.4 (actual release is 21) and because of
Starting Kodi from DebianThe Debian version of Kodi contains code changes which are done by Debian themselves. Hence they have to call it "Kodi from Debian". Have you tried using LibreELEC to see if the issue also exists?
RE: Slow Recently Added Episodes - RTam1990 - 2024-08-02
Yes, I know I am using an older version of Kodi and "Kodi by Debian". This was the version that was/is available in the apt repositories. As I said, I am unable and cannot update at this time. I also said it happened with previous versions. I was using Libreelec previously and the Recently added episodes was slow then as well, in my setup at least. Unfortunately, Libreelec is not available for the RK3588/Orange Pi currently. It looks like it may be a while, if ever, that it is available. Another reason that I'd rather not use Libreelec is that it is a standalone Kodi version While Kodi by Debian is running within Linux Ubuntu. This allows me to do other things while Kodi is/can be running.
Thank you
RE: Slow Recently Added Episodes - RTam1990 - 2024-08-02
(2024-08-02, 00:53)RTam1990 Wrote: Yes, I know I am using an older version of Kodi and "Kodi by Debian". This was the version that was/is available in the apt repositories. As I said, I am unable and cannot update at this time. I also said it happened with previous versions. I was using Libreelec previously and the Recently added episodes was slow then as well, in my setup at least. Unfortunately, Libreelec is not available for the RK3588/Orange Pi currently. It looks like it may be a while, if ever, that it is available. Another reason that I'd rather not use Libreelec is that it is a standalone Kodi version While Kodi by Debian is running within Linux Ubuntu. This allows me to do other things while Kodi is/can be running.
Thank you
Oops, forgot to mention that I was unaware that Kodi by Debian had other changes...are you sure of that? Please point me to where I can see these changes...just curious.
RE: Slow Recently Added Episodes - RTam1990 - 2024-08-02
These are the things that I'm going to try to look into:
SQL
- slow query log
- general log
- trying to run the query that the slow response is using
- analysing the query
- are there any full table scans being done?
- review indices that are being used or not being used or missing
- any way to improve the query
- are there any full table scans being done?
Data
- determining the amount of data being retrieved by this section
Any other ideas?
Thank you
RE: Slow Recently Added Episodes - RTam1990 - 2024-08-02
(2024-08-02, 01:01)RTam1990 Wrote: analysing the query
Forgot to ask if I can be provided with the query or where to get it.
Also, can't figure out how to edit my own posts. Any assistance is appreciated.
Thank you
RE: Slow Recently Added Episodes - Hitcher - 2024-08-02
(2024-08-02, 01:02)RTam1990 Wrote: Also, can't figure out how to edit my own posts. Any assistance is appreciated.You haven't met the requirements yet.
RE: Slow Recently Added Episodes - Hitcher - 2024-08-02
(2024-08-02, 00:06)RTam1990 Wrote: It's the amount of time to render the sectionCould you post a video please?
RE: Slow Recently Added Episodes - RTam1990 - 2024-08-04
I found something in the slow query log.
This is the query:
select * from episode_view WHERE ((episode_view.dateAdded > '2012-01-01')) AND ((episode_view.playCount IS NULL OR episode_view.playCount < 1));
This looks like it's what the recently added episodes section is using.
The query is performing a full scan on some table, not currently sure which one(s). The number of rows being scanned for this query is about 60 million rows.
When I manually run the query, it seems to take the same amount of time that the Kodi interface takes to refresh the section.
I'm still looking into it. It seems there aren't any indices for the playCount and playAdded columns in the files table.
As for a video, not sure if I can do that. Not a high priority at this time. Also, it's pretty clear what I'm describing so not sure what a video will add to it.
RE: Slow Recently Added Episodes - Hitcher - 2024-08-04
When you say 'the amount of time to render' to me that means the images are taking a long time to get drawn, hence why I asked for a video. But it seems you meant for the list to get updated.
RE: Slow Recently Added Episodes - RTam1990 - 2024-08-04
Yes. It looks like the Movies page and the widgets on it, get completely rendered before the TV Shows page. On the TV Shows page, everything gets rendered except the Recently added episodes widget/section. There is only a spinning circle. Then, after about 4 minutes or so, the entire widge/section is displayed...images and navigable parts. I believe that is where the query is running and after the query completes and returns, then the images are displayed all at once.
Is the episode_view used elsewhere?
I'm asking as I was thinking of changing the episode view or trying to streamline it. That way, the remainder of the code is not affected. I've created a couple of indices on the episode table but it's not having any effect. I think there are still more indices to work on. From my understanding, it doesn't look like you can create an index on a view so that's why I'm working on the base tables. I'm not sure if the view will use those indices, but I thought it would.
One observation I made is that the more TV shows and episodes, this query becomes slower and slower, of course. The slow query log goes back quite a while and when there were many fewer shows and episodes, the response was good/great. I think that was the time I was reloading my library.
Another observation is that the number of rows examined is huge, currently at around 60 million, while the rows sent is small, currently at around 23 thousand.
Still working on it. One of the next things I'll look at is the SQL that created the episode_view. If there are a bunch of joins that aren't optimized, that could cause this.
If you can let me know other places that view is used, that will help me test any changes.
RE: Slow Recently Added Episodes - Hitcher - 2024-08-04
(2024-08-04, 01:58)RTam1990 Wrote: There is only a spinning circle.OK, that rules out anything to do with the skinning engine/skin code as it's querying the database.
(2024-08-04, 01:58)RTam1990 Wrote: Another observation is that the number of rows examined is huge, currently at around 60 millionHow many TV shows/episodes do you have?
RE: Slow Recently Added Episodes - pkscout - 2024-08-04
(2024-08-04, 00:46)RTam1990 Wrote: The query is performing a full scan on some table, not currently sure which one(s). The number of rows being scanned for this query is about 60 million rows.
You have 60 million episodes of TV shows? That seems well outside the norm and is likely causing the issue. I feel like there's something going on with your database if you've ended up with 60 million rows in the episodes table (the episodes_view mostly grabs from the episodes table).