Slow access to tvshow library with large MySQL database
#1
Hi,

I've been talking to Marantz/Sopparus about this and he told me that he had already went thru this with the Music Library a year ago (223731 (thread)).

When opening a tvshow in the library it queries which seasons are available for that show ID (which is as it should).
Problem is how the view query is constructed.
Code:
DEBUG: RunQuery took 7435 ms for 2 items query: SELECT * FROM season_view  WHERE season_view.idShow = 539

This is most likely not a problem with smaller databases (i myself haven't noticed it until recently, which is probably because it's been getting slower over a long period of time).

We did abit of tinkering with the queries and figured out that the issue is that season_view is requesting it's content from several other views.
Code:
SELECT    s.idSeason AS idSeason,
        s.idShow AS idShow,
        s.season AS season,
        s.name AS name,
        p.strPath AS strPath,
        t.c00 AS showTitle,
        t.c01 AS plot,
        t.c05 AS premiered,
        t.c08 AS genre,
        t.c14 AS studio,
        t.c13 AS mpaa,
        COUNT(distinct e.idEpisode) AS episodes,
        COUNT(f.playCount) AS playCount,
        MIN(e.c05) AS aired
FROM seasons AS s
        INNER JOIN tvshowlinkpath AS lp ON (s.idShow = lp.idShow)
        JOIN path AS p ON (p.idPath = lp.idPath)
        JOIN tvshow AS t ON (s.idShow = t.idShow)
        JOIN episode AS e ON (s.idShow = e.idShow) AND (e.c12=s.season)
        JOIN files AS f ON (e.idFile = f.idFile)
WHERE t.idShow=526
GROUP BY idSeason;

Only thing different really from the original view query is that this one doesn't retreive its content from the views, but from the base tables.
The query is almost instant when run manually, it does have a slightly slower response time when made into a view.

The result after the view was changed for kodi is still (for me 6 seconds) way faster.
Code:
DEBUG: RunQuery took 1468 ms for 1 items query: SELECT * FROM season_view  WHERE season_view.idShow = 844

I know that this is probably not in line with the Kodi way of doing this, but I wanted to make you aware of the issue and open a discussion on if/how to resolve it Smile
This probably happens to the other views aswell, but i haven't found a good way of speeding those up.
Reply
#2
This is normal with the way Mysql works.

You do not give your mysql version but I'm pretty sure you are still 5.5 or less, update to latest 5.6 and see improvements.
Reply
#3
Ah I'm sorry, just followed how Marantz wrote his post actually so I totally forgot.

Im using MySQL 5.7.12-0ubuntu1.1 running on Ubuntu 16.04.
Also Marantz said he has the same issues with my DB on MariaDB 10.1.
Reply
#4
this is explain extended on the original season view.

"id" "select_type" "table" "type" "possible_keys" "key" "key_len" "ref" "rows" "filtered" "Extra"
"1" "PRIMARY" "<derived2>" "ALL" \N \N \N \N "248970" "100,00" "Using where"
"2" "DERIVED" "episode" "ALL" "id_episode_file_2,ix_episode_season_episode,ix_episode_show2" \N \N \N "24897" "100,00" "Using where; Using temporary; Using filesort"
"2" "DERIVED" "seasons" "ref" "ix_seasons" "ix_seasons" "10" "MyVideos99.episode.idShow,MyVideos99.episode.c12" "1" "100,00" ""
"2" "DERIVED" "seasons" "eq_ref" "PRIMARY" "PRIMARY" "4" "MyVideos99.episode.idSeason" "1" "100,00" "Using index"
"2" "DERIVED" "tvshow" "eq_ref" "PRIMARY" "PRIMARY" "4" "MyVideos99.episode.idShow" "1" "100,00" "Using index"
"2" "DERIVED" "bookmark" "ref" "ix_bookmark" "ix_bookmark" "10" "MyVideos99.episode.idFile,const" "1" "100,00" "Using where; Using index"
"2" "DERIVED" "files" "eq_ref" "PRIMARY,ix_files" "PRIMARY" "4" "MyVideos99.episode.idFile" "1" "100,00" "Using where"
"2" "DERIVED" "path" "eq_ref" "PRIMARY" "PRIMARY" "4" "MyVideos99.files.idPath" "1" "100,00" "Using index"
"2" "DERIVED" "files" "eq_ref" "PRIMARY" "PRIMARY" "4" "MyVideos99.episode.idFile" "1" "100,00" ""
"2" "DERIVED" "<derived3>" "ref" "key0" "key0" "4" "MyVideos99.episode.idShow" "10" "100,00" ""
"3" "DERIVED" "tvshow" "index" "PRIMARY" "PRIMARY" "4" \N "281" "100,00" ""
"3" "DERIVED" "tvshowlinkpath" "ref" "ix_tvshowlinkpath_1" "ix_tvshowlinkpath_1" "5" "MyVideos99.tvshow.idShow" "1" "100,00" "Using index"
"3" "DERIVED" "path" "eq_ref" "PRIMARY" "PRIMARY" "4" "MyVideos99.tvshowlinkpath.idPath" "1" "100,00" "Using where"
"3" "DERIVED" "<derived4>" "ref" "key0" "key0" "4" "MyVideos99.tvshow.idShow" "54" "100,00" ""
"4" "DERIVED" "tvshow" "index" \N "PRIMARY" "4" \N "281" "100,00" "Using index"
"4" "DERIVED" "episode" "ref" "ix_episode_show2" "ix_episode_show2" "5" "MyVideos99.tvshow.idShow" "54" "100,00" ""
"4" "DERIVED" "files" "eq_ref" "PRIMARY" "PRIMARY" "4" "MyVideos99.episode.idFile" "1" "100,00" "Using where"



with modified view and not using views

"id" "select_type" "table" "type" "possible_keys" "key" "key_len" "ref" "rows" "filtered" "Extra"
"1" "PRIMARY" "<derived2>" "ALL" \N \N \N \N "15174" "100,00" "Using where"
"2" "DERIVED" "tvshow" "ALL" "PRIMARY" \N \N \N "281" "100,00" "Using where; Using temporary; Using filesort"
"2" "DERIVED" "tvshowlinkpath" "ref" "ix_tvshowlinkpath_1,ix_tvshowlinkpath_2" "ix_tvshowlinkpath_1" "5" "MyVideos99.tvshow.idShow" "1" "100,00" "Using where; Using index"
"2" "DERIVED" "path" "eq_ref" "PRIMARY" "PRIMARY" "4" "MyVideos99.tvshowlinkpath.idPath" "1" "100,00" ""
"2" "DERIVED" "episode" "ref" "id_episode_file_2,ix_episode_season_episode,ix_episode_show2" "ix_episode_show2" "5" "MyVideos99.tvshow.idShow" "54" "100,00" "Using where"
"2" "DERIVED" "seasons" "ref" "ix_seasons" "ix_seasons" "10" "MyVideos99.tvshow.idShow,MyVideos99.episode.c12" "1" "100,00" ""
"2" "DERIVED" "files" "eq_ref" "PRIMARY" "PRIMARY" "4" "MyVideos99.episode.idFile" "1" "100,00" ""
Reply
#5
Sadly right Now kodi uses really complex views that create optimization problems.. Season_view is the slower one.. It's a view of a view of a view... MySQL will never be able to optimize that
Reply
#6
(2016-07-04, 10:08)phate89 Wrote: Sadly right Now kodi uses really complex views that create optimization problems.. Season_view is the slower one.. It's a view of a view of a view... MySQL will never be able to optimize that

it is possible to replace the view with the one that doesnt use other views?
Reply
#7
Without changing how kodi behaves you can only do it with nested queries having the same problem..


Inviato dal mio iPhone utilizzando Tapatalk
Reply
#8
ok, well for others reading using the query above cuts the time in half and from what I can tell after a few days using it, there are no issues.

if you just copy paste the query in the first post and put it in the view, remove line WHERE t.idShow=526 and its good to go
Reply
#9
(2016-07-04, 12:13)marantz Wrote: ok, well for others reading using the query above cuts the time in half and from what I can tell after a few days using it, there are no issues.

if you just copy paste the query in the first post and put it in the view, remove line WHERE t.idShow=526 and its good to go
That query isn't OK. A TV show can have more than one path. This means you will get wrong counts and avg if there is more than one path per TV show
Reply
#10
Bummer

Skickat från min E6553 via Tapatalk
Reply

Logout Mark Read Team Forum Stats Members Help
Slow access to tvshow library with large MySQL database0