Problem scanning huge source (mem leak)?
#69
All good stuff @Rusendusen, thank you. I think I have some ideas now about some of it at least, and that is always the first step to a solution.

You have nearly 371k of artists (it includes musicians, producers, etc.) and because after the small change none of them need to be deleted as part of the clean up all of them are in tmp_delartists. While the select query works with
"WHERE idArtist NOT IN (SELECT idArtist FROM tmp_delartists)"
when you test it (and says 0 records), delete with the same clause quietly raises a MySQL 1205 timeout error, or just hangs.

(EDITED AGAIN)
I may have a fix, confim this you could apply this SQL manually (from Workbench or whatever)
Code:
SELECT COUNT(1) as artistcount FROM artist;
CREATE TEMPORARY TABLE tmp_delartists (idArtist integer) CHARACTER SET utf8 COLLATE utf8_general_ci;
INSERT INTO tmp_delartists select idArtist from song_artist;
INSERT INTO tmp_delartists select idArtist from album_artist;
INSERT INTO tmp_delartists VALUES(1);
SELECT COUNT(1) as artists_raw FROM tmp_delartists;
CREATE TEMPORARY TABLE tmp_keep (idArtist integer primary key) CHARACTER SET utf8 COLLATE utf8_general_ci;
INSERT INTO tmp_keep SELECT DISTINCT idArtist from tmp_delartists;
SELECT COUNT(1) as artists_keep FROM tmp_keep;
SELECT COUNT(1) as artists_remove FROM artist WHERE idArtist NOT IN (SELECT idArtist FROM tmp_keep);
DELETE FROM artist WHERE idArtist NOT IN (SELECT idArtist FROM tmp_keep);
That last line should do nothing, there is nothing to delete, but it should not lock.

Both clean db from settings, and the smaller clean up at the end of library update (does not remove songs from missing sources in case they are temp off line), use the same SQL for cleaning artists. So you will hit this issue from either route.

I need to have a closer look at clean on update to see why cancel isn't working for you, but again I have ideas.

Kodi has never had so many artists, and MySQL just does not like the way it tries to clean up with that many.
Reply


Messages In This Thread
RE: Problem scanning source - by DaveBlake - 2017-02-12, 13:00
RE: Problem scanning source - by Uatschitchun - 2017-02-12, 21:51
RE: Problem scanning huge source (mem leak)? - by DaveBlake - 2017-03-03, 11:42
Logout Mark Read Team Forum Stats Members Help
Problem scanning huge source (mem leak)?0