[Croneter]

After spending several hours trying to improve the whole code responsible for adding / writing media items retrieved from a media provider into Kodi's database I realized that I only had to adjust the SQL transaction handling to improve the performance by a factor of up to 50. I only tested this on SQLite (so no idea what the performance with MySQL is) and the improvements vary depending on whether the SQLite database is located on a HDD or SDD but either way the performance improvement should be significant. On my dev computer importing 60 movies to an HDD took ~80s and on an SSD it took ~9s. With the improvements this went down to ~1.4s. Now when importing a large library the main time is spent on retrieving the media items from the media provider - adding the retrieved items to the database is very fast.

And last but not least these new test builds are based on Kodi v19 (final).

Downloads: https://github.com/Montellese/xbmc/wiki/...#downloads
Changelog: https://github.com/Montellese/xbmc/wiki/...erformance

I found the same thing with regards to transactions, specifically commits.  A commit with each record insert or similar will be very slow and as you observed storage media dependent on the severity of the slowness.  I batch up inserts and then do periodic commits.  It made a huge difference in performance.  Similar to what you've seen.  On my Intel NUC with an SSD I can get around 200 inserts a second.  13K records in just over a minute.


Jeff

. I just imported ~9K movies from a Plex media server and it took 95 minutes of which 85 minutes were spent on retrieving the items and 10 minutes were spent on writing the items to the database.

I'd think that PlexKodiConnect is a couple of magnitudes faster than that, though I can't test on a large library. Took months to improve... Have you looked at its code, specifically the MySQL and Plex metadata retrieval part?