[RELEASE] Texture Cache Maintenance utility
#68
v0.3.1 - a pretty major update. Smile
  • Multiple thread support added to the c and C options. Default is 2 threads, set your own value using the download.threads = n property. This is a "global" thread setting for all media classes (albums, movies, tvshows etc.), and you can individually fine-tune the number of threads by specifying additional per-class properties, eg. download.threads.albums = 2, download.threads.songs = 5, download.threads.tvshows = 10, and for everything else downloads.threads will be used.

  • New option: nc. This is essentially the same as c but doesn't download anything - think of this as a dry run version of c, it will show you what needs to be cached. Isn't multi-threaded, as there is no benefit when just parsing data.

  • Lots of code re-factoring. I may have broken something, and some things that used to work may no longer work on older versions of Python. Sorry. Only able to test with v2.7.3, though a very quick spin on a Dharma/Python v2.6.6 machine didn't highlight anything obvious.

  • Sanity checks added at script initialisation, so if your web server isn't running and it's needed for the option you're trying to use, then the script will inform you, and abort. Same for the JSON-RPC interface, JSON API version, and the sqlite3 Texture13 database.

  • Source code now hosted on github: https://github.com/MilhouseVH/texturecache.py. Just download the latest version of texturecache.py rather than the whole repository, though you can do that too if you like.

Multi-threaded download has been tested on OpenELEC/Raspberry Pi, Ubuntu 12.10/x86 and also Windows (by charrua - many thanks).

Running with 10 threads is not a problem, not even for the Raspberry Pi (though see [1], below)! Caching performance with additional threads is significantly improved, though obviously beyond a certain number of threads it's a case of diminishing returns, so don't go too mad... Wink

Enjoy.

1. Just one caveat... and it's a fairly major one... with the Raspberry Pi I'm seeing the web server fall over and effectively die - no further web requests accepted - and then a short while later taking the whole Pi with it, necessitating a power cycle.

This problem seems to be due to OMX errors (log of failure - nothing more appears after this - and below it, from a separate run, this time successful)... So far I'm only seeing this problem with tvshows on my Raspberry Pi, but that could just be random luck as it may be image dependent, and I've not seen this problem on Ubuntu, or heard of an issue on Windows. For all other media classes - albums, artists, songs, sets, movies, tags - these can pre-load with 10 threads, not a problem, but tvshows, just using one thread: power cycle. Hmm.

It's definitely not multi-thread related as I can provoke the problem with the old v0.3.0 "serial" version, and with the new version when download.threads.tvshows = 1, so effectively single-threaded. It even happens on the very first web request (which is for Files.Preparedownload), and sometimes after 900+, so definitely looks like an internal XBMC issue on Raspberry Pi (tested both stock OE release and latest Rbej build, unfortunately no difference).

Edit: Looking at that debug log again, it might not be Files.Preparedownload but the actual download - the incoming request isn't obvious in the log - and subsequent caching that is causing the webserver to fall over, certainly after this situation has occurred, no further requests will be serviced by the web server, and eventually the entire Pi will hang. When this problem occurs, the GUI will freeze immediately.

Edit2: Now fixed, this caveat no longer applies. See here - make sure Pi has 29 March firmware or later.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply


Messages In This Thread
RE: [RELEASE] Texture Cache maintenance utility - by Milhouse - 2013-03-29, 14:59
Crash on Gotham on OS X - by desepticon - 2014-05-29, 17:57
Cleaning - by AleisterHH - 2018-05-28, 22:03
RE: Cleaning - by Milhouse - 2018-05-28, 22:16
qax genre not updated - by Just-Me_A-User - 2018-06-12, 22:06
RE: qax genre not updated - by Milhouse - 2018-06-12, 23:40
Logout Mark Read Team Forum Stats Members Help
[RELEASE] Texture Cache Maintenance utility17