The script has been tested with local installations of XBMC Frodo on OpenELEC R-Pi and Ubuntu 12.10, and also Windows with Python 2.7.3 running against a remote Frodo installation of XBMC. A separate properties file can be used to specify non-default configurations (see property file details at the end of this post).
This is not an addon, and it requires that the webserver is enabled in XBMC on port 8080 (unless another port is specified in the properties file). It also assumes you are comfortable working at the command line (ssh in Linux, a large CMD window may also work OK for Windows users).
Summary of features
- [c, C] Automatically cache missing artwork (c), or with option C force the re-caching of existing artwork by removing first then downloading. Can use multiple threads (default is 2)
- [nc] Identify those items that require caching (and would be cached by c option)
- [lc, lnc] Same as c and nc, but only considers those media (movies, tvshows/episodes) added since the modification timestamp of the file identified by the property lastrunfile
- [p, P] Prune texture cache by identifying (p) or removing (P) accumulated cruft such as image previews, previously deleted movies/tv shows/music whose artwork remains in the texture cache even after cleaning the database. Essentially, remove any cached file that is no longer associated with an entry in the media library and is therefore just wasting disk space
- [s, S] Search texture cache database for specific entries and view database content - can help explain reasons for incorrect artwork. S will return only results for items that no longer exist in the Thumbnails folder (but are still in the database).
- [x, X] Extract rows from texture cache database, with optional SQL filter. X will return only those database results for items that no longer exist in the Thumbnails folder
- [Xd] Remove those rows from the texture cache database that do not have corresponding files in the Thumbnails folder (ie. remove same rows identified by X)
- [d] Delete specific database rows and corresponding files from the texture cache using database row identifier (see s/S/x/X)
- [r, R] Reverse query cache, identifying "orphaned" files that are no longer referenced in the texture cache database. Use R option to auto-delete these files.
- [j, J, jd, Jd] Query media library using JSON API, and export content as JSON (suitable for further external processing). J and Jd will include additional user defined fields; jd and Jd will "decode" (unquote) all URLs
- [qa] Perform QA check on media library recently added items, identifying missing properties (eg. plot, mpaa certificate, artwork etc.). Default QA period is previous 30 days. Add property qa.file = yes to verify file exists during QA. Add additional QA fields using qa.art.*, qa.blank.* and qa.zero.* properties.
- [qax] Like the qa option, but performs a remove and then rescan of any media found to fail the QA tests
- [set, testset] Set values on movie, set, tvshow, episode, musicvideo, artist, album and song. Pass parameters on the command line, or as a batch of data read from stdin. testset will perform a dry-run. See [setting fields](#setting-fields-in-the-media-library) for more details.
- [remove] Remove specified library item from media library, ie. remove movie 123
- [imdb] Update IMDb fields on movies and tvshows. Pipe output into set option to apply changes to the media library using JSON. Uses http://www.omdbapi.com to query latest IMDb details based on imdbnumber (movies) or title and season/pisode # (tvshows). Movies without imdbnumber will not be updated. Specify movie fields to be updated with the property @imdb.fields.movies from the following: top250, title, rating, votes, year, runtime, genre, plot and plotoutline - the default value is: rating, votes, top250. Specify tvshow fields to be updated with the property @imdb.fields.tvshows from the following: title, rating, votes, year, runtime, genre, plot and plotoutline. Specify additional fields to the default by prefixing the list with +, eg. @imdb.fields.movies=+year,genre to update movie ratings, votes, top250, year and genre. See log for old/new values. See announcement for further important details.
- [purge hashed|unhashed|all] Delete cached artwork containing specified patterns, with or without lasthashcheck, or if it doesn't matter all eg. purge unhashed youtube iplayer imdb.com
- [purgetest hashed|unhashed|all] Dry-run version of purge - will show what would be removed during an actual purge
- [watched] Backup and restore movie and tvshow watched lists to a text file. Watched list will be restored keeping more recent playcount, lastplayed and resume points unless property watched.overwrite=yes is specified, in which case the watched list will be restored exactly as per the backup.
- [duplicates] List movies that appear more than once in the media library with the same imdb number
- [missing] Locate media files missing from the specified media library and source label, eg. missing movies "My Movies"
- [ascan, vscan] Initiate audio/video library scan, either entire library or a specific path (see sources). The exit status is the number of items added during each scan, ie. 0 or +n.
- [aclean, vclean] Clean audio/video library
- [directory] Obtain directory listing for a specific path (see sources)
- [sources] List of sources for a specific media class (video, music, pictures, files, programs)
- [status] Display status of client - ScreenSaver active, IsIdle (default period 600 seconds, or user specified) and active Player type (audio or video), plus title of any media currently being played.
- [monitor] Display client event notifications as they occur
- [power] Set power state of client - suspend, hibernate, shutdown or reboot
- [wake] Use Wake Over LAN to wake a suspended/hibernating remote client. Specify the MAC address of the remote client in the network.mac property (ie. network.mac=xx:xx:xx:xx:xx:xx). When the client is no longer required, suspend or hibernate it with the relevant power option
- [exec, execw] Execute the specified addon, with optional parameters. eg. exec script.artwork.downloader silent=true mediatype=tvshow. Use execw to wait, but this rarely has any effect (possibly not implemented by JSON?)
- [setsetting] Set the value of the named setting, eg. setsetting locale.language English
- [getsetting] Get the current value of the named setting, eg. getsetting locale.language
- [getsettings] View details of all settings, or those where pattern is contained within id, eg. getsettings debug to view details of all debug-related settings
- [debugon, debugoff] Enable/Disable debugging
- [play, playw] Play specified item (local file, playlist, internet stream etc.), optionally waiting until playback is ended
- [stop] Stop playback
- [pause] Toggle pause/playback of currently playing media
- [config] View current configuration
- [version] View current installed version
- [update] Manually update to latest available version if not already installed. Only required if checkupdate or autoupdate properties are set to no as by default the script will automatically update itself (if required) to the latest version whenever it is executed.
For instructions on how to run a Python script, see here.
Windows users will need to install Python 2.7.x or Python 3.3.x, if not already installed (see how-to).
Download the single Python script file - use "Save as" in your browser - from github. A default properties file is available on github, rename this to texturecache.cfg in order to use it, although in many cases it's not required and should just be considered a template - pick and choose what options you wish to override.
Alternatively, non-Windows users can download the script directly at the command line using curl:
curl https://raw.githubusercontent.com/MilhouseVH/texturecache.py/master/texturecache.py -o texturecache.py
chmod +x ./texturecache.py
ATV2 (iOS) users
Python 2.6+ is required to run this script, and although Python can be installed on iOS using "apt-get install python", the version installed (typically v2.5.1 - check with "python --version") is very old and lacks language features required by the script. It is possible to install a more recent Python 2.7.3 package as follows:
rm -f python*.deb
dpkg -i python*.deb
Let's say the poster image for my "Dr. No" movie is corrupted, and I want to delete it so that XBMC will automatically re-cache it (hopefully correctly) next time it is displayed:
1) Execute: ./texturecache.py s "Dr. No" to search for my Dr. No related artwork
2) Several rows should be returned, relating to different cached artwork - one row will be for the poster, the other fanart, and there may also be rows for other image types too (logo, clearart etc.). This is what I get:
000226|5/596edd13.jpg|0720|1280|0011|2013-03-05 02:07:40|2013-03-04 21:27:37|nfs://192.168.0.3/mnt/share/media/Video/Movies/James Bond/Dr. No (1962)[DVDRip]-fanart.jpg
000227|6/6f3d0d94.jpg|0512|0364|0003|2013-03-05 02:07:40|2013-03-04 22:26:38|nfs://192.168.0.3/mnt/share/media/Video/Movies/James Bond/Dr. No (1962)[DVDRip].tbn
Matching row ids: 226 227
3) Since I want to remove only the poster (.tbn) I can execute ./texturecache.py d 227 and both the database row and cached poster image will be deleted. If I wanted to remove both images, I would simply execute ./texturecache.py d 226 227 and the two rows and their corresponding cached images will be removed.
Alternatively, ./texturecache.py C movies "Dr. No" would achieve the same result including re-caching the deleted items.
Format of "dumped" records
When displaying rows from the database, the following fields (columns) are shown:
rowid, cachedurl, height, width, usecount, lastusetime, lasthashcheck, url
Additional usage examples
To view your most recently accessed artwork:
./texturecache.py x | sort -t"|" -k6
-- or --
./texturecache.py x "order by lastusetime asc"
or your top 10 accessed artwork...
./texturecache.py x | sort -t"|" -k5r | head -10
-- or --
./texturecache.py x "order by usecount desc" 2>/dev/null | head -10
Use texturecache.py to identify artwork for deletion, then cutting and pasting the matched ids into the "d" option or via a script:
For example, to delete those small remote thumbnails you might have viewed when downloading artwork (and which still clutter up your cache):
./texturecache.py s "size=thumb"
<then cut & paste the ids as an argument to ./texturecache.py d>
and the same, but automatically:
IDS=$(./texturecache.py s "size=thumb" 2>&1 1>/dev/null | sed "s/.*: //")
[ -n "$IDS" ] && ./texturecache.py d $IDS
-- or --
Delete any artwork that has not been accessed since a particular date:
./texturecache.py x "where lastusetime <= '2013-03-05'"
or hasn't been accessed more than once:
./texturecache.py x "where usecount <= 1"
Query the media library returning JSON results:
First, just the default fields for a particular class (movies), filtering for a specific item (Avatar):
./texturecache.py j movies "Avatar"
With "extrajson.movies = trailer, streamdetails, file" in the properties file, the same query but returning the extra fields too:
./texturecache.py J movies "Avatar"
"file": "nfs://192.168.0.3/mnt/share/media/Video/Movies/Avatar (2009)[DVDRip].m4v",
Optional Properties File
By default the script will run fine on distributions where the .xbmc/userdata folder is within the users Home folder (ie. userdata=~/.xbmc/userdata). To override this default, specify a properties file with a different value for the userdata property.
The properties file should be called texturecache.cfg, and be in the same directory as the texturecache.py script. What follows is an example properties file with the default values shown - DO NOT paste this file as your own properties, set only the properties you require in your own file:
sep = |
userdata = ~/.xbmc/userdata
dbfile = Database/Textures13.db
thumbnails = Thumbnails
xbmc.host = localhost
webserver.port = 8080
rpc.port = 9090
download.threads = 2
qaperiod = 30
qa.file = no
qa.fail.urls = ^video, ^music
cache.castthumb = no
cache.hideallitems = no
cache.ignore.types = ^video
prune.retain.previews = yes
prune.retain.pictures = no
logfile.verbose = yes
checkupdate = yes
orphan.limit.check = yes
watched.overwrite = no
imdb.fields.movies = rating, votes, top250
imdb.fields.tvshows = rating, votes
purge.minlen = 5
bin.tvservice = /usr/bin/tvservice
hdmi.force.hotplug = no
The dbfile and thumbbnails properties represent folders that are both relative to the userdata property.
Specify webserver.username and webserver.password if you require webserver authentication.
extrajson.* properties allow the specification of additional JSON audio/video fields to be returned by the J query option.
Cast thumbnails will not be updated by default, so specify cache.castthumb = yes if you require cast artwork to be re-cached.
Ignore specific URLs when pre-loading the cache (c/C options), by specifying comma delimited regex patterns for the cache.ignore.types property. Default value is image://video. Set to None to process all URLs. Any matching URL will not be cached (c/C), nor will it be deleted when forcing a reload (option C).
Specify a filename for the logfile property, to log detailed processing information.
See github for full changelog and further details in README.
Run the script without arguments for usage and current configuration information.
Hopefully someone else will find this useful!
For mklocal.py, see here
For explanation of dropped/skipped/ignored, see here