• 1
  • 115
  • 116
  • 117(current)
  • 118
  • 119
  • 197
[RELEASE] Texture Cache Maintenance utility
(2015-11-18, 03:31)MANswers Wrote: By the way, can i ask a very basic stupid question, whats the point of cleaning your thumbnails? what thumbnails? why use this utility, how does it benifit

Kodi caches artwork - both local (LAN) and remote (http) artwork - in the "texture cache" which consists of a local database (Textures13.db) and the Thumbnails folder. It does this for a few reasons, though mainly to improve performance when displaying artwork as Kodi doesn't have to keep downloading it from the web or local LAN each time, also so Kodi can optimise the format and size of the cached artwork suitable for the display device (low powered devices might use smaller, lower quality artwork than a "big" i7 system, or might be able to hardware decode some jpg formats but not others - the cached artwork will always be hardware decodable).

Unfortunately Kodi isn't very good at removing old items from the texture cache - each time you browse through the available videos in the YouTube add-on, those thumbnails of YouTube videos are being added to your cached, and are never removed. Movies or TV Shows and episodes that are removed from the library never have their thumbnails removed from the cache. This can lead to the cache "bloating" with hundreds, possibly thousands, of thumbnails you'll never display again.

This "bloat" can result in the cache increasing in size to many GBs, and some systems, eg. Apple TV, Android etc., have only limited space available for the texture cache, so "cleaning" the cache every now and again is a means to reclaim the available space by deleting the thumbnails you no longer need, while still keeping the thumbnails you do use.

It's probably worth cleaning the cache once a month. You can schedule this script to run using cron (or whatever the Windows equivalent is), either on the Kodi client itself or from a central system. The command to clean the texture cache is "texturecache.py P" (P for prune, as that's what it does... selectively getting rid of the useless stuff). It's not a quick process, as it has to do a lot of work and can take a few minutes, so running it overnight when you're not using the systems would be advised - you certainly don't want to run it at startup.

Once you've started to schedule one maintenance job, scheduling others becomes a piece of cake... pre-loading the cache with all artwork ("texturecache c"), updating imdb ratings and votes for movies and tvshows, eliminating artwork from the cache that Kodi has forgotten about ("texturecache.py R") or that Kodi thinks is in the cache when it's not ("texturecache.py Xd" - although this last one is best run on the client itself), etc. etc.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
@Milhouse
thx for your quick answer.
Quote:
(2015-11-18, 00:10)Nepomuk Wrote: can you make a example for a windows-batch-file where more then on command is in use please?

It's just a text file, one command on each line...

Code:
@echo off
c:\texturecache.py c movies
c:\texturecache.py c tvshows
c:\texturecache.py imdb movies | c:\texturecache.py set
c:\texturecache.py imdb tvshows | c:\texturecache.py set
In one batch-file? So the batch processing the commands step by step? I mean process the 1. step, waiting until 1. step is finished, the 2. step and so on?
The 3. and 4. command will update the imdb information, that are defined in the cfg-file. Is that right? And where will the script write the updated informations - only in the kodi media libary (Mysql-db) or in the movie.nfo too?

Quote:would work, assuming you have texturecache.py and (if needed) texturecache.cfg in c:\, and the python path is added to your PATH variable (see guide for Windows users).

(2015-11-18, 00:10)Nepomuk Wrote: Where I can use the sep = | in the texturecache.cfg? For what's that seperator?

I have tried to persuade people *NOT* to use values in their texturecache.cfg that they don't need (because they're the same as the default value), or understand. I've provided a sample file on github, which was probably a mistake, and it's not intended that you use it verbatim/as-is - most people don't need a configuration file at all.
I don't use the cfg file but I will understand for what this variables are and what I can change here.

Quote:Regarding the sep field, it's the separator between fields in the output from the "s" option, eg.:
Code:
./texturecache.py s avatar @sep=#
014301#f/f6fd12f4.png#0562#1000#0002#2015-07-10 03:27:49#2015-07-10 04:27:48#nfs://192.168.0.3/mnt/share/media/Video/MoviesSD/Avatar (2009)[DVDRip]-clearart.png
014829#0/05ae89b1.png#0310#0800#0002#2015-07-10 03:31:25#2015-07-10 04:31:10#nfs://192.168.0.3/mnt/share/media/Video/MoviesSD/Avatar (2009)[DVDRip]-logo.png
015466#d/d843551b.jpg#0000#0000#0294#2015-09-13 22:27:39#2015-09-13 23:27:01#nfs://192.168.0.3/mnt/share/media/Video/MoviesSD/Avatar (2009)[DVDRip]-fanart.jpg
016161#c/cfdfd21f.jpg#0000#0000#0329#2015-10-26 08:23:13#2015-10-25 23:27:09#nfs://192.168.0.3/mnt/share/media/Video/MoviesSD/Avatar (2009)[DVDRip]-poster.jpg
Matching row ids: 14301 14829 15466 16161

I very much doubt there's any reason to change the value of sep, but the option is there...
Now I understand that and I don't need the var sep.

Quote:
(2015-11-18, 00:10)Nepomuk Wrote: When I have a texturecache.cfg in the same folder as texturecache.py - have I use it in the command string like e.g. "texturecache.py C movies @config=texturecache.cfg" or can I omitting "@config=texturecache.cfg"?

No, it will be used automatically. If you use the "config" option it will display the name and path of the texturecache.cfg being used:
Code:
~/projects$ ./texturecache.py/texturecache.py config
Current properties (if exists, read from /home/neil/projects/texturecache.py/texturecache.cfg):
...

The @config property is only needed when the configuration file is not found in one of the default locations, or is not using the default name (texturecache.cfg).
Thx for this declaration.

Quote:
(2015-11-18, 00:10)Nepomuk Wrote: I have scraped my movies and TVshows with EmberMediamanager and not with kodi. So all needed files e.g. movie.nfo, movie-fanart.jpg, movie-clearlogo.png ... are local in every movie folder. The Thumbnails-Folder is substitute to my server. This folder blow up more and more. Now I want to synch mysql-kodi-db and every local Textures13.db with the movie files.

Path substitution of the Thumbnails folder when shared between multiple clients is explained quite clearly in the Kodi Wki (see contents of the orange box) as being an unsupported option, and one that doesn't work. Just stop using it, and use local Thumbnails folders on each client or provide each client with their own network folder, but whatever you do, do not share Thumbnails folders.
Thats right and I had read this notice. And I know that this method is unsupported by kodi. In the paste the only problem I get was that after some month several arts are not present at some clients, so I deleted the Thumbnail-folder and the textures13.db on every client, but I got the tip to use your script to hold folder and db clean.

Quote:
(2015-11-18, 00:10)Nepomuk Wrote: I do not want that texturecache.py download informationen from the WAN and save this in the different db's or in the .nfo-file. Do I need a special parameter that texturecache.py read/use only the local files or informations?

If you don't want the script to cache remote artwork (ie. artwork where the path begins with http) then add the following to your texturecache.cfg:
Code:
cache.ignore.types=+^http

and this will add (hence the +) the ^http pattern to the existing ignored "types" (the default value is " ^video, ^music", which ignores embedded video and music artwork, as these can't be cached by this script).
That's fine when I can block it by adding this string to cfg-file.

Quote:
(2015-11-18, 00:10)Nepomuk Wrote: Does texturecache.py synch the extra-arts and special-arts like extrathumb.jpg or clearlogo.png too?

Yes, "extra-arts" will be cached if you add the following to texturecache.cfg:
Code:
cache.extrafanart = yes
cache.extrathumbs = yes

Also, VideoExtras is supported:
Code:
cache.videoextras = yes

The default for all of the above settings is "no".

"special-arts" (clearlogo.png etc.) will be cached automatically, as will any artwork type referenced by your media library, eg. discart, clearart etc.

Use "cache.artwork" to cache specific artwork, eg. "cache.artwork=poster,fanart" would cache only poster and fanart - the default is to cache all artwork referenced by the media library.

Note that extrafanart/extrathumbs/videoextras artwork is not referenced by the media library, hence the separate options to enable this functionality (which is barely tested, by the way, as I doubt anyone has used it until now...).
Thx, I will added this three strings to cfg-file.
Where I can enable the separate options for extrafanart/extrathumbs/videoextras artwork in kodi? I have this files in the movie-folders but I don't find this option in kodi itself.

Quote:
(2015-11-18, 00:10)Nepomuk Wrote: I do not understand the principle/expiration of texturecache.py.
Does the script look first in the kodi.db (mysql-db), find the path to a movie, synch then textures13.db
or
does the script look in the movie sources first then add/synch the informations in kodi.db then synch textures13.db?

The former - the script compares the artwork paths from the media library against the artwork paths from the texture cache, and caches whatever is missing. In the case of extrafanart/extrathumbs/videoextras artwork, the script scans the directories for the relevant artwork (and then compares against the texture cache).
Now I understand the method of your script more - a little bit more. Big Grin

(2015-11-18, 06:27)Milhouse Wrote:
(2015-11-18, 03:31)MANswers Wrote: By the way, can i ask a very basic stupid question, whats the point of cleaning your thumbnails? what thumbnails? why use this utility, how does it benifit

Kodi caches artwork - both local (LAN) and remote (http) artwork - in the "texture cache" which consists of a local database (Textures13.db) and the Thumbnails folder. It does this for a few reasons, though mainly to improve performance when displaying artwork as Kodi doesn't have to keep downloading it from the web or local LAN each time, also so Kodi can optimise the format and size of the cached artwork suitable for the display device (low powered devices might use smaller, lower quality artwork than a "big" i7 system, or might be able to hardware decode some jpg formats but not others - the cached artwork will always be hardware decodable).
The best would be when we can change this in the kodi-options - maybe in the future. Angel

Quote:Unfortunately Kodi isn't very good at removing old items from the texture cache - each time you browse through the available videos in the YouTube add-on, those thumbnails of YouTube videos are being added to your cached, and are never removed. Movies or TV Shows and episodes that are removed from the library never have their thumbnails removed from the cache. This can lead to the cache "bloating" with hundreds, possibly thousands, of thumbnails you'll never display again.

This "bloat" can result in the cache increasing in size to many GBs, and some systems, eg. Apple TV, Android etc., have only limited space available for the texture cache, so "cleaning" the cache every now and again is a means to reclaim the available space by deleting the thumbnails you no longer need, while still keeping the thumbnails you do use.

It's probably worth cleaning the cache once a month. You can schedule this script to run using cron (or whatever the Windows equivalent is), either on the Kodi client itself or from a central system. The command to clean the texture cache is "texturecache.py P" (P for prune, as that's what it does... selectively getting rid of the useless stuff). It's not a quick process, as it has to do a lot of work and can take a few minutes, so running it overnight when you're not using the systems would be advised - you certainly don't want to run it at startup.

Once you've started to schedule one maintenance job, scheduling others becomes a piece of cake... pre-loading the cache with all artwork ("texturecache c"), updating imdb ratings and votes for movies and tvshows, eliminating artwork from the cache that Kodi has forgotten about ("texturecache.py R") or that Kodi thinks is in the cache when it's not ("texturecache.py Xd" - although this last one is best run on the client itself), etc. etc.
Are this your recommend commands AND in that order to hold Thumbnails-folder and local texture13.db clean and up to date:
With only this texturecache.cfg:
Code:
cache.ignore.types=+^http
cache.extrafanart = yes
cache.extrathumbs = yes
cache.videoextras = yes
cache.castthumb = yes
Code:
@echo off
texturecache.py P
texturecache.py c movies
texturecache.py c sets
texturecache.py c tvshows
texturecache.py imdb movies | c:\texturecache.py set   <-- maybe only who want this
texturecache.py imdb tvshows | c:\texturecache.py set  <-- maybe only who want this
texturecache.py R
texturecache.py Xd
What is with the "qa" or "qax" command - I don't use it or it's better to insert in the bath file and when so in which order?
Reply
(2015-11-18, 14:33)Nepomuk Wrote: In one batch-file?

If you like, yes.

(2015-11-18, 14:33)Nepomuk Wrote: So the batch processing the commands step by step? I mean process the 1. step, waiting until 1. step is finished, the 2. step and so on?

Yes.

(2015-11-18, 14:33)Nepomuk Wrote: The 3. and 4. command will update the imdb information, that are defined in the cfg-file. Is that right?

Yes.

(2015-11-18, 14:33)Nepomuk Wrote: And where will the script write the updated informations - only in the kodi media libary (Mysql-db) or in the movie.nfo too?

Only to the media library, the script never makes any changes to nfo files.

(2015-11-18, 14:33)Nepomuk Wrote: Where I can enable the separate options for extrafanart/extrathumbs/videoextras artwork in kodi?

You need to install a third-party skin that supports Extrafanart/Extrathumbs. VideoExtras is an addon, and probably only works if the skin supports it. I've never used any of these features/add-ons personally - I suggest you read the Kodi Wiki, look elsewhere on the forum where this sort of thing is discussed, and/or contact the skin/add-on authors for support.

(2015-11-18, 14:33)Nepomuk Wrote: Are this your recommend commands AND in that order to hold Thumbnails-folder and local texture13.db clean and up to date:
With only this texturecache.cfg:
Code:
cache.ignore.types=+^http
cache.extrafanart = yes
cache.extrathumbs = yes
cache.videoextras = yes
cache.castthumb = yes
Code:
@echo off
texturecache.py P
texturecache.py R
texturecache.py Xd
texturecache.py c movies
texturecache.py c sets
texturecache.py c tvshows
texturecache.py imdb movies | c:\texturecache.py set   <-- maybe only who want this
texturecache.py imdb tvshows | c:\texturecache.py set  <-- maybe only who want this


I changed the order of the commands slightly - it would make more sense to run the "consistency check" (P, R, Xd) commands first, followed by the caching ("c") commands, then the imdb updates.

You may need to add the c:\ path to every call to texturecache.py, but this all depends on how you've installed it, and where.

I'm assuming you're running the script on the Kodi client, as "texturecache.py Xd" will only work if it has direct access to the Thumbnails folder - all the other commands you've listed should work remotely across the network, if you add "kodi.host=<ip_address>" where <ip_address> is the ip address of the Kodi client you wish to update.

(2015-11-18, 14:33)Nepomuk Wrote: What is with the "qa" or "qax" command - I don't use it or it's better to insert in the bath file and when so in which order?

The qa/qax options will run quality checks on your media library, looking for movies with missing artwork, missing plots etc. - qa will report problems, qax will report problems and, depending on your configuration, remove and reload movies/episodes. For instance with qax, movies/episodes that are missing metadata can be refreshed (remove/re-scan) in the hope the missing metadata will be loaded (either by scraping new information from the internet, or by loading an updated nfo). Movies or episodes that are missing artwork can be reloaded once the artwork is found locally.

I use qax on my tvshows to flag up any episodes with missing plots, and to reload new episodes that are missing thumbs - quite often the thumb won't be downloaded until a day or so after the episode, and a daily qax will automatically refresh the episode once the thumb is available.

The default qa/qax settings will identify empty/missing plots, missing standard (poster/fanart/thumb) artwork. One thing to be aware of is that by default it only checks movies/episodes added in the previous 30 days, so you might want to set qaperiod=9999 in order to check your entire library.

Use the following commands to refresh movies/episodes with missing information:
Code:
c:\texturecache.py qax movies
c:\texturecache.py qax tvshows
It probably doesn't matter much where you add it in your batch, but before the caching ("c") commands probably makes the most sense so that any removed/rescraped media item can be re-cached if necessary.

To flag up movies and tvshows with missing clearart and clearlogo artwork (in addition to fanart, poster, and thumb), use the following:
Code:
c:\texturecache.py qax movies @qa.art.movies="+clearart,clearlogo"
c:\texturecache.py qax tvshows  @qa.art.tvshows.tvshow="+clearart,clearlogo"
If/when the missing artwork is found locally, the movie or episode will be automatically refreshed.

If you wanted to be warned about missing (blank) streamdetails or movie studios (in addition to plot and mpaa):
Code:
c:\texturecache.py qa movies @qa.blank.movies="+streamdetails,studio"

Example output:
Code:
./texturecache.py qa tvshows @qa.blank.tvshows.episode="+streamdetails" @qa.art.tvshows.tvshow="+clearlogo,?clearart"
TVShow   [Connections                                       ]: WARN (missing clearart)
TVShow   [Dalziel and Pascoe                                ]: WARN (missing clearart)
Episode  [Doctor Who (2005), Season 9 Episode 9x08          ]: FAIL (missing plot)
TVShow   [Dynamo: Magician Impossible                       ]: WARN (missing clearart)
TVShow   [Frank Herbert's Children of Dune                  ]: WARN (missing clearart)
TVShow   [Frank Herbert's Dune                              ]: WARN (missing clearart)
TVShow   [Great Expectations (2011)                         ]: WARN (missing clearart)
TVShow   [Halt and Catch Fire                               ]: WARN (missing clearart)
TVShow   [Hamish Macbeth                                    ]: WARN (missing clearart; missing clearlogo)
TVShow   [Outnumbered                                       ]: WARN (missing clearart)
Episode  [Saving Hope, Season 4 Episode 4x07                ]: WARN (URL thumb ^video, local not found)
Episode  [Saving Hope, Season 4 Episode 4x08                ]: FAIL (missing plot), WARN (missing thumb, local not found)
TVShow   [The Second Coming                                 ]: WARN (missing clearart)
TVShow   [Treasure Island (2012)                            ]: WARN (missing clearart)
"FAIL" means the item has failed QA and would have been be reloaded/refreshed had the qax command been used.

The "?" preceding clearart indicates that even if this artwork type is missing, it's not a fail, just a warning (and so wouldn't trigger a refresh when using qax).

After updating the NFO for Doctor Who, Season 9 Episode 8 with the missing plot details...
Code:
./texturecache.py qax tvshows "doctor who"
Episode  [Doctor Who (2005), Season 9 Episode 9x08          ]: FAIL (missing plot)
Rescanning directory: nfs://192.168.0.3/mnt/share/media/Video/TVShows/Doctor Who (2005)/Season 9...
Updating Library: New episodeid 13844 [Doctor Who (2005) S09E08 (The Zygon Inversion (2))]

Anyway, hope you get the idea.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
hi,
i've used successfully this script on windows, but can't make it works on an android box...
i've downloaded qpython form the market, copied the script and.cfg inside the qpython script folder, but can't launch it...
Many thanks for the help
all my backgrounds are black now...
Reply
Quote:
(2015-11-18, 14:33)Nepomuk Wrote: Where I can enable the separate options for extrafanart/extrathumbs/videoextras artwork in kodi?
You need to install a third-party skin that supports Extrafanart/Extrathumbs. VideoExtras is an addon, and probably only works if the skin supports it. I've never used any of these features/add-ons personally - I suggest you read the Kodi Wiki, look elsewhere on the forum where this sort of thing is discussed, and/or contact the skin/add-on authors for support.
I had never used this in the paste too but maybe in the future, so I had scraped this arts when every movie I had scraped.

Quote:
(2015-11-18, 14:33)Nepomuk Wrote: Are this your recommend commands AND in that order to hold Thumbnails-folder and local texture13.db clean and up to date:
With only this texturecache.cfg:
Code:
cache.ignore.types=+^http
cache.extrafanart = yes
cache.extrathumbs = yes
cache.videoextras = yes
cache.castthumb = yes
Code:
@echo off
texturecache.py P
texturecache.py R
texturecache.py Xd
texturecache.py c movies
texturecache.py c sets
texturecache.py c tvshows
texturecache.py imdb movies | c:\texturecache.py set   <-- maybe only who want this
texturecache.py imdb tvshows | c:\texturecache.py set  <-- maybe only who want this


I changed the order of the commands slightly - it would make more sense to run the "consistency check" (P, R, Xd) commands first, followed by the caching ("c") commands, then the imdb updates.

You may need to add the c:\ path to every call to texturecache.py, but this all depends on how you've installed it, and where.
I have customized my files now.
Here texturecache.cfg
Code:
########################################
#### GLOBAL USE ##############
#
thumbnails = \\192.168.2.210\srv-kodi/kodi_userdata\Thumbnails\
webserver.username = myusername
webserver.password = mypassword
# [qa] Perform QA check on media library items, identifying missing properties (eg. plot, mpaa certificate, artwork etc.).
# Default QA period is previous 30 days, configurable with qaperiod.
qaperiod = 9999
# Cast thumbnails will not be cached by default, so specify cache.castthumb = yes if you require cast artwork to be re-cached, or considered when pruning.
cache.castthumb = yes
# Ignore specific URLs when pre-loading the cache (c/C/nc options), by specifying comma delimited regex patterns for the cache.ignore.types property. Default values are ^video and ^music (not that these patterns are applied after the image:// prefix has been removed from the url). Set to none (no argument) to process all URLs. Any URL that matches one of the ignore types will not be considered for re-caching (and will be counted as "ignored").
# kein WAN download, nur local
cache.ignore.types = +^http
# Extrafanart will not be cached by default or considered when pruning. Specify cache.extra = yes to cache/prune both extrafanart and extrathumbs, or cache.extrafanart = yes or cache.extrathumbs = yes to enable just extrafanart or just extrathumbs.
cache.extrafanart = yes
cache.extrathumbs = yes
cache.videoextras = yes

########################################

section = master

########################################
#### CLIENTS ##############
# e.g.
# then "texturecache.py @section=bedroom c" will re-cache all artwork for client bedroom
#
[master]
kodi.host = localhost
userdata = C:\Users\nepomuk\AppData\Roaming\Kodi\userdata
download.threads = 10
# Specify a filename for the logfile property, to log detailed processing information. Prefix the filename with + to force flushing.
lastrunfile=C:\Users\nepomuk\Eigene Programme\Kodi_texturecache\master_lrf.dat
# Specify a filename for the logfile property, to log detailed processing information. Prefix the filename with + to force flushing.
logfile = C:\Users\nepomuk\Eigene Programme\Kodi_texturecache\master_log.txt
  
[livingroom]
kodi.host = 192.168.2.80
userdata = C:\Users\xbmc\AppData\Roaming\Kodi\userdata
download.threads = 10
lastrunfile=C:\Users\nepomuk\Eigene Programme\Kodi_texturecache\livingroom_lrf.dat
logfile = C:\Users\nepomuk\Eigene Programme\Kodi_texturecache\livingroom_log.txt

[kid1]
kodi.host = 192.168.1.71
userdata = C:\Users\kid1\AppData\Roaming\Kodi\userdata
download.threads = 6
lastrunfile=C:\Users\nepomuk\Eigene Programme\Kodi_texturecache\kid1_lrf.dat
logfile = C:\Users\nepomuk\Eigene Programme\Kodi_texturecache\kid1_log.txt
  
[kid2]
kodi.host = 192.168.1.72
userdata = C:\Users\kid2\AppData\Roaming\Kodi\userdata
download.threads = 6
lastrunfile=C:\Users\nepomuk\Eigene Programme\Kodi_texturecache\kid2_lrf.dat
logfile = C:\Users\nepomuk\Eigene Programme\Kodi_texturecache\kid2_log.txt
########################################

I found "allow.recacheall = " in this thread here but sorry I can't remember for what it is. Can you explane it for me again please?

And here the batch-file for master:
Code:
@echo off
texturecache.py P
texturecache.py R
texturecache.py Xd
texturecache.py c movies
texturecache.py c sets
texturecache.py c tvshows
and here for livingroom:
Code:
@echo off
texturecache.py @section=livingroom P
texturecache.py @section=livingroom R
texturecache.py @section=livingroom c movies
texturecache.py @section=livingroom c sets
texturecache.py @section=livingroom c tvshows

Quote:I'm assuming you're running the script on the Kodi client, as "texturecache.py Xd" will only work if it has direct access to the Thumbnails folder - all the other commands you've listed should work remotely across the network, if you add "kodi.host=<ip_address>" where <ip_address> is the ip address of the Kodi client you wish to update.
That's a little problem for me because the Thumbnails folder is on a separate server with is Solarix running and nothing else.
I have to run "texturecache.py Xd" on my PC in my home office. This PC have a data rights and connected by a 1G Lan network. Is that nevertheless OK?

Quote:
(2015-11-18, 14:33)Nepomuk Wrote: What is with the "qa" or "qax" command - I don't use it or it's better to insert in the bath file and when so in which order?

The qa/qax options will run quality checks on your media library, looking for movies with missing artwork, missing plots etc. - qa will report problems, qax will report problems and, depending on your configuration, remove and reload movies/episodes. For instance with qax, movies/episodes that are missing metadata can be refreshed (remove/re-scan) in the hope the missing metadata will be loaded (either by scraping new information from the internet, or by loading an updated nfo). Movies or episodes that are missing artwork can be reloaded once the artwork is found locally.

I use qax on my tvshows to flag up any episodes with missing plots, and to reload new episodes that are missing thumbs - quite often the thumb won't be downloaded until a day or so after the episode, and a daily qax will automatically refresh the episode once the thumb is available.

The default qa/qax settings will identify empty/missing plots, missing standard (poster/fanart/thumb) artwork. One thing to be aware of is that by default it only checks movies/episodes added in the previous 30 days, so you might want to set qaperiod=9999 in order to check your entire library.

Use the following commands to refresh movies/episodes with missing information:
Code:
c:\texturecache.py qax movies
c:\texturecache.py qax tvshows
It probably doesn't matter much where you add it in your batch, but before the caching ("c") commands probably makes the most sense so that any removed/rescraped media item can be re-cached if necessary.

To flag up movies and tvshows with missing clearart and clearlogo artwork (in addition to fanart, poster, and thumb), use the following:
Code:
c:\texturecache.py qax movies @qa.art.movies="+clearart,clearlogo"
c:\texturecache.py qax tvshows  @qa.art.tvshows.tvshow="+clearart,clearlogo"
If/when the missing artwork is found locally, the movie or episode will be automatically refreshed.

If you wanted to be warned about missing (blank) streamdetails or movie studios (in addition to plot and mpaa):
Code:
c:\texturecache.py qa movies @qa.blank.movies="+streamdetails,studio"

Example output:
Code:
./texturecache.py qa tvshows @qa.blank.tvshows.episode="+streamdetails" @qa.art.tvshows.tvshow="+clearlogo,?clearart"
TVShow   [Connections                                       ]: WARN (missing clearart)
TVShow   [Dalziel and Pascoe                                ]: WARN (missing clearart)
Episode  [Doctor Who (2005), Season 9 Episode 9x08          ]: FAIL (missing plot)
TVShow   [Dynamo: Magician Impossible                       ]: WARN (missing clearart)
TVShow   [Frank Herbert's Children of Dune                  ]: WARN (missing clearart)
TVShow   [Frank Herbert's Dune                              ]: WARN (missing clearart)
TVShow   [Great Expectations (2011)                         ]: WARN (missing clearart)
TVShow   [Halt and Catch Fire                               ]: WARN (missing clearart)
TVShow   [Hamish Macbeth                                    ]: WARN (missing clearart; missing clearlogo)
TVShow   [Outnumbered                                       ]: WARN (missing clearart)
Episode  [Saving Hope, Season 4 Episode 4x07                ]: WARN (URL thumb ^video, local not found)
Episode  [Saving Hope, Season 4 Episode 4x08                ]: FAIL (missing plot), WARN (missing thumb, local not found)
TVShow   [The Second Coming                                 ]: WARN (missing clearart)
TVShow   [Treasure Island (2012)                            ]: WARN (missing clearart)
"FAIL" means the item has failed QA and would have been be reloaded/refreshed had the qax command been used.

The "?" preceding clearart indicates that even if this artwork type is missing, it's not a fail, just a warning (and so wouldn't trigger a refresh when using qax).

After updating the NFO for Doctor Who, Season 9 Episode 8 with the missing plot details...
Code:
./texturecache.py qax tvshows "doctor who"
Episode  [Doctor Who (2005), Season 9 Episode 9x08          ]: FAIL (missing plot)
Rescanning directory: nfs://192.168.0.3/mnt/share/media/Video/TVShows/Doctor Who (2005)/Season 9...
Updating Library: New episodeid 13844 [Doctor Who (2005) S09E08 (The Zygon Inversion (2))]

Anyway, hope you get the idea.
Hope I understand QA right. This is a tool where I can verify the kodi media library off missing informations and arts IN the kodi media library. And when missing this things I can add it by using "qax". And the best it's when I use this commands seperately to my main-batch-file or can I integrate this two commands at then end of the bat-file?
Thats looks great but I think I will use that in a second step in the future because I have to be more handsome with your script first by holding my datas consistent.
Reply
So I made my first run. Here is the result:
texturecache.py P I have forgotten to log.
texturecache.py Xd had no log-output

texturecache.py R
Code:
Orphaned file found: name [0\049bc003.png], created [Sun Aug 23 20:00:57 2015], size [12,082], REMOVING...

Summary: 1,360 files; Total size: 45,603 KB
texturecache.py c movies
Code:
Cache pre-load activity summary for "movies":

              |  cast.thumb | extrafanart | extrathumbs |    fanart   |    poster   |    thumb    |    TOTAL    
--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------
Cached        |     1115    |     3412    |      16     |      1      |      1      |      -      |     4545    
Deleted       |      -      |      -      |      -      |      -      |      -      |      -      |      0      
Duplicate     |      -      |      -      |      -      |      -      |      -      |      -      |      0      
Error         |      -      |      -      |      -      |      -      |      -      |      -      |      0      
Ignored       |     104     |      -      |      -      |      -      |      -      |      2      |     106    
Skipped       |    18564    |      -      |     4056    |     1042    |     1044    |      -      |    24706    
Undefined     |      -      |      -      |      -      |      -      |      -      |      -      |      0      
================================================================================================================
TOTAL         |    19783    |     3412    |     4072    |     1043    |     1045    |      2      |    29357    
Download Time | 00:00:24.45 | 00:02:08.27 | 00:00:00.92 | 00:00:00.46 | 00:00:00.24 |      -      | 00:02:33.37

  Threads Used: 10
   Min/Avg/Max: 16.93 / 29.67 / 58.58 downloads per second
   Min/Avg/Max: 00.05 / 00.34 / 01.45 seconds per download

       Loading: 00:00:06.35
       Parsing: 00:00:10.54
     Comparing: 00:00:03.03
   Downloading: 00:02:33.37
TOTAL RUNTIME: 00:02:53.54

texturecache.py c sets
Code:
Cache pre-load activity summary for "sets":

              |    banner   |   clearart  |  clearlogo  |    fanart   |    poster   |    TOTAL    
--------------+-------------+-------------+-------------+-------------+-------------+-------------
Cached        |      50     |      29     |      52     |      34     |      -      |     165    
Deleted       |      -      |      -      |      -      |      -      |      -      |      0      
Duplicate     |      -      |      -      |      -      |      -      |      -      |      0      
Error         |      -      |      -      |      -      |      -      |      -      |      0      
Ignored       |      -      |      -      |      -      |      -      |      -      |      0      
Skipped       |      -      |      -      |      -      |      28     |      62     |      90    
Undefined     |      -      |      -      |      -      |      -      |      -      |      0      
==================================================================================================
TOTAL         |      50     |      29     |      52     |      62     |      62     |     255    
Download Time | 00:00:01.20 | 00:00:02.42 | 00:00:02.54 | 00:00:01.49 |      -      | 00:00:06.22

  Threads Used: 10
   Min/Avg/Max: 12.00 / 26.54 / 35.78 downloads per second
   Min/Avg/Max: 00.07 / 00.36 / 00.98 seconds per download

       Loading: 00:00:00.08
       Parsing: 00:00:00.01
     Comparing: 00:00:02.80
   Downloading: 00:00:06.22
TOTAL RUNTIME: 00:00:09.14

texturecache.py c tvshows
Code:
Cache pre-load activity summary for "tvshows":

              |    banner   |  cast.thumb | extrafanart |    fanart   |    poster   |    thumb    |    TOTAL    
--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------
Cached        |      15     |      -      |      6      |      39     |      52     |      -      |     112    
Deleted       |      -      |      -      |      -      |      -      |      -      |      -      |      0      
Duplicate     |     311     |      -      |      -      |     714     |     724     |      -      |     1749    
Error         |      -      |      -      |      -      |      -      |      -      |      -      |      0      
Ignored       |      -      |     154     |      -      |      -      |      -      |      -      |     154    
Season-all    |      4      |      -      |      -      |      4      |      12     |      -      |      20    
Skipped       |      3      |      17     |      -      |      12     |      13     |     717     |     762    
Undefined     |      -      |      -      |      -      |      -      |      -      |      -      |      0      
================================================================================================================
TOTAL         |     333     |     171     |      6      |     769     |     801     |     717     |     2797    
Download Time | 00:00:00.39 |      -      | 00:00:00.35 | 00:00:01.39 | 00:00:01.20 |      -      | 00:00:02.70

  Threads Used: 10
   Min/Avg/Max: 33.00 / 41.52 / 38.21 downloads per second
   Min/Avg/Max: 00.05 / 00.23 / 00.55 seconds per download

       Loading: 00:00:03.23
       Parsing: 00:00:00.15
     Comparing: 00:00:02.80
   Downloading: 00:00:02.70
TOTAL RUNTIME: 00:00:08.89

Makes sense to bump "download.threads" from 10 to maybe 20?
I don't understand why so much movie items are ignored or skipped. Huh
Why are the duplicate arts in tv-show not deleted automatically- because I see it at the second run that this 1749 files are available.
Reply
(2015-11-18, 16:52)onlyyou Wrote: hi,
i've used successfully this script on windows, but can't make it works on an android box...
i've downloaded qpython form the market, copied the script and.cfg inside the qpython script folder, but can't launch it...
Many thanks for the help
all my backgrounds are black now...

Sorry no idea about Android.

You can always run the script on the Windows PC and provide the IP address of your Android device (@kodi.host), that will work for a lot of the options.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2015-11-18, 18:13)Nepomuk Wrote: I found "allow.recacheall = " in this thread here but sorry I can't remember for what it is. Can you explane it for me again please?

It allows you to forcibly recache all items for a particular media type (ie. "C movies") which isn't something you would normally want to do, or might only do by mistake, but if you know what you're doing then you can enable this option.

Reading the source code will probably answer a lot of questions.

(2015-11-18, 18:13)Nepomuk Wrote: I have to run "texturecache.py Xd" on my PC in my home office. This PC have a data rights and connected by a 1G Lan network. Is that nevertheless OK?

No idea what the question is, sorry.

(2015-11-18, 21:04)Nepomuk Wrote: Makes sense to bump "download.threads" from 10 to maybe 20?

Not really, 10 is probably sufficient.

(2015-11-18, 21:04)Nepomuk Wrote: I don't understand why so much movie items are ignored or skipped. Huh
Why are the duplicate arts in tv-show not deleted automatically- because I see it at the second run that this 1749 files are available.

There's a link at the bottom of the first post. Or read the source code.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
I followed the instructions to install both the python, the scripts, edit the texturecache.py script and executed the first command, and i get these errors, any idea what i am doing wrong?

Quote:C:\Users\Monu>C:\Users\Monu\AppData\Local\Programs\Python\Python35\python.exe c:\texturecache.py
Traceback (most recent call last):
File "c:\texturecache.py", line 8413, in <module>
main(sys.argv[1:])
File "c:\texturecache.py", line 8117, in main
loadConfig(argv)
File "c:\texturecache.py", line 7626, in loadConfig
gConfig = MyConfiguration(argv)
File "c:\texturecache.py", line 146, in __init__
config.readfp(cfg)
File "C:\Users\Monu\AppData\Local\Programs\Python\Python35\lib\configparser.py", line 761, in readfp
self.read_file(fp, source=filename)
File "C:\Users\Monu\AppData\Local\Programs\Python\Python35\lib\configparser.py", line 716, in read_file
self._read(f, source)
File "C:\Users\Monu\AppData\Local\Programs\Python\Python35\lib\configparser.py", line 1108, in _read
raise e
configparser.ParsingError: Source contains parsing errors: '<Huh>'
[line 5]: '<!DOCTYPE html>\n'
[line 115]: '</form>\n'
[line 134]: '</div>\n'
[line 141]: '</div>\n'
[line 194]: '</ul>\n'
[line 206]: '</h1>\n'
[line 225]: '</a> </li>\n'
[line 232]: '</a> </li>\n'
[line 239]: '</a> </li>\n'
[line 249]: '</a> </li>\n'
[line 255]: '</a> </li>\n'
[line 259]: '</nav>\n'
[line 273]: '</div>\n'
[line 286]: '</div>\n'
[line 295]: '</div>\n'
[line 1011]: '</div>\n'
[line 1101]: '</div>\n'
[line 1162]: '</td>\n'
[line 1248]: '</table>\n'
[line 1252]: '</div>\n'
[line 1259]: '</form></div>\n'
[line 1285]: '</a>\n'
[line 1295]: '</div>\n'
[line 1322]: '</html>\n'
Reply
Your texturecache.cfg configuration file appears to be a web page...
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2015-11-19, 03:15)Milhouse Wrote: Your texturecache.cfg configuration file appears to be a web page...

THanks for helping out with this.
It looks like its working now. the first command returned results with options i can choose to make this work.

Can i ask my first question.. Smile
Just today i have rescraped all my metadata, nfo, that contains tags, actor, director, release year, studio, infact artwork also.. so its like building from the scratch.. i have rescraped about 1000+ movies using Media companion.
Now, i need to replace this new data into the database, so that all the new info is reflected in Kodi.
What options should i use. I realize i can also, delete textures13.db file and then run update, i believe here using Texture cache utility is the right way to go.
What options should i use in...

Please Note. I dont need texture cache to download anything, neither artwork neither metadata info for the nfo files. I already have all the nfo data in each of the movie folders and artwork as well. All i am wanting to do is, texture cache to delete the old db, including thumbnails and rebuild all the databases, thumbnails from the newly scraped data for which i used Media Companion.

Code:
C:\Users\Monu\AppData\Local\Programs\Python\Python35\python.exe c:\texturecache.py
Reply
by the way out of curiosity i executed the P command, and it ran for about 1 hr, did a lot of stuff, the cmd prompt was full of action. Smile

after this i ran C and it errored out.

not sure if these are the commands i need to run for what i wish to achieve (i mentioned in the most above). Please help.

Code:
C:\>C:\Users\Monu\AppData\Local\Programs\Python\Python35\python.exe c:\texturecache.py C
Traceback (most recent call last):
  File "c:\texturecache.py", line 8413, in <module>
    main(sys.argv[1:])
  File "c:\texturecache.py", line 8121, in main
    if not checkConfig(argv[0]): sys.exit(2)
  File "c:\texturecache.py", line 7703, in checkConfig
    data = wcomms.sendJSON(REQUEST, "libPing", checkResult=False, useWebServer=True)
  File "c:\texturecache.py", line 2179, in sendJSON
    return self.sendWeb("POST", "/jsonrpc", id, request, {"Content-Type": "application/json"}, timeout=timeout)
  File "c:\texturecache.py", line 2166, in sendWeb
    return json.loads(data) if data != "" else ""
  File "C:\Users\Monu\AppData\Local\Programs\Python\Python35\lib\json\__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "C:\Users\Monu\AppData\Local\Programs\Python\Python35\lib\json\decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Users\Monu\AppData\Local\Programs\Python\Python35\lib\json\decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Please review my previous post .
Reply
(2015-11-19, 05:32)MANswers Wrote: All i am wanting to do is, texture cache to delete the old db, including thumbnails and rebuild all the databases, thumbnails from the newly scraped data for which i used Media Companion.

If your plan is to rescan your entire library from scratch then shut down Kodi, delete the video database (MyVideos*.db, or MySQL equivalent) and texture cache (Textures13.db and Thumbnails folder) using File Explorer, then restart Kodi so that you can populate the now empty databases by performing a library scan - there's no need to involve texturecache.py.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2015-11-19, 08:26)MANswers Wrote: after this i ran C and it errored out.

Looks like the Kodi webserver has died/crashed - check your kodi.log for errors.

Do you get this error every time when running "c:\texturecache.py C" ? What about if you restart the Kodi client and then run "c:\texturecache.py C"?

If you are still having this problem after restarting the Kodi client, run the following:

Code:
C:\>C:\Users\Monu\AppData\Local\Programs\Python\Python35\python.exe c:\texturecache.py C @logfile=c:\tc.log

then upload c:\tc.log to a pastebin site and paste the link.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2015-11-19, 09:03)Milhouse Wrote:
(2015-11-19, 05:32)MANswers Wrote: All i am wanting to do is, texture cache to delete the old db, including thumbnails and rebuild all the databases, thumbnails from the newly scraped data for which i used Media Companion.

If your plan is to rescan your entire library from scratch then shut down Kodi, delete the video database (MyVideos*.db, or MySQL equivalent) and texture cache (Textures13.db and Thumbnails folder) using File Explorer, then restart Kodi so that you can populate the now empty databases by performing a library scan - there's no need to involve texturecache.py.

also, i forgot to mention earlier, when i go to http://localhost:8080/ amazon echo's ha bridge webpage opens up. I have configured amazon echo HA bridge on this port. is it cos of this?

i have changed the port within kodi, to 8081 and now localhost:8081 perfectly displays kodi webpage.
no errors in the kodi log as well.
i will test out the rest of the part tomorrow and upload the log to pastebin tomorrow morning. its 1.30 am in chicago. Sad the wife's staring at me while she stepped out to get water. Big Grin
Reply
  • 1
  • 115
  • 116
  • 117(current)
  • 118
  • 119
  • 197

Logout Mark Read Team Forum Stats Members Help
[RELEASE] Texture Cache Maintenance utility17