• 1
  • 67
  • 68
  • 69(current)
  • 70
  • 71
  • 197
[RELEASE] Texture Cache Maintenance utility
(2014-07-22, 01:33)theowiesengrund Wrote: just a quick question. I was browsing my library and looking for "obscure/rare" movies, therefore i would love to sort/list my movies according to the imdb vote numbers. Is this somehow possibile with your script, for example with a json query?

If you want a list of your movies sorted by imdb votes, the following should work (in *nix):
Code:
./texturecache.py query movies 'votes != ""' | sed 's/"//g' | sort -t= -k2n
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2014-07-22, 11:17)gargamon Wrote: I was thinking about this again. It seems to me that when you're doing the "C" option, you really want to overwrite whatever is there, whether it's nothing, a corrupted file, or who knows what. Wouldn't that imply that you no longer trust the database and want to rewrite that row anyway? I know it would be slower than now, but the added functionality of being able to do "C tvshows some_show_name" is much more versatile than doing "Xd"on a particular row when you may want to do 20 or more rows. Maybe a new command could handle this.

"C" will remove the row if it's there, whether it has an associated thumbnail file or not. Removing the row will also remove the thumbnail file (if it exists). If the row doesn't exist (but the thumbnail file does) then the "C" option will simply overwrite the thumbnail file when creating the new database row (as, indeed, would "c") So either way, the current "C" functionality should correct rows with missing thumbnail files, or thumbnail files that are no longer referenced by database rows. No extra functionality required.

"Xd" will by default scan the entire Textures13.db database, rather than individual rows, and will remove those rows with missing files. Same for "R".

I don't really know why "Xd" worked for you when "C" didn't, my guess would be that you didn't reboot (restart XBMC) which may have resulted in some lingering cache issues (OS file system cache, etc.).
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2014-07-22, 11:30)Milhouse Wrote:
(2014-07-22, 01:33)theowiesengrund Wrote: just a quick question. I was browsing my library and looking for "obscure/rare" movies, therefore i would love to sort/list my movies according to the imdb vote numbers. Is this somehow possibile with your script, for example with a json query?

If you want a list of your movies sorted by imdb votes, the following should work (in *nix):
Code:
./texturecache.py query movies 'votes != ""' | sed 's/"//g' | sort -t= -k2n

Awesome! Thank you.
Reply
can picons be pre-cached?
Reply
(2014-07-22, 13:09)Milhouse Wrote:
(2014-07-22, 11:17)gargamon Wrote: I was thinking about this again. It seems to me that when you're doing the "C" option, you really want to overwrite whatever is there, whether it's nothing, a corrupted file, or who knows what. Wouldn't that imply that you no longer trust the database and want to rewrite that row anyway? I know it would be slower than now, but the added functionality of being able to do "C tvshows some_show_name" is much more versatile than doing "Xd"on a particular row when you may want to do 20 or more rows. Maybe a new command could handle this.

"C" will remove the row if it's there, whether it has an associated thumbnail file or not. Removing the row will also remove the thumbnail file (if it exists). If the row doesn't exist (but the thumbnail file does) then the "C" option will simply overwrite the thumbnail file when creating the new database row (as, indeed, would "c") So either way, the current "C" functionality should correct rows with missing thumbnail files, or thumbnail files that are no longer referenced by database rows. No extra functionality required.

I just did a quick test by manually deleting the TMZ icon file in the cache. I left the db alone. Neither the "c" or "C" command caused the database row to be rewritten. The "c" command showed a skipped in the summary. The "C" command showed an error in the summary. I believe I provided debug logging of this earlier, but it's very easy to duplicate. I rebooted and tried the commands again with no change.

I think there's a small bug here. Let me know if I can add anything further.

And I still think your tool is indispensable in maintaining your XBMC. Thanks for all the hard work.
Reply
(2014-07-23, 19:56)stuCONNERS Wrote: can picons be pre-cached?

What are picons?

Edit: Turns out these are TV channel logos. Whether they can be cached depends on a few things, mainly: 1) Does XBMC support picons? 2) Is the location of the original picon artwork "discoverable", ie. available via the "PVR.GetChannels" method?

Try: "./texturecache.py jd pvr.tv" (or pvr.radio) and see what artwork is available.

Maybe some additional background on picons.

(2014-07-24, 04:12)gargamon Wrote: The "C" command showed an error in the summary. I believe I provided debug logging of this earlier, but it's very easy to duplicate. I rebooted and tried the commands again with no change.

I think there's a small bug here. Let me know if I can add anything further.

Ah yes.

When downloading (caching) artwork, it's a 2 step process - obtain the "real" url of the artwork to be cached by passing the current url to the "Files.PrepareDownload" JSON method, then actually caching the artwork with an http request to the XBMC webserver using this real url.

When using the "C" method, the database row is deleted only once the real url is obtained from "Files.PrepareDownload", so if this call fails - which presumably is the case for TMZ - then the database row is not removed as it won't be possible to (re-)cache this artwork, so it's better to leave the user with what they already have. The row would however be removed when using "Xd" as this option doesn't care about "Files.PrepareDownload".

The reason the "Files.PrepareDownload" call fails is because the original artwork is no longer available or accessible. Removing the database row before the call to "Files.PrepareDownload" is not an automatic option (however, see later) as the already cached thumbnail may be perfectly fine - you might have been "freshening" other artwork for the same movie/tvshow/addon, and you'd be left with less artwork than you had before.

In the case of TMZ, what you are experiencing is a pretty rare corner case (remote artwork unavailable, database row is present but the thumbnail missing), which would be fixed by running "Xd".

However, there is one other option available to you - pre-deletion.

Enable with "@Download.predelete=yes" (default value is "auto") and this will delete all of the artwork to be re-cached before any attempt is made to download, rather than deleting artwork during each download thread (and then only if the artwork is accessible). Enabling this option will of course delete cached artwork that cannot then be re-cached.

The pre-delete option exists to prevent a remote system with direct SQLite access - eg. a PC with Databases and Thumbnails mounted belonging to a remote XBMC client, eg. a Raspberry Pi - from making multiple overlapping db deletions (inside each download thread) which could potentially lead to SQLite corruption.

Pre-deletion is now unnecessary following the addition of the JSON Textures API in Gotham which avoids any potential SQLite corruption, but is something that would still be used by older clients that don't support the Textures API. It may also be useful in your case even though your client doesn't technically require it.

So either run "Xd" from time to time as part of a standard maintenance routine, or try enabling pre-deletion (though this has a downside, as explained above). Generally speaking, you shouldn't need to use either option on a regular basis - if you do, that would suggest a problem elsewhere.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
I just wanted to say
thank you thank you thank you thank you thank you thank you

your script is a life/time saver
I had to do a full reinstall with about 300 movies and 1500 TV show eps, I had backed up everything except the horrible thumbnails folder... with your script everything was back up within minutes.

thank you thank you Smile
Reply
@Milhouse after putting that command into putty i got the following outout

{
"channel": "BBC Four HD",
"channelid": 92,
"channeltype": "tv",
"hidden": false,
"lastplayed": "1970-01-01",
"locked": false,
"thumbnail": "image:///storage/.xbmc/picons/BBC Four HD.png/"
},
Reply
You may be able to cache that, try with "texturecache.py c pvr.tv bbc\ four\ hd".

One reason it might fail is that XBMC will only allow access to directories defined within sources.xml - I'm not entirely sure which JSON methods are restricted, so you'll need to test it and see. Are you seeing these picons in the GUI without caching with texturecache.py? If you're having problems, check your xbmc.log first.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
yeah they appear fine, i was wondering if itll help the epg loader a little faster

| thumbnail | TOTAL
--------------+-------------+-------------
Cached | 80 | 80
Deleted | - | 0
Duplicate | 342 | 342
Error | - | 0
Ignored | - | 0
Skipped | 248 | 248
Undefined | 414 | 414
==========================================
TOTAL | 1084 | 1084
Download Time | 00:00:19.48 | 00:00:19.48

Threads Used: 2
Min/Avg/Max: 1.25 / 3.02 / 4.50

Loading: 00:00:05.99
Parsing: 00:00:02.50
Comparing: 00:00:27.24
Downloading: 00:00:20.07
TOTAL RUNTIME: 00:00:56.26
Reply
(2014-07-25, 00:00)stuCONNERS Wrote: i was wondering if itll help the epg loader a little faster

It should help avoid any UI delays due to image caching. Whether this has a noticeable effect depends on how many images the EPG is loading. EPG delays are more than likely not image related.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
it does help. My epg has 600+ picons, When precached the epg loads smoother.
Reply
Hello.

I have been doing some (a little) reading on texturecache.py after having some problems with scraping/updating my library.

Being relatively new to xbmc experience I presume that this script is meant for fixing scraping errors. Right?

How about preventing scraping errors? Or maybe scraping errors are a given?

Regards.

PS
I have been facing the following:
CCurlFile::FillBuffer - Failed: HTTP response code said error(22)
CCurlFile::CReadState::Connect, didn't get any data from stream.
ADDON::CScraper::Run: Unable to parse web site
Reply
(2014-08-01, 14:59)nistsio Wrote: Hello.

I have been doing some (a little) reading on texturecache.py after having some problems with scraping/updating my library.

Being relatively new to xbmc experience I presume that this script is meant for fixing scraping errors. Right?

How about preventing scraping errors? Or maybe scraping errors are a given?

Regards.

PS
I have been facing the following:
CCurlFile::FillBuffer - Failed: HTTP response code said error(22)
CCurlFile::CReadState::Connect, didn't get any data from stream.
ADDON::CScraper::Run: Unable to parse web site

The script can help you to diagnose scraper issues by allowing you to view the collected meta data (jd option), but basic communication problems (such as yours) it can't help with - either your internet connection isn't working, or the remote web site is not responding (offline). Turn on debugging to see more detail, but it's nothing this script can help with.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Thanks for the quick reply.

These errors I mentioned are actually taken from a debug log.

Anyway; keep up the good work.

Cheers!
Reply
  • 1
  • 67
  • 68
  • 69(current)
  • 70
  • 71
  • 197

Logout Mark Read Team Forum Stats Members Help
[RELEASE] Texture Cache Maintenance utility17