• 1
  • 4
  • 5
  • 6(current)
  • 7
  • 8
  • 197
[RELEASE] Texture Cache Maintenance utility
#76
(2013-03-29, 22:07)charrua Wrote: Just two small cosmetic suggestions:
-show the processing time in the statistics
-put the TOTAL column as first one instead of last
(other than that, I believe that the script has reached the first stage of maturity with v0.3.1 Wink)
Congrats on a job well done, my friend!

I'll see about adding some timing stats... not sure about moving the Total column though - on the right is normal, isn't it?

(2013-03-29, 22:07)charrua Wrote: The next step (just in case you want it to gain popularity) is transforming the script to an ADD-ON (as previously suggested) Laugh

Anyone willing to take up that challenge is more than welcome to fork my repository! Smile

Though what would be gained from running it as an addon. I'd say more would be lost - it would be slower (add-on are always slow to start, stop etc.), less useful (you couldn't pipe, grep or otherwise mangle the output for more complicated processing), and scheduling would be more difficult (maybe scheduling is possible for addons - not really sure).

Maybe the script could serve as the basis for an addon, which just calls out to texturecache.py, crunches the results and presents them to the user - an add-on wrapper.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#77
(2013-03-29, 22:51)MilhouseVH Wrote:
(2013-03-29, 22:07)charrua Wrote: Just two small cosmetic suggestions:
-show the processing time in the statistics
-put the TOTAL column as first one instead of last
(other than that, I believe that the script has reached the first stage of maturity with v0.3.1 Wink)
Congrats on a job well done, my friend!
I'll see about adding some timing stats... not sure about moving the Total column though - on the right is normal, isn't it?
Having TOTALS next to the descriptions column will be easier to read (at least for me... Smile)

(2013-03-29, 22:51)MilhouseVH Wrote:
(2013-03-29, 22:07)charrua Wrote: The next step (just in case you want it to gain popularity) is transforming the script to an ADD-ON (as previously suggested) Laugh
Anyone willing to take up that challenge is more than welcome to fork my repository! Smile
Good point!

(2013-03-29, 22:51)MilhouseVH Wrote: Though what would be gained from running it as an addon. I'd say more would be lost - it would be slower (add-on are always slow to start, stop etc.), less useful (you couldn't pipe, grep or otherwise mangle the output for more complicated processing), and scheduling would be more difficult (maybe scheduling is possible for addons - not really sure).
Yes, all advanced features will probably be lost (or hidden behind a good advanced configuration screen), but it will also be a much more easier and accesible way to use the basic and very useful functionalities of your great script (specially for beginers).

(2013-03-29, 22:51)MilhouseVH Wrote: Maybe the script could serve as the basis for an addon, which just calls out to texturecache.py, crunches the results and presents them to the user - an add-on wrapper.
This can be a good first approach.
Reply
#78
(2013-03-29, 21:13)MilhouseVH Wrote: It should certainly be deleting both the database row and the corresponding cached file - you can confirm this by enabling the logfile option (logfile = filename) and then looking to see if any SQL deletions took place. Though I'm not sure how sqlite3 manages its database size as rows are removed - maybe it never shrinks the database (potentially expensive and counter productive) and holds on to the now free space to be re-used in the future? You can get a row count by running the x option and piping the results through "wc -l", or use the "f" option - there's a rowcount summary at the end, along with the amount of disk space used by the corresponding files.

Sure will try next time.

(2013-03-29, 21:13)MilhouseVH Wrote: Again thanks. Though not sure what other switches to add right now, I'm fresh out of ideas... currently just tweaking the code and hoping to get to the bottom of this Pi killer issue. If you've got any suggestions for the future, fire away!

Anything that can benefit from a multithread could really really be added this, idk the script or scripting or any coding to make educated guesses, but If thers any Im sure you will know what to do. Your doing pretty well so far!.

(2013-03-29, 21:13)MilhouseVH Wrote: Not sure how long it will last - I only started it as a way to learn a new language, and to more easily extract rows from sqlite3 database - and now look at it... Smile

Yea, things tend to get out of hand but as long as its helping and doing some job, its always a good idea. I started posting to help others and now am doing the odd xbmc wiki imrpovements/additions and am trying to get XWMM a push to get more developers interested and the ones that already worked on it to continue improving it, but Im not a coder, and Any projects like that need developers so they dont struggle, and its struggling.

I'de love to have someone half as skilled as you join these side projects which help many users, though keeping them going is not so easy. Perhaps now you have your stuff on github I can mess with a couple of ideas I had for you and maybe others will join in.

btw if you ever feel like you can help XWMM come by and see whats its like so far @ https://github.com/slash2009/XWMM or http://forum.xbmc.org/showthread.php?tid=60643

Will go mess your gitcode in a jiffy and make a PR and do something else documentation related.

uNi

[/quote]
Reply
#79
OK done some damage in your repo tryng to improve a couple of things. Thansk for merging. Wink

Now all those radials can be called from script if needed and all is easier and not so clutered for you deving and maybe nickr and his promise to help you.

We could use someone like you @XWMM if you ever feel that way inclined please send some json and js our way.

uNi
Reply
#80
Thanks for your help on github. Still finding my feet around a lot of this stuff but will have a nose around XVMM and see what's going on ... Smile
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#81
Thx for the tip, MinhouseVH, works like a charm!
-> enable webserver... maybe you add this to your faqs... Wink
Perfekt!
Reply
#82
@millhouse

If this addon is updating improving you can consider adding a version check/pull/ask/display changelog via script too when script is first run.

uNi
Reply
#83
(2013-03-30, 18:24)uNiversal Wrote: @millhouse

If this addon is updating improving you can consider adding a version check/pull/ask/display changelog via script too when script is first run.

uNi

Done.

v0.3.4 is up.

Script will now perform an automatic version check (against Github) with every execution, informing the user when it is out of date. Disable version check by adding "checkupdate = no" to properties. Users can "force" a version check with the "version" option, and update to the latest version by running the "update" option - this will download the new script from Github and replace the current script file.

The update process will use an md5 hash to validate against corrupt downloads, unfortunately that's just increased the workload and there's a good chance I'll forget to update the hash... Smile

I'll leave displaying the changelog for now, if the user is interested (most aren't) it's on Github. Besides, I've changed it to use markdown now and it's not so easy to read on a plain console! I'll show the URL for the Changelog as part of the "version" option. Smile

Next step is to add some times to the stats...
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#84
(2013-03-30, 18:20)hydra75 Wrote: Thx for the tip, MinhouseVH, works like a charm!
-> enable webserver... maybe you add this to your faqs... Wink
Perfekt!

I've added a quick mention in the third para of the first post, as I reckon most people will fail to make it all the way to the end so I'd better get it in early. Smile
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#85
v0.3.5 on Github - hopefully the version check/update process works...

* Added stats timings to the c/C/nc summary.

Interesting exercise trying to calculate elapsed time across multiple threads. The download times in the table of results will be shown per artwork type (ie. fanart, poster etc.), however the total download time may (and often will) be less than the sum of the individual times as downloads will overlap when using multiple threads. The total download time should equal the sum of individual download times only when using a single thread, or when downloading only a single artwork type.

The "Download" time at the very end of the summary will normally exceed the sum of the individual times, as it will include thread management overhead (startup, shutdown etc.)

* Added additional JSON artwork queries for Genre and Addon when pruning the texture cache.

* Added property "prune.retain.types", which are comma delimited regex patterns that will be kept when pruning, eg. "prune.retain.types = ^http://www.wiziwig.tv/" to keep wiziwig.tv artwork (as used by SportsDevil addon).
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#86
(2013-03-31, 20:29)MilhouseVH Wrote: v0.3.5 on Github - hopefully the version check/update process works...
* Added stats timings to the c/C/nc summary.
Interesting exercise trying to calculate elapsed time across multiple threads. The download times in the table of results will be shown per artwork type (ie. fanart, poster etc.), however the total download time may (and often will) be less than the sum of the individual times as downloads will overlap when using multiple threads. The total download time should equal the sum of individual download times only when using a single thread, or when downloading only a single artwork type.
The "Download" time at the very end of the summary will normally exceed the sum of the individual times, as it will include thread management overhead (startup, shutdown etc.)
* Added additional JSON artwork queries for Genre and Addon when pruning the texture cache.
* Added property "prune.retain.types", which are comma delimited regex patterns that will be kept when pruning, eg. "prune.retain.types = ^http://www.wiziwig.tv/" to keep wiziwig.tv artwork (as used by SportsDevil addon).
Great new version! Always exceeding expectations...Big Grin
The detailed time stats are amazing (and ETAs are very useful).
New suggestion:
-Possibility to add more than one XBMC client to the config file: as it is now, I'm using several cfg files (one for each XBMC client) and before using the script I have to rename the proper cfg file for the client I want to run the script on. Having sections in the cfg file for each XBMC client would add order to this process.
Thanks again for this wonderful script!
Reply
#87
Yeah, I'd have to rework entirely the way the argument parsing is handled, which would be a major pain, particularly as much of the functionality expects the script to be executed on the client (where the host would be localhost). Will give it some thought.

Though off the top of my head, wouldn't it be easier (certainly for me! Wink) if you had a directory per client, each containing the required config file? Then just cd to client1, run the script (which can be in a shared/common location all clients), then cd to client2, running the script against client2... The script should (since v0.3.1 or so) look in the current directory first for a config file, then if it doesn't find a config file, it will look in the script folder.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#88
(2013-04-01, 17:29)MilhouseVH Wrote: Yeah, I'd have to rework entirely the way the argument parsing is handled, which would be a major pain, particularly as much of the functionality expects the script to be executed on the client (where the host would be localhost). Will give it some thought.
Though off the top of my head, wouldn't it be easier (certainly for me! Wink) if you had a directory per client, each containing the required config file? Then just cd to client1, run the script (which can be in a shared/common location all clients), then cd to client2, running the script against client2... The script should (since v0.3.1 or so) look in the current directory first for a config file, then if it doesn't find a config file, it will look in the script folder.
It's a good alternative, but it would be neat to maintain only one cfg file and updating the script only once.

Suggestion 2:
-Handling CTRL-C for graceful exits Smile
e.g.
Code:
import signal
import sys
def signal_handler(signal, frame):
        #Call the stats routine here
        sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
Reply
#89
(2013-04-01, 17:49)charrua Wrote: Suggestion 2:
-Handling CTRL-C for graceful exits Smile

It tries already, by wrapping the call to main in a try/exception that catches KeyboardInterrupt and SystemExit but sometimes it's possible to interrupt the interpreter before main has even started, and you'll get a nasty traceback. Most times it shouldn't be a problem... I think.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#90
(2013-04-01, 17:52)MilhouseVH Wrote:
(2013-04-01, 17:49)charrua Wrote: Suggestion 2:
-Handling CTRL-C for graceful exits Smile
It tries already, by wrapping the call to main in a try/exception that catches KeyboardInterrupt and SystemExit but sometimes it's possible to interrupt the interpreter before main has even started, and you'll get a nasty traceback. Most times it shouldn't be a problem... I think.
You're right, probably I'd tried to CTRL-C an old version of the script, now I see that in the last one it's already working fine.
Reply
  • 1
  • 4
  • 5
  • 6(current)
  • 7
  • 8
  • 197

Logout Mark Read Team Forum Stats Members Help
[RELEASE] Texture Cache Maintenance utility17