• 1
  • 71
  • 72
  • 73(current)
  • 74
  • 75
  • 197
[RELEASE] Texture Cache Maintenance utility
It was no prompt its the default progress bar dialogue with cancel button on, which should not be visible during video playback, like none should.
Reply
Are you able to test one of my test builds, one that includes PR5324 - anything from #0909 onwards.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
I dont use OE, but I compile myself.

That PR is not what I meant, it sounds like all or nothing job, when you have video/music fullscreen playing, dialogs or kay toasts should remain in background/ delayed until playback stopped or something. Thats what happens with kai toast for e.g when addons update. But ill test this PR.
Reply
Does anyone know how to this on start-up? I have been using cron but my needs have changed and running from cron isnt really required anymore. Running on start-up is going to be a better option for me now.
Reply
Startup of what, the machine or xbmc? If the former, try adding a cron entry with a time of "@reboot", or add it to your distributions preferred startup system (assuming it's one of the Pis in your signature, adding it to /etc/rc.local should work). If you're running OpenELEC, add it to /storage/.config/autostart.sh.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
@Milhouse I have my script i used with cron, So gathering I can use the same script as i know it works fine on openelec. Any idea how to add it to autostart.sh?
Reply
Just edit autostart.sh with vi or nano over ssh, or access the \\<ipaddress>\Configfiles share on your OpenELEC Pi and create/edit the autostart.sh file using an editor that can create Unix-format files (Notepad++, TextPad etc. on Windows). The autostart.sh doesn't require execute permissions so no need for chmod +x.

You'll want to fork your script so that it doesn't prevent Kodi from loading, and your script will need to wait ~20 seconds until xbmc.bin has started (so that texturecache.py has got something to talk to), in which case adding something like the following to the end of autostart.sh should work:
Code:
(sleep 20; /storage/mytexturescript.sh) &
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
I'm getting a truckload of Duplicates and I've tried a whole host of options but the number never seems to decrease. Which flag can I run to remove the duplicates?
Reply
(2014-09-16, 15:20)raspberry_pd Wrote: I'm getting a truckload of Duplicates and I've tried a whole host of options but the number never seems to decrease. Which flag can I run to remove the duplicates?

There's usually nothing wrong with duplicates. I've explained previously what they are.

They're extremely common in TV shows because of the hierarchical nature of the data, and there's nothing you can/should do about that.

Not so common in movies or music unless you've repeatedly used the same artwork for different movies, albums or artists in which case you should (if you want to) replace with alternative artwork.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Thanks Milhouse.

Do you know a way to force XBMC to download the latest artwork from TVDB? I have a few cases where my main view - Posters in the Amber skin, horizontally squishes baners into posters when there is a legitimate poster available on the TVDB.

I've been using
Quote:./texturecache.py C
though it takes a while so has not finished running through yet (Raspbmc keeps freezing on me overnight).
Reply
(2014-09-17, 15:02)raspberry_pd Wrote: Do you know a way to force XBMC to download the latest artwork from TVDB? I have a few cases where my main view - Posters in the Amber skin, horizontally squishes baners into posters when there is a legitimate poster available on the TVDB.

I'm not entirely sure what the problem is.

If your media library has banner artwork in place of posters then there's not much texturecache.py can do to fix that - whatever metadata scraper you used to scrape your library has done a poor job and you'll need to fix your media library before you will be able to cache the correct artwork.

If your media library contains references to the correct artwork (posters) then its difficult to understand how this could be cached incorrectly (as banners).

You can view the artwork for a tvshow with
Code:
./texturecache.py jd tvshows <showname>

and see what banners or posters are in use.

More likely though it's a skin problem of some sort, with banners being displayed with the incorrect aspect ratio. Or you are missing posters from your media library so banners are used instead - you'll need to use something like the Artwork Downloader addon to try and fill in any missing artwork. Or use local artwork and re-scrape.

(2014-09-17, 15:02)raspberry_pd Wrote: I've been using
Quote:./texturecache.py C
though it takes a while so has not finished running through yet

If you just want to re-cache the artwork just for tvshows:
Code:
./texturecache.py C tvshows

(2014-09-17, 15:02)raspberry_pd Wrote: (Raspbmc keeps freezing on me overnight).

Try a different power supply, that shouldn't happen (the freezing). Or try a more recent build of whatever distribution you're running.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2014-09-17, 17:17)Milhouse Wrote:
(2014-09-17, 15:02)raspberry_pd Wrote: Do you know a way to force XBMC to download the latest artwork from TVDB? I have a few cases where my main view - Posters in the Amber skin, horizontally squishes baners into posters when there is a legitimate poster available on the TVDB.

I'm not entirely sure what the problem is.

If your media library has banner artwork in place of posters then there's not much texturecache.py can do to fix that - whatever metadata scraper you used to scrape your library has done a poor job and you'll need to fix your media library before you will be able to cache the correct artwork.

If your media library contains references to the correct artwork (posters) then its difficult to understand how this could be cached incorrectly (as banners).

You can view the artwork for a tvshow with

Code:
./texturecache.py jd tvshows <showname>

and see what banners or posters are in use.

Thanks, very handy to have the code's author available to wade through the mountain of power script flags Smile

I'm just using the default XBMC TVDB scraper. I understand your point for sure. I was kind of hoping to short circuit the problem with texturecache because there appears no other way to fix my media library.

It's strange. Here's an example:

Code:
pi@raspbmc:~$ ./texturecache.py jd tvshows 'Kim Philby'
[
  {
    "art": {
      "banner": "image://http://thetvdb.com/banners/graphical/280304-g.jpg/",
      "fanart": "image://http://thetvdb.com/banners/fanart/original/280304-2.jpg/"
    },
    "file": "smb://<...>K/Kim.Philby-.His.Most.Intimate.Betrayal/",
    "label": "Kim Philby: His Most Intimate Betrayal",

So the problem here seems to be that the scraper has not downloaded a poster, just fanart and this banner:

http://thetvdb.com/banners/graphical/280304-g.jpg

and the skin (Amber) has chosen to use the banner - squished horizontally - instead of showing an empty spot.

However, a poster clearly exists:

http://thetvdb.com/banners/posters/280304-1.jpg

The poster may not have existed when I first added that show to my library but it does now and a "Refresh" on this show alone does not download the poster. Very strange.

(2014-09-17, 17:17)Milhouse Wrote: More likely though it's a skin problem of some sort, with banners being displayed with the incorrect aspect ratio. Or you are missing posters from your media library so banners are used instead - you'll need to use something like the Artwork Downloader addon to try and fill in any missing artwork. Or use local artwork and re-scrape.

Thanks for the tip. The frustrating part is I've just migrated to Raspbmc from using XBian for a long time. I managed to manually move across my library with watched/unwatched etc and moved the Thumbnails folder as well (if for no other reason than to try and reduce the load on thetvdb) but before I did that, whilst testing the initial Raspbmc install with fresh settings on a handful of my TV shows, it scraped much better. It pulled down more posters that have only been added to thetvdb recently.

I hope the problem is not due to me being stuck with all my watched statuses, and media library metadata, in an old XBMC database format. I loosely remember reading that the XBMC database schema changes often enough to be a ball ache (at least, when using an external MySQL db, which I'm not).

I'm very happy to tweak away at curating my library to present it as cleanly as possible, just need the skills/tools Smile Sounds like Artwork Downloader might be worth a shot, thanks again.

Speaking of local artwork, I'm interested in using your mklocal.py again though I remember the syntax was somewhat involved and I certainly don't remember it now, it's been a while Smile

(2014-09-17, 17:17)Milhouse Wrote:
(2014-09-17, 15:02)raspberry_pd Wrote: I've been using
Quote:./texturecache.py C
though it takes a while so has not finished running through yet

If you just want to re-cache the artwork just for tvshows:
Code:
./texturecache.py C tvshows

Thanks again for the tips. That should save a chunk of time.
Reply
(2014-09-17, 20:57)raspberry_pd Wrote: I hope the problem is not due to me being stuck with all my watched statuses, and media library metadata, in an old XBMC database format. I loosely remember reading that the XBMC database schema changes often enough to be a ball ache (at least, when using an external MySQL db, which I'm not).

Backing up and restoring your movie/tvshow watched statuses is fairly straightforward:

To backup the watched statuses of an existing library:
Code:
./texturecache.py watched movies backup movies.dat
./texturecache.py watched tvshows backup tvshows.dat

Rescan/rescrape a fresh library, then restore your old watched statuses:
Code:
./texturecache.py watched movies restore movies.dat
./texturecache.py watched tvshows restore tvshows.dat

It can be quite slow to restore the status unless you're running a recent Helix (in which case it's much quicker!)

(2014-09-17, 20:57)raspberry_pd Wrote: I'm very happy to tweak away at curating my library to present it as cleanly as possible, just need the skills/tools Smile Sounds like Artwork Downloader might be worth a shot, thanks again.

Give AD a go, it should (in theory) help fill in some of your tvshow blanks.

Also try running "./texturecache.py qa tvshows @qaperiod=9999" - this may flag up missing artwork (and possibly other issues).

(2014-09-17, 20:57)raspberry_pd Wrote: Speaking of local artwork, I'm interested in using your mklocal.py again though I remember the syntax was somewhat involved and I certainly don't remember it now, it's been a while Smile
Yeah, it's a bit fiddly to get going but once you have it working it's something that I think can be quite useful. mklocal.py can convert your existing original remote artwork (if still available) to local, and can also assign any new local artwork (logos, clearart, discart etc.) it finds in your filesystem to your movies/tvshows. Any artwork no longer available remotely or found locally will be removed from the media library (unless you want them kept) allowing addons (such as AD) to repopulate the library with fresh artwork.

If you're not sure of the options, run "./mklocal.py -h". Have a look at the previous posts on mklocal, and if you get stuck then ask.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2014-09-14, 02:30)Milhouse Wrote: Are you able to test one of my test builds, one that includes PR5324 - anything from #0909 onwards.

Ive tested it and seems to work. (compiled that patch ontop of master)

Ive left a message on github regarding the comments someone made about off-line or non available sources, which been brought up as a reason/blocker, it is a bug I linked both the report which hasn't been fixed in master, howvver there's a patch for it but none does anything about it. So anyway. thx for bringing that patch to my attention, does indeed fix issue.
Reply
(2014-05-13, 16:23)Milhouse Wrote:
Code:
usage: mklocal.py [-h] [-l DIRECTORY] [-p PATH] [-A PATH] [-i [FILENAME]] [-o [FILENAME]]
                  [--dryrun] [-r] [--ignorebadprefix] [-a TYPE [TYPE ...]]
                  [-c TYPE [TYPE ...]] [-s [TYPE [TYPE ...]]] [-e [TYPE [TYPE ...]]] [-1]
                  [-nk] [--info] [-q | -v]

Downloads specific artwork types (default: clearart, clearlogo) based on urls in media
library (ie. original source) creating local versions. Avoids retrieving artwork from XBMC
Texture Cache as this is often resized, resampled and of lower quality. Optionally output
data that can be used to update the media library to use new local versions of artwork,
replacing any current remote versions. The same data will also remove invalid remote
artwork from the library.

optional arguments:
  -h, --help             show this help message and exit
  -l DIRECTORY, --local DIRECTORY
                         Local DIRECTORY into which artwork will be WRITTEN, eg.
                         /freenas/media/
  -p PATH, --prefix PATH
                         XBMC PATH prefix (eg. nfs://192.168.0.3/mnt/share/media/) that
                         will be substituted by --local DIRECTORY when traversing media
                         files. This is typically the root of the media source as defined
                         in sources.xml
  -A PATH, --altlocal PATH
                         Alternate local directory which may contain artwork that can be
                         READ and copied to --local, could be the original source folder
  -i [FILENAME], --input [FILENAME]
                         Optional FILENAME containing JSON movie/tvshow data for
                         processing. Read from stdin if FILENAME is - or not specified
  -o [FILENAME], --output [FILENAME]
                         Output a data structure suitable for consumption by
                         texturecache.py [test]set, used to update an XBMC media library
                         converting remote urls into local urls. Written to stdout if
                         FILENAME is - or not specified
  --dryrun               Don't create anything (although downloads will be attempted)
  -r, --readonly         Don't download (or, if specified, copy from --altlocal) new
                         artwork, only use existing --local artwork
  --ignorebadprefix      Don't display a warning for media files with a path that does not
                         match that set by --prefix
  -a TYPE [TYPE ...], --artwork TYPE [TYPE ...]
                         Artwork TYPE(s) for download, eg. "--artwork discart banner"
                         Specify TYPE:SUFFIX if SUFFIX differs from TYPE, eg. "--artwork
                         thumb:poster" would create "thumb" library items with a "-poster"
                         filename suffix
  -c TYPE [TYPE ...], --check TYPE [TYPE ...]
                         Check the named artwork TYPE(s) - or "all" - and warn if any
                         internet (http) URLs are detected
  -s [TYPE [TYPE ...]], --season [TYPE [TYPE ...]]
                         For TV Shows, process season items (default: poster banner
                         landscape)
  -e [TYPE [TYPE ...]], --episode [TYPE [TYPE ...]]
                         For TV Shows, process episode items (default: thumb)
  -1, --singlefolders    Movies are in individual folders so don't use the movie-name as a
                         prefix
  -nk, --nokeep          Don't keep artwork if not able to match with pre-existing local
                         artwork
  --info                 Display informational output to stdout
  -q, --quiet            Silence warnings about missing artwork (NEEDED etc.)
  -v, --verbose          Display diagnostic output

Minimum required arguments are --local, --prefix, --artwork (and --input, which if not specified is read from stdin).

For example, the following would download any remote artwork of the poster/fanart/clearart/clearlogo variety to a movie library rooted at /freenas/media, then associate any already existing or any just downloaded artwork with each movie, outputting changes to newartwork.dat:

Code:
./texturecache.py jd movies zombieland | ./mklocal.py --local /freenas/media --prefix nfs://192.168.0.3/mnt/share/media --artwork poster fanart clearlogo clearart --output newartwork.dat"

I just got this:

Code:
mklocal.py: error: unrecognized arguments: --artwork poster fanart

after trying this:

Code:
./texturecache.py jd tvshows | ./mklocal.py --local /media/drobofs/ --prefix smb://DROBOFS/public/tv/ --artwork poster fanart  --output newartwork.dat

Has that --artwork argument been removed?

It appears the texturecache.py part is still proceeding but will it parse on it's results to mklocal.py?

Eventually I got this: Sad

Code:
Traceback (most recent call last):
  File "./texturecache.py", line 7437, in <module>
    main(sys.argv[1:])
  File "./texturecache.py", line 7261, in main
    extraFields=_extraFields, query=_query)
  File "./texturecache.py", line 4012, in jsonQuery
    jcomms.dumpJSON(data, decode, ensure_ascii)
  File "./texturecache.py", line 2375, in dumpJSON
    self.logger.out(json.dumps(data, indent=2, ensure_ascii=ensure_ascii, sort_keys=True), newLine=True)
  File "./texturecache.py", line 833, in out
    sys.stdout.write("%-s%s" % (udata, NL))
  File "/usr/lib/python2.7/codecs.py", line 352, in write
    self.stream.write(data)
IOError: [Errno 32] Broken pipe
Reply
  • 1
  • 71
  • 72
  • 73(current)
  • 74
  • 75
  • 197

Logout Mark Read Team Forum Stats Members Help
[RELEASE] Texture Cache Maintenance utility17