Guest - Testers are needed for the reworked CDateTime core component. See... https://forum.kodi.tv/showthread.php?tid=378981 (September 29) x
  • 1
  • 26
  • 27
  • 28(current)
  • 29
  • 30
  • 197
[RELEASE] Texture Cache Maintenance utility
Great script!!

I just added a few hundred images to the mymusic32.artistinfo table in the strFanart and strImage fields. The images show up when when I browse to the artist and select to view fanart or thumbs but the images do not show when I just scroll through the list of artists (I am giving XBMC time to download the images from the URLs I added). WIll running this with argument 'C' cause XBMC to download and assign these new images as thumbnails and fanart? This would go much faster than going into artist information each one and selecting 'Get Thumb' and 'Get Fanart' and selecting the image from the list. If not with this script do you know of some other way to achieve this with minimal manual effort?

Thanks
XBMC Frodo front end on WIndows 7
MySQL Backend on Windows 8 based server
NextPVR back end
Reply
(2013-10-06, 01:22)skj88 Wrote: WIll running this with argument 'C' cause XBMC to download and assign these new images as thumbnails and fanart?

It should do, as long as you see the correct thumbnail and fanart urls when you run:
Code:
./texturecache.py jd artists

then
Code:
./texturecache.py C artists

will forcibly re-cache just your artist thumbs.

(2013-10-06, 01:22)skj88 Wrote: do you know of some other way to achieve this with minimal manual effort?

Nope.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2013-10-06, 01:22)skj88 Wrote: I just added a few hundred images to the mymusic32.artistinfo table in the strFanart and strImage fields.

To be honest, I'm not sure if that's the right thing to do - have you looked at the "art" table and how it relates to artists, albums, songs etc via the media_id and media_type columns? I wouldn't be at all surprised if your SQL changes are not showing up when you run "jd artists".
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2013-10-03, 23:46)MilhouseVH Wrote:
(2013-10-03, 14:25)CaptainKen Wrote: While not all my sets contain set artwork, MSAA should drill down and use the first movie fanart and poster, but they do not.

Although that seems to be what MSAA did for Ocean's Collection - as it couldn't access the set fanart (as it was blocked), it drilled down and selected the fanart for one of the movies (1960's version). Try running MSAA again (preferably on a Linux box) and you'll probably find it selects the correct fanart for Ocean's Collection this time.

(2013-10-03, 14:25)CaptainKen Wrote: I'll get a verbose log Saturday as I'm running late now for airport.

OK but not sure when I became support for someone else's add-on! Wink Smile

I'm pretty sure this is not a texturecache related issue. I'm also pretty sure that caching artwork with mangled filenames is no longer a problem, so if your movie sets are not caching the correct artwork then most likely there's a problem with MSAA assigning the wrong artwork in your media library. You can see which artwork is assigned to your movie sets using "jd sets", so if MSAA has assigned incorrect artwork, that will need fixing in MSAA (or your file system, if files are blocked or don't exist).

I'll take a look at your next log - if you're running MSAA again, also include the xbmc.log too.

Update
- Adding set fanart and poster to LOR now shows the set artwork. (Odd that only some movies without set artwork would not drill down to display other artwork for the set.)
- Ocean's poster was a png file, so changed it to jpg and it now shows properly Confused
- Ice Age was a jpeg file, so changed to jpg and now shows properly Confused

I've started to compile a MSAA troubleshooting check list! If you have anything to add, have at it.

If the fanart for a set isn't showing up:
- Press the "I" Info button and Backspace to force a refresh
- Make sure the fanart and poster are jpg, not png, jpeg or any other format
- Check the file permissions
- Make sure the Read-only or Hidden file attributes are not checked
- Make sure within the file attribute window that there isn't a Security Block:
"This file came from another computer and might be blocked to help protect this computer."
- Run "texturecache.py C sets" to recache
- Run "texturecache.py jd sets > LogFilePathAndNameHere" to see what artwork is being used for the set
- Even though MSAA is supposed to drill down and use fanart and poster from a movie in the set, if none exist for the set itself, sometimes you must have actual artwork for the actual set.

THANK YOU FOR ALL YOUR HELP AND PATIENCE!

I'd like to make a contribution to your cause, please point me to where I can do this.
HTPC: Dell Optiplex 7050 SFF i7-7700 quad-core, 3.6GHz, 16GB
NAS: Synology DS1813+ and DX513, Hybrid RAID (SHR) 48TB usable space
My Media Center | www.CaptainKen.us | www.YouTube.com/KenInGilbert
Reply
(2013-10-06, 04:55)CaptainKen Wrote: - Ocean's poster was a png file, so changed it to jpg and it now shows properly Confused

I thought MSAA supported png? Maybe not, mine are all jpg. It's a shame such a useful addon is no longer supported.

(2013-10-06, 04:55)CaptainKen Wrote: I'd like to make a contribution to your cause, please point me to where I can do this.

Thanks but kind words are generally sufficient! I wrote this to scratch my own itch, if it's useful to others (or I can help resolve related issues) that's just a bonus! Smile
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Version 1.0.2
* Add: New options, set and testset, to allow limited modification of movie, tvshow, episode, musicvideo, artist, album and song library items. Use testset to verify the request is valid before performing any update.

Example:

Code:
./texturecache.py set movie 312
                      art.clearlogo "nfs://myserver/movies/thismovie-logo.png" \
                      art.clearart "nfs://myserver/movies/thismovie-clearart.png" \
                      playcount 12 \
                      trailer "http://www.totaleclips.com/Player/Bounce.aspx?eclipid=e121648&bitrateid=449&vendorid=102&type=.mp" \
                      tag "['horror', 'zombies']"

to set clearlogo, clearart, playcount and tag fields for the movie with the movieid 312.

Most basic fields can be specified (eg. plot, trailer, playcount, art etc. - see JSON API v6 for details of which fields can be specified on the Set*Details calls). However modification of more complex fields - such as cast, streamdetails etc. - is not supported by JSON. Also, the file field cannot be modified.

In addition, for the sake of efficiency, batches of data can also be read from stdin. In the following example, the two movies and one tv show are to be updated. The fields being updated are specified by the "items" list within each movie or tv show (in each case, setting new clearart and clearlogo artwork urls).

The file update.dat contains the following information:

Code:
[
  {
    "libraryid": 1,
    "items": {
      "art.clearart": "nfs://192.168.0.3/mnt/share/media/Video/MoviesSD/9 (2009)[DVDRip]-clearart.png",
      "art.clearlogo": "nfs://192.168.0.3/mnt/share/media/Video/MoviesSD/9 (2009)[DVDRip]-logo.png"
    },
    "type": "movie",
    "title": "9"
  },
  {
    "libraryid": 358,
    "items": {
      "art.clearart": "nfs://192.168.0.3/mnt/share/media/Video/MoviesHD/Classics/12 Angry Men (1957)[BDRip]-clearart.png",
      "art.clearlogo": "nfs://192.168.0.3/mnt/share/media/Video/MoviesHD/Classics/12 Angry Men (1957)[BDRip]-logo.png"
    },
    "type": "movie",
    "title": "12 Angry Men"
  },
  {
    "libraryid": 115,
    "items": {
      "art.clearart": "nfs://192.168.0.3/mnt/share/media/Video-Private/TVShows/Arrested Development/clearart.png",
      "art.clearlogo": "nfs://192.168.0.3/mnt/share/media/Video-Private/TVShows/Arrested Development/logo.png"
    },
    "type": "tvshow",
    "title": "Arrested Development"
  }
]
and to apply the update:
Code:
cat update.dat | ./texturecache.py set

* Added of tools/mktools.py which can read in the output from "texturecache.py jd movies" or "texturecache.py jd tvshows" and convert remote artwork to local. It will retrieve the original remote artwork from the web site and write it into your media directory. Output from mklocal.py can be fed into "texturecache,py set" to re-point your media library so that it now uses the local artwork. Run mklocal.py in different ways to download remote artwork, or just assign existing local artwork to your media library. See --help for more details.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Is it possible to preload the previews for the movie addons content? Let's say to preload the preview images of the 10 first list entries of all subscribed YouTube channels.
Would be a great help for small devices like the Rpi to improve the navigation, when I could preload those images at night.
Reply
(2013-10-19, 13:44)namtih Wrote: Is it possible to preload the previews for the movie addons content? Let's say to preload the preview images of the 10 first list entries of all subscribed YouTube channels.
Would be a great help for small devices like the Rpi to improve the navigation, when I could preload those images at night.

Not unless the preview image urls are in the media library (and they're not likely to be). To be honest, this sounds more like a request for the addon author - add a function that would pre-load preview images on a scheduled basis.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
I am getting this while running [c] option

Code:
E:\Backup\backup\Misc>texturecache.py c
Traceback (most recent call last):
  File "E:\Backup\backup\Misc\texturecache.py", line 4478, in <module>
    main(sys.argv[1:])
  File "E:\Backup\backup\Misc\texturecache.py", line 4282, in main
    if not checkConfig(argv[0]): sys.exit(2)
  File "E:\Backup\backup\Misc\texturecache.py", line 4039, in checkConfig
    data = jcomms.sendJSON(REQUEST, "libPing", timeout=7.5, checkResult=False, useWebServer=True)
  File "E:\Backup\backup\Misc\texturecache.py", line 919, in sendJSON
    data = self.sendWeb("POST", "/jsonrpc", json.dumps(request), {"Content-Type": "application/json"}, timeout=timeout)
  File "E:\Backup\backup\Misc\texturecache.py", line 887, in sendWeb
    raise httplib.HTTPException("Remote web host requires webserver.username/webserver.password properties")
httplib.HTTPException: Remote web host requires webserver.username/webserver.password properties

I can successfully Prune texture cache.. (p, P) Some help please?
~Web and graphic designer... Linux enthusiast... Python Fan... A Gooner~
[AMD A10-7850K 3.7 Ghz, Radeon R7]
[Fedora 27] | [Kodi - 17.6 / Skin - Grid]

Image
Reply
(2013-10-19, 15:05)akya Wrote: Some help please?

It's telling you what the problem is - your XBMC web server has a password/username.

You need to add the following properties to your texturecache.cfg file:
Code:
webserver.username = your-username
webserver.password = your-password

Or alternatively remove the username/password from your webserver settings in the XBMC GUI.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2013-10-19, 15:14)MilhouseVH Wrote:
(2013-10-19, 15:05)akya Wrote: Some help please?

It's telling you what the problem is - your XBMC web server has a password/username.

You need to add the following properties to your texturecache.cfg file:
Code:
webserver.username = your-username
webserver.password = your-password

Or alternatively remove the username/password from your webserver settings in the XBMC GUI.
Thank you ! Smile
~Web and graphic designer... Linux enthusiast... Python Fan... A Gooner~
[AMD A10-7850K 3.7 Ghz, Radeon R7]
[Fedora 27] | [Kodi - 17.6 / Skin - Grid]

Image
Reply
A brief note on the mklocal.py tool, which is basically a quick method to populate your media source with original quality artwork (unlike Export Library, which uses potentially image quality compromised artwork from the texture cache).

Example usage:
Code:
./texturecache.py jd movies | ./mklocal.py --local /freenas/media --prefix nfs://192.168.0.3/mnt/share/media --artwork fanart poster clearlogo clearart --output fixup.dat

where /freenas/media is the OS mount point for your media, such that XBMC media source nfs://192.168.0.3/mnt/share/media is mapped to /freenas/media and vice versa.

The above example would process each of your movies for the specified artwork types - fanart, poster, clearlogo and clearart - and, if the artwork file doesn't already exist, and the library url is remote (ie. http://), the artwork item will be downloaded from the source web site and written to the --local folder alongside the corresponding movie file.

Artwork will be named by appending the artwork type to the movie file name, eg. "/freenas/media/Video/MoviesHD/Classics/12 Angry Men (1957)[BDRip]-poster.jpg" will be mapped to "nfs://192.168.0.3/mnt/share/media/Video/MoviesHD/Classics/12 Angry Men (1957)[BDRip]-poster.jpg".

Once you have written the artwork to your media folders, update your media library to use the new local urls with the following call:
Code:
cat fixup.dat | ./texturecache.py set

Also works for tvshows. Add "--season" to also download season artwork, although note there are no JSON methods to modify season artwork so these cannot be set automatically (fixed in recent Gotham nightly, requires JSON API v6.12.0+). Add "--episode" to also download episode thumbs, which can be set automatically.

You can of course test this by not writing to your media folder:
Code:
./texturecache.py jd movies | ./mklocal.py --local /freenas/media/TestFolder --prefix nfs://192.168.0.3/mnt/share/media --artwork fanart poster clearlogo clearart --output fixup.dat
as this will download the remote artwork to TestFolder, and should replicate your existing movie directory structure. Once you are comfortable all is working OK, specify the correct --local folder for your media library. Of course no changes will be applied to your media library until you run the changes through "./texturecache.py set".
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Just wanted to say thanks for filling this huge void in native XBMC functionality.
I've updated my old Thumb-cleaning tool to point to this page
XBMC.MyLibrary (add anything to the library)
ForTheLibrary (Argus TV & XBMC Library PVR Integration)
SageTV & XBMC PVR Integration
Reply
Version 1.0.3 (25/10/2013)
* Fix: issue #9, error during prune when dds file already deleted
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
@MilhouseVH: PR for adding texture cache accessibility via JSON-RPC is up:

https://github.com/xbmc/xbmc/pull/3499

Would appreciate your comments/testing. It might not have everything you want yet, but you should be able to get most stuff from the database at least.

Cheers,
Jonathan
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
Reply
  • 1
  • 26
  • 27
  • 28(current)
  • 29
  • 30
  • 197

Logout Mark Read Team Forum Stats Members Help
[RELEASE] Texture Cache Maintenance utility17
This forum uses Lukasz Tkacz MyBB addons.