• 1
  • 141
  • 142
  • 143(current)
  • 144
  • 145
  • 197
[RELEASE] Texture Cache Maintenance utility
Alright, I'll move it into a separate thread then. You've been very helpful in this process, so once again thank you very much for staying with me.

(2016-12-15, 19:57)vbat99 Wrote: With all the hassle your going through, why not use a offline scraping program like mediacompanion, scrape all movie data to be with your mother, pick your artwork etc, and then set Kodi scrapes to local info only to import your movie data.

Just my 2 cents ...

Thanks for the suggestion. I don't know to what level I could automate said program (because I don't want to open it every time I add a new film), but it seems to have command line options so that's neat. However, it appears to be Windows only, which is something I want to avoid since I want to move on to another OS in the future.

Hopefully I'll just get this thing fixed and use the texturecache script. Here's the thread (with the video) if you're curious.
Reply
I noticed that the P argument deleted all programs/addons thumbs, which I suppose is normal behaviour as programs and addons are not part of any media library in Kodi.
As a user of Wintermute's excellent Advanced Emulator Launcher, I was wondering if there was a way to completely recache artworks for this addon.

Texturecache.py C addons doesn't do it. But maybe I am missing something, or maybe it is not currently possible.
I understand the addon do use json databases so that may be something doable, and that would be really nice since users of the addon can have quite large db and associated artworks

Many thanks in advance
Reply
I see that your first page indicates that either v2 or v3 Python for Windows can be used. Do you have a recommendation of which you feel might be more stable and future proof with your utility?
HTPC: Dell Optiplex 7050 SFF i7-7700 quad-core, 3.6GHz, 16GB
NAS: Synology DS1813+ and DX513, Hybrid RAID (SHR) 48TB usable space
My Media Center | www.CaptainKen.us | www.YouTube.com/KenInGilbert
Reply
(2016-12-23, 20:04)CaptainKen Wrote: I see that your first page indicates that either v2 or v3 Python for Windows can be used. Do you have a recommendation of which you feel might be more stable and future proof with your utility?

I do my best to support both v2 and v3, so you should be OK with either. I'm pretty sure both Python v2 and v3 will be supported for the foreseeable future on Windows.

Python v2 is by far the most common version I see running this script, with a ratio of 17:3 in favour of v2 (based on 2.96M version checks).
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2016-12-23, 20:40)Milhouse Wrote:
(2016-12-23, 20:04)CaptainKen Wrote: I see that your first page indicates that either v2 or v3 Python for Windows can be used. Do you have a recommendation of which you feel might be more stable and future proof with your utility?

I do my best to support both v2 and v3, so you should be OK with either. I'm pretty sure both Python v2 and v3 will be supported for the foreseeable future on Windows.

Python v2 is by far the most common version I see running this script, with a ratio of 17:3 in favour of v2 (based on 2.96M version checks).

Thank you.
HTPC: Dell Optiplex 7050 SFF i7-7700 quad-core, 3.6GHz, 16GB
NAS: Synology DS1813+ and DX513, Hybrid RAID (SHR) 48TB usable space
My Media Center | www.CaptainKen.us | www.YouTube.com/KenInGilbert
Reply
(2016-09-14, 11:54)Milhouse Wrote: "Orphaned file" means you have an add-on that is stuffing the animated file into your Thumbnails folder without creating a corresponding entry (row) in Textures13.db. This is an abuse of the texture cache system. It's possible you might not even want these large animated files in your texture cache as (if you have many of them) it will lead to a lot of bloat.

For the texture cache to work optimally the add-on should also be creating the associations between the movie and the artwork (eg. poster-animated, fanart-animated as alternatives to poster and fanart).

I'd suggest contacting the author of whatever add-on is creating these gif files in your Thumbnails folder and ask them to consider a better joined-up solution.

That would be me ;-)

Animated gifs are used by some users and I offer a few tools to help them grab them.
The preference is to not store these animated gifs at all and just link to the remote location and let the Kodi texture cache do it's magic OR store them alongside with the media (by user preference) and in that case, the Kodi texture cache will also do it's magic and create a local copy of the image.

BUT... It seems that the Texture cache isn't capapable of handling the animated gifs. As soon as they're located outside a local location, the GIF is not animated anymore.
Makes sense as the texture cache creates a local cached version for the specified format which I guess is not aware of multiple frames within the image...

This is true for ALL animated gifs as far as I'm aware of, not only this specific use case with animated posters but also if you try to set a animated gif as your skin background.

I deciced to just store them locally for the time being and look at the Kodi source to find for an answer to fix the root cause.
The Thumbnails folder seemed like a safe place to them as that's used to store... cached images.

If the user cleans up the Thumbnails folder, the images will be redownloaded if needed by request.

What would be the best solution in your opinion:

1) Try the fix the root cause so the texture cache is ignored for multiframe/animated gif files.

2) Place the gifs in another local location, like temp or addon_data folder

3) ....


BTW: The artwork is actually written to the Kodidb by json API, meaning an artwork item animatedposter is added.

BTW2: The images are only downloaded if the user explicitely enabled animated artwork.
Reply
Hi Marcel.

(2017-01-04, 23:44)marcelveldt Wrote: What would be the best solution in your opinion:

1) Try the fix the root cause so the texture cache is ignored for multiframe/animated gif files.

2) Place the gifs in another local location, like temp or addon_data folder

3) ....

Ultimately #1 would be best, as ideally Kodi should handle GIF artwork in a similar way it does current non-GIF artwork. I'm a little undecided if GIF artwork should be cached, as the files can be quite large, but to be consistent with other artwork then it should be cached, as this also means remote (internet-based) GIF artwork can be used.

However until that happens, placing these GIF files in a different location would avoid filling the Thumbnails folder with (currently) uncacheable artwork, particularly if users subsequently try to "fix" their cache by running a command such as "texturecache.py R" as this will delete all the GIF files.

(2017-01-04, 23:44)marcelveldt Wrote: BTW: The artwork is actually written to the Kodidb by json API, meaning an artwork item animatedposter is added.

OK, that's half the battle. Smile

So you are creating entries such as the following, from this post:
Code:
osmc@SimorghOSMC:~$ ./texturecache.py jd movies Zootopia
[                        
  {
    "art": {
      "animatedfanart": "image://special://thumbnails/animatedgifs/tt2948356_fanart.gif/",
      "animatedposter": "image://special://thumbnails/animatedgifs/tt2948356_poster.gif/",
      "fanart": "image://smb://10.0.1.120:445/Sohrab15/Muvees 15/Zootopia (2016)/Zootopia 2016 1080p BluRay x264 DTS-JYK-fanart.jpg/",
      "poster": "image://smb://10.0.1.120:445/Sohrab15/Muvees 15/Zootopia (2016)/Zootopia 2016 1080p BluRay x264 DTS-JYK-poster.jpg/"
    },
    "file": "smb://10.0.1.120:445/Sohrab15/Muvees 15/Zootopia (2016)/Zootopia 2016 1080p BluRay x264 DTS-JYK.mkv",
    "label": "Zootopia",
    "movieid": 434,
    "title": "Zootopia"
  }

In that case the only risk texturecache.py poses to these GIF images is when the "R" (orphan) command is run, as it will identify all of the files in the animatedgifs folder as "orphan" files since they're not being referenced by a row in Textures13.db.

Two options here... 1) don't run the "R" command, or 2) I add an option to ignore certain "classes" of orphan files.

If the animated files were stored outside of the Thumbnails folder, then #2 isn't necessary as #1 no longer poses a problem.

If better GIF integration ever happens in Kodi, then #2 again won't be required as any cached GIF artwork would no longer be seen as "orphaned" (unless it really is an orphan!) However I realise that improved GIF support in Kodi is going to take a long time, and may never happen at all.

So yes, relocating the GIF artwork outside of the Thumbnails folder would IMHO be a good first step, with a long term aim to improve GIF integration (best of luck!) ;-)
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Thank you for your answers @millhouse @marcelveldt

Hope you find the best solution for this behaviour.
Reply
Last night i re-installed my Kodi setup and for some reason my PVR icons are missing.
So i thought running texture cache util would resolve this but it didn't.

./texturecache.py C pvr.tv

# ./texturecache.py C pvr.tv

Cache pre-load activity summary for "pvr.tv":

| thumbnail | TOTAL
--------------+-------------+-------------
Cached | - | 0
Deleted | - | 0
Duplicate | - | 0
Error | - | 0
Ignored | - | 0
Skipped | - | 0
Undefined | - | 0
==========================================
TOTAL | - | 0

Loading: 00:00:00.00
Parsing: 00:00:00.00
TOTAL RUNTIME: 00:00:00.00

It looks like it doesn't detect PVR at all although viewing PVR itself works just fine so is EPG data.

Any ideas what goes wrong here? (latest Krypton version on LibreELEC)
Reply
Can you send me a link to your @logfile=tc.log
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2017-01-07, 17:17)Milhouse Wrote: Can you send me a link to your @logfile=tc.log

This? http://pastebin.com/1fmgFScj

And a verbose version: http://pastebin.com/XTy7m9VH

Did found something strange i think, i see this in the log.
2017-01-08 11:09:30.395583:MainThread: libPVR.PARSING JSON DATA: {"id":"libPVR","jsonrpc":"2.0","result":{"System.GetBool(pvrmanager.enabled)":false}}

If that's for PVR to work then it's funny it works currently.

/edit:
Looks like pvr manager is broken. I removed pvr client, rebooted, reinstalled newer version but pvr manager won't start but LiveTV works.
Reply
pvrmanager is now possibly a hangover from the earlier implementation of PVRs and needs to be disregarded with more recent builds of Kodi - I'll have a look at your log and try to get to the bottom of it.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2017-01-05, 02:19)Milhouse Wrote: So yes, relocating the GIF artwork outside of the Thumbnails folder would IMHO be a good first step, with a long term aim to improve GIF integration (best of luck!) ;-)

OK, thanks for the detailed answer. So, I will proceed to change the default location.
Any suggestions for that new location ? Just the addon_data folder of my addon ?

Best option is indeed to improve the texture cache so the animated gifs are supported and the images can be located anywhere but like you said yourself that won't be easy to fix. I took a quick peek at some of the code in Kodi's source and got dizzy very fast so maybe I should start with something easier first ;-)

Can we agree that I keep the old images in place and only put new images in the new location (or redownloaded once if they're removed) ?
Or do you prefer if I put some move action in code ?
Reply
(2017-01-08, 11:58)schumi2004 Wrote: This? http://pastebin.com/1fmgFScj

And a verbose version: http://pastebin.com/XTy7m9VH

Did found something strange i think, i see this in the log.
2017-01-08 11:09:30.395583:MainThread: libPVR.PARSING JSON DATA: {"id":"libPVR","jsonrpc":"2.0","result":{"System.GetBool(pvrmanager.enabled)":false}}

If that's for PVR to work then it's funny it works currently.

/edit:
Looks like pvr manager is broken. I removed pvr client, rebooted, reinstalled newer version but pvr manager won't start but LiveTV works.

I've pushed an update, v2.3.4, which uses an alternative method to determine if PVR is enabled as pvrmanager.enabled is no longer available in Kodi 18. Thanks for reporting.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2017-01-11, 00:59)marcelveldt Wrote: Any suggestions for that new location ? Just the addon_data folder of my addon ?

That would be the best choice, I think, as the artwork being created is then "owned" completely by the add-on.

(2017-01-11, 00:59)marcelveldt Wrote: Can we agree that I keep the old images in place and only put new images in the new location (or redownloaded once if they're removed) ?
Or do you prefer if I put some move action in code ?

I have no objection to old images being kept where they are, so long as users are aware their old artwork will be removed by the "R" command. Smile
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
  • 1
  • 141
  • 142
  • 143(current)
  • 144
  • 145
  • 197

Logout Mark Read Team Forum Stats Members Help
[RELEASE] Texture Cache Maintenance utility17