• 1
  • 155
  • 156
  • 157(current)
  • 158
  • 159
  • 197
[RELEASE] Texture Cache Maintenance utility
(2017-11-12, 00:22)wallacebw Wrote:
(2017-11-10, 20:58)Milhouse Wrote: @wallacebw thanks I've added the support for https to mklocal.py, you'll need to re-download the mklocal.py script to update (it doesn't auto-update).

Thanks, working. One more thing, is is easy to add support for animatedfanart and animatedposter art types?

Possibly, at the very least it would require gif support in addition to png and jpg, so this patch: http://sprunge.us/IYjN

You will probably need to add the artwork types on the command line, ie. "--artwork animatedfanart animatedposter"

Other than that I don't know much about animated artwork, so can't really say what else might not work. I'll take a patch, though.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2017-11-12, 00:52)Milhouse Wrote:
(2017-11-12, 00:22)wallacebw Wrote:
(2017-11-10, 20:58)Milhouse Wrote: @wallacebw thanks I've added the support for https to mklocal.py, you'll need to re-download the mklocal.py script to update (it doesn't auto-update).

Thanks, working. One more thing, is is easy to add support for animatedfanart and animatedposter art types? 

Possibly, at the very least it would require gif support in addition to png and jpg, so this patch: http://sprunge.us/IYjN

You will probably need to add the artwork types on the command line, ie. "--artwork animatedfanart animatedposter"

Other than that I don't know much about animated artwork, so can't really say what else might not work. I'll take a patch, though. 

Thanks... looks like there is also an extension reference on line 666, but it may not be worth merging.  I believe script.module.metadatautils identifies them and adds them to a folder belonging to the service.  e.g.:

      "animatedfanart": "image://special://profile/addon_data/script.module.metadatautils/animatedgifs/tt1253863_fanart.gif/",
      "animatedposter": "image://special://profile/addon_data/script.module.metadatautils/animatedgifs/tt1253863_poster.gif/",

ok, I may need to look for another solution or write something custom, as shifting local artwork may cause excess scope creep and be outside of the intent of this program.

Thanks again for the fast response.
Reply
Problem: Certain TV Shows are repeatedly getting merged together after both items are scanned in using texturecache.py.  The first is always joined to the second and no longer has its own database entry.

Background:  Fresh install of Kodi Krypton 17.5, Windows 10, texturecache.py 2.4.4, "Home" TV show, "Jillian Michaels" workout video.
"Home" .nfo file (named tvshow.nfo within the directory the single TV show is in) - https://pastebin.com/raw/9QHnSfWL
Jillian Michaels .nfo file (named tvshow.nfo within the directory the single TV show is in) - https://pastebin.com/raw/0NqJuR8V

Procedure:
  1. "Set Content" for the documentary TV show "Home".
  2. Scan in media using texturecache.py
    Code:
    texturecache.py vscan "<Path_to_Media>\Home [2009]"
    Quote:Rescanning directory:<Path_to_Media>\Home [2009]...
    Updating library: New tvshowid 1 [Home]
    Updating library: New episodeid 1 [Home S01E01 (Home)]
  3. Scan in artwork using texturecache.py
    Code:
    texturecache.py jd tvshows | tools\mklocal.py --singlefolders --ignorebadprefix --local "<Path_to_Media>\Home [2009]" --prefix "<Path_to_Media>\Home [2009]" --artwork poster fanart clearlogo clearart discart characterart:character banner landscape --season --episode --output | texturecache.py set
    Quote:Progress: 3 of 3
  4. Cache artwork using texturecache.py
  5. Check in Kodi, and "Home" indeed has its very own and separate entry.
  6. "Set Content" for the exercise show "Jillian Michaels - Banish Fat, Boot Metabolism".
  7. Scan in media using texturecache.py
    Code:
    texturecache.py vscan "Path_to_Media\Jillian Michaels - Banish Fat, Boost Metabolism [2009]"
    Quote:Rescanning directory: Path_to_Media\Jillian Michaels - Banish Fat, Boost Metabolism [2009]...
    Updating library: New tvshowid      1 [Jillian Michaels - Banish Fat, Boost Metabolism]
    Updating library: New episodeid     2 [Jillian Michaels - Banish Fat, Boost Metabolism S01E01 (Banish Fat, Boost Metabolism)]
  8. Scan in artwork using texturecache.py
    Code:
    texturecache.py jd tvshows | tools\mklocal.py --singlefolders --ignorebadprefix --local "<Path_to_Media>\Jillian Michaels - Banish Fat, Boost Metabolism [2009]" --prefix "<Path_to_Media>\Jillian Michaels - Banish Fat, Boost Metabolism [2009]" --artwork poster fanart clearlogo clearart discart characterart:character banner landscape --season --episode --output | texturecache.py set
    Quote:Progress: 4 of 4
  9. Cache artwork using texturecache.py
  10. Check in Kodi, and "Home" now shows as an episode in the Jillian Michaels exercise.  Furthermore, "Home" no longer has its own entry in Kodi anymore.
  11. I checked the database using texturecache.py to see what I could.  It seems as if "Home" somehow magically got merged into the Jillian Michaels workout video.
    Code:
    texturecache.py j tvshows Banish
    The output is here - https://pastebin.com/raw/KJuMEWnT

Suggestions: Any ideas what to do next? I'm not sure if this is a bug in texturecache.py or Kodi, or if I am just overlooking something.  Thanks in advance!
Reply
@radigast this is a scanning bug in Kodi.

The texturecache.py script is simply asking Kodi to perform library scans of the relevant folders - what Kodi does after that is entirely down to Kodi, and in this case it's going a little bit wrong.

I would suggest opening a ticket on trac.kodi.tv, and I would say that steps 1, 2, 5, 6, 7 and 10 are the only relevant steps for the ticket - caching has nothing to do with this.

It's clear that when you scan the second show ("Jillian Michaels ...") that Kodi is overwriting the first TV show ("Home") as both are being created with the tvshow id #1, which is why all the episodes appear to be merged together as they're both linked to tvshow id #1. Kodi should be allocating a new tvshow id #2 to the second tvshow ("Jillian Michaels ..."), and then linking the new episode #2 to tvshow id #2.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Hi Milhouse,

Thank you for the script, it's really cool!

A few questions:
- is there a way to locate the thumbnail of a local picture in the Thumbnails folder, by using the filename? I modified a picture and want Kodi to grab the new on as the thumbnail.
- I am accessing Kodi (LibreELEC) remotely from macOS 10.12.6. How do I correctly point to the Userdata folder on my Kodi box? I now use /Volumes/Userdata/, but that means the folder has to be mounted from within macOS. Is there a way to point directly to the folder using the IP address of the box?

Thank you!
Reply
(2017-11-14, 22:43)musicadi Wrote: - is there a way to locate the thumbnail of a local picture in the Thumbnails folder, by using the filename? I modified a picture and want Kodi to grab the new on as the thumbnail.

Use "texturecache.py s filename", eg.
Code:
texturecache.py s /storage/.kodi/addons/plugin.video.netflix/resources/fanart.jpg
will return:
Code:
042009|6/6f56e082.jpg|0720|1280|0009|2017-11-05 22:13:43|2017-11-05 22:11:48|/storage/.kodi/addons/plugin.video.netflix/resources/fanart.jpg
Matching row ids: 42009

where "6/6f56e082.jpg" is the filename within the Thumbnails folder.

You can delete the cached artwork (and matching database row) with "texturecache.py d 42009"

However if you've modified the original thumbnail, then simply re-cache it.

(2017-11-14, 22:43)musicadi Wrote: - I am accessing Kodi (LibreELEC) remotely from macOS 10.12.6. How do I correctly point to the Userdata folder on my Kodi box? I now use /Volumes/Userdata/, but that means the folder has to be mounted from within macOS.

Mounting locally on the machine running the script is one option.

The other option is to push the script to the remote machine and run it locally on the remote machine, as this is easy enough to accomplish with a combination of scp and ssh particularly when scripting a complete solution.

(2017-11-14, 22:43)musicadi Wrote: Is there a way to point directly to the folder using the IP address of the box?

No.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2017-11-12, 17:40)Milhouse Wrote: @radigast this is a scanning bug in Kodi.
Thanks for the tip, Milhouse!  As it turns out, this seems to be fixed in Krypton 17.6; those same database glitches appear to have been rectified.  Sage advice, as usual!
Reply
Question:   is there a way to translate the special:// paths (http://kodi.wiki/view/Special_protocol) to traditional filesystem paths in scripts like mklocal.py?  I enabled gif support and added a workaround for my use case, but it wouldn't be needed if there was an easy way to translate these paths.

For reference, i replaced line 522 from:
Code:
newsource = source
 
to
Code:

    if "special://profile/" in source:
       newsource = source.replace("special://profile/","/storage/.kodi/userdata/",1)
    else:
       newsource = source
Reply
Not really, because special://profile is dependent on both the current system and also your current profile in Kodi. And writing GIF files into userdata (and sometimes directly into Thumbnails) is just wrong, so the way GIF support has been implemented should be fixed making it less of a kludge.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Hi all

I think it hasn't been discussed here, but free keys for OMDB API are available since 11/02 : http://www.omdbapi.com

It's working fine with 'omdb.apikey=xxxx' in texturecache.cfg Smile
Kodi 18.1 running on
- Nvidia Shield 2017
- Beelink GT1 Ultimate (S912 / 3Gb RAM) @ CoreELEC 9.0.1
- Mini M8S II (S905X / 2 Gb RAM) @ CoreELEC 9.0.1
Reply
(2017-11-28, 09:15)Fredouye Wrote: Hi all

I think it hasn't been discussed here, but free keys for OMDB API are available since 11/02 : http://www.omdbapi.com

It's working fine with 'omdb.apikey=xxxx' in texturecache.cfg Smile

Unfortunately with a 1,000 daily query limit most users will blow through that just with movies, let alone episodes. A weekly limit of maybe 10K or 20K queries would be more usable. It's better than nothing, but the limit is too low to reliably update a moderately sized library even if you were to update movies on a Sunday and episodes a day or two later.

I forsee issues being reported from users in future asking why the imdb update starts to fail after "a few hundred" successful attempts (answer: API key limit exceeded). I'm probably going to have to code a check that detects when the API query is failing due to the limit and ends the run rather than continue throwing more queries at the site which will all fail.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
I've recently discovered that ever since implementing a new HTPC and switching to Kodi Krypton in July, AD (set to use local) doesn't download or cache extra artwork such as disc, clearart, banner, etc. This includes extra artwork that I manual put in place. I manually scrape posters, fanart and extrathumbs, which all show up.

Does the AD issue have to do with this post from Shedrock?

I was hoping that TC, would cache it, but no luck. This is what I run regularly from a DOS batch file, perhaps I'm not using the correct command:

texturecache.py P @logfile=C:\Temp\TC-P.log
texturecache.py R @logfile=C:\Temp\TC-R.log
texturecache.py R cache.ignore.types=.*\.gif,.*\.GIF
texturecache.py Xd
texturecache.py c video @logfile=C:\Temp\TC-C-video.log
texturecache.py c sets @logfile=C:\Temp\TC-C-sets.log

The 2nd issue is the it's reporting errors indicating that some *-thumbs.jpg aren't valid. I've confirmed that this is true for some media, but not for all that it shows. The ones that it claims are missing, but are there can be opened and viewed so they aren't corrupt. I was also under the impression that TC would create the missing ones.

Here are my TC Logs
HTPC: Dell Optiplex 7050 SFF i7-7700 quad-core, 3.6GHz, 16GB
NAS: Synology DS1813+ and DX513, Hybrid RAID (SHR) 48TB usable space
My Media Center | www.CaptainKen.us | www.YouTube.com/KenInGilbert
Reply
(2017-12-04, 16:39)CaptainKen Wrote: Does the AD issue have to do with this post from Shedrock?

Hard to say - Artwork Downloader has many issues due to lack of maintenance, so I'd suggest using Artwork Beef instead which is actively maintained.

(2017-12-04, 16:39)CaptainKen Wrote: I was hoping that TC, would cache it, but no luck. This is what I run regularly from a DOS batch file, perhaps I'm not using the correct command:

This script will only cache what is already in your media library. If Artwork Downloader isn't adding local artwork to your media library then there is nothing for Texture Cache to cache. You could use mklocal.py as an alternative to Artwork Downloader/Artwork Beef if you only want to load/associate local artwork.

(2017-12-04, 16:39)CaptainKen Wrote: The 2nd issue is the it's reporting errors indicating that some *-thumbs.jpg aren't valid. I've confirmed that this is true for some media, but not for all that it shows. The ones that it claims are missing, but are there can be opened and viewed so they aren't corrupt.
I can't tell anything from your logs as they are not complete - I think you have "logfile.verbose = no" which removes a lot of useful information.

It would help if you mentioned which files the script claims are missing, but you are sure exist. Otherwise I don't know which ones are being correctly flagged as missing.

If the script is not able to cache artwork then you should look in your kodi.log for any errors as it is Kodi that is not able to download the artwork.

(2017-12-04, 16:39)CaptainKen Wrote: I was also under the impression that TC would create the missing ones.
If you mean associate new local artwork with movies - no, that's the job of mklocal.py (or Artwork Downloader/Beef etc.).

If you mean replace an already cached version of an image then yes, the script will do that - so long as Kodi is able to download/access the artwork.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2017-11-08, 09:33)Milhouse Wrote: Add @logfile=tc.log to your command line, and run the command for the movie you've changed, eg "texturecache.py C movies avatar @cache.refresh=3 @logfile=tc.log" then send me the contents of tc.log. It should be retrieving the modification timestamp of poster.jpg and determining it is within the last 3 days.
 Sorry it took me a lifetime to get around to doing this. Here it is > https://pastebin.com/UVEmCkVw

The contents of the folder are: az.jpg (renamed from poster.jpg, still is the poster attached to the movie) / Ma Nuit Chez Maud.en.srt / Ma Nuit Chez Maud.mkv / movie.nfo / poster.jpg (new poster)
Reply
(2017-12-13, 03:24)firewater Wrote: The contents of the folder are: az.jpg (renamed from poster.jpg, still is the poster attached to the movie) / Ma Nuit Chez Maud.en.srt / Ma Nuit Chez Maud.mkv / movie.nfo / poster.jpg (new poster)

Based on your log:

1. The poster artwork attached to the movie is poster.jpg:
Code:
{
   "id":"libMovies",
   "jsonrpc":"2.0",
   "result":{
      "limits":{
         "end":1,
         "start":0,
         "total":1
      },
      "movies":[
         {
            "art":{
               "poster":"image://M:\Media\Video\Movie\Kodi\Ma Nuit Chez Maud\poster.jpg/"
            },
            "label":"My Night at Maud's",
            "movieid":7,
            "title":"My Night at Maud's"
         }
      ]
   }
}

2. The contents of your movie folder is:
Code:
{
   "id":"libDirectory",
   "jsonrpc":"2.0",
   "result":{
      "files":[
         {
            "file":"M:\\Media\\Video\\Movie\\Kodi\\Ma Nuit Chez Maud\\az.jpg",
            "filetype":"file",
            "label":"az.jpg",
            "lastmodified":"2017-11-04 06:12:42",
            "type":"unknown"
         },
         {
            "file":"M:\\Media\\Video\\Movie\\Kodi\\Ma Nuit Chez Maud\\Ma Nuit Chez Maud.en.srt",
            "filetype":"file",
            "label":"Ma Nuit Chez Maud.en.srt",
            "lastmodified":"2017-11-18 02:29:33",
            "type":"unknown"
         },
         {
            "file":"M:\\Media\\Video\\Movie\\Kodi\\Ma Nuit Chez Maud\\Ma Nuit Chez Maud.mkv",
            "filetype":"file",
            "label":"Ma Nuit Chez Maud.mkv",
            "lastmodified":"2017-11-02 03:03:24",
            "type":"unknown"
         },
         {
            "file":"M:\\Media\\Video\\Movie\\Kodi\\Ma Nuit Chez Maud\\movie.nfo",
            "filetype":"file",
            "label":"movie.nfo",
            "lastmodified":"2017-11-05 10:54:05",
            "type":"unknown"
         },
         {
            "file":"M:\\Media\\Video\\Movie\\Kodi\\Ma Nuit Chez Maud\\poster.jpg",
            "filetype":"file",
            "label":"poster.jpg",
            "lastmodified":"2017-12-12 22:16:42",
            "type":"unknown"
         }
      ],
      "limits":{
         "end":5,
         "start":0,
         "total":5
      }
   }
}

The timestamp on the poster.jpg file is within 3 days (script run at 2017-12-12 22:21:58)

The az.jpg file is no longer associated with the movie - it's just a random file in the movie folder.

3. The poster artwork, poster.jpg, is being deleted, and re-cached:
Code:
2017-12-12 22:22:00.009307:MainThread: QUEUE ITEM: {3, "movies", "poster", "My Night at Maud's", None, None, "M:\Media\Video\Movie\Kodi\Ma Nuit Chez Maud\poster.jpg", 7, "b/b97f155a.jpg", 7, False}
2017-12-12 22:22:00.010309:MainThread: Creating 1 download thread(s) for multi-access sites
2017-12-12 22:22:00.010309:Thread-1  : preparedl.JSON WEB REQUEST: [POST] [{"method": "Files.PrepareDownload", "params": {"path": "image://M%3a%5cMedia%5cVideo%5cMovie%5cKodi%5cMa%20Nuit%20Chez%20Maud%5cposter.jpg/"}, "jsonrpc": "2.0", "id": "preparedl"}]
2017-12-12 22:22:00.013310:Thread-1  : preparedl.RECEIVED WEB DATA: 200, OK, {"id":"preparedl","jsonrpc":"2.0","result":{"details":{"path":"image/image%3a%2f%2fM%253a%255cMedia%255cVideo%255cMovie%255cKodi%255cMa%2520Nuit%2520Chez%2520Maud%255cposter.jpg%2f"},"mode":"redirect","protocol":"http"}}
2017-12-12 22:22:00.013310:Thread-1  : Deleting old image from cache with id [7], cachedurl [b/b97f155a.jpg] for filename [M:\Media\Video\Movie\Kodi\Ma Nuit Chez Maud\poster.jpg]
2017-12-12 22:22:00.013310:Thread-1  : libTextures.JSON SOCKET REQUEST: [{"method": "Textures.RemoveTexture", "params": {"textureid": 7}, "jsonrpc": "2.0", "id": "libTextures"}]
2017-12-12 22:22:00.013310:Thread-1  : RPC connection established with IPv6
2017-12-12 22:22:00.136392:Thread-1  : libTextures.BUFFER RECEIVED (len 50)
2017-12-12 22:22:00.136392:Thread-1  : libTextures.PARSING JSON DATA: {"id":"libTextures","jsonrpc":"2.0","result":"OK"}
2017-12-12 22:22:00.136392:Thread-1  : libTextures.PARSING COMPLETE, elapsed time: 0.000000 seconds
2017-12-12 22:22:00.136392:Thread-1  : libTextures.FINISHED, elapsed time: 0.123082 seconds
2017-12-12 22:22:00.136392:Thread-1  : Proceeding with download of URL [/image/image%3a%2f%2fM%253a%255cMedia%255cVideo%255cMovie%255cKodi%255cMa%2520Nuit%2520Chez%2520Maud%255cposter.jpg%2f]
2017-12-12 22:22:00.136392:Thread-1  : loadImage.DIRECT WEB REQUEST: [GET], [/image/image%3a%2f%2fM%253a%255cMedia%255cVideo%255cMovie%255cKodi%255cMa%2520Nuit%2520Chez%2520Maud%255cposter.jpg%2f]
2017-12-12 22:22:00.390562:Thread-1  : loadImage.RECEIVED WEB DATA: 200, OK, <raw data>
2017-12-12 22:22:00.390562:Thread-1  : Successfully downloaded image with size [1024] bytes, attempts required [1], filename [M:\Media\Video\Movie\Kodi\Ma Nuit Chez Maud\poster.jpg]

Code:
.
              |    fanart   |    poster   |    TOTAL
--------------+-------------+-------------+-------------
Cached        |      -      |      1      |      1
Deleted       |      -      |      1      |      1
Duplicate     |      -      |      -      |      0
Error         |      -      |      -      |      0
Ignored       |      -      |      -      |      0
Skipped       |      -      |      -      |      0
Undefined     |      -      |      -      |      0
========================================================
TOTAL         |      -      |      2      |      2
Download Time |      -      | 00:00:00.00 | 00:00:00.00

4. Summary: Looks to be working fine to me. What do you think is the problem?
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
  • 1
  • 155
  • 156
  • 157(current)
  • 158
  • 159
  • 197

Logout Mark Read Team Forum Stats Members Help
[RELEASE] Texture Cache Maintenance utility17