• 1
  • 181
  • 182
  • 183(current)
  • 184
  • 185
  • 197
[RELEASE] Texture Cache Maintenance utility
(2020-02-26, 21:06)Milhouse Wrote: In addition to the debug kodi log please also upload the matching texturecache log so that activity can be correlated.

Kodi Debug Log here

texturecache log here

In the kodi debug log I see the following lines that jump out to me, the last two lines repeat. I've removed the last lines of the log where the screensaver starts as that made the log too long to hastebin

bash:
2020-02-26 18:26:22.664 T:139927058704128  NOTICE: VideoInfoScanner: Starting scan ..
2020-02-26 18:26:22.665 T:139927058704128   DEBUG: CAddonSettings[metadata.themoviedb.org]: loading setting definitions
2020-02-26 18:26:22.665 T:139927058704128   DEBUG: CAddonSettings[metadata.themoviedb.org]: trying to load setting definitions from old format...
2020-02-26 18:26:22.666 T:139927058704128   DEBUG: CAddonSettings[metadata.themoviedb.org]: loading setting values
2020-02-26 18:26:24.583 T:139927058704128   DEBUG: VideoInfoScanner: Scanning dir 'smb://192.168.0.2/HD Films/' as not in the database
2020-02-26 18:26:24.583 T:139927058704128   DEBUG: CAddonSettings[metadata.themoviedb.org]: loading setting definitions
2020-02-26 18:26:24.583 T:139927058704128   DEBUG: CAddonSettings[metadata.themoviedb.org]: trying to load setting definitions from old format...

......any thoughts are appreciated, I'm at a loss here.
Reply
Can you zip the entire kodi log? Just zip both logs together and upload a single archive file. The texturecache script log starts at 18:32:44, your kodi.log ends at 18:28:09, so you've cut off all the useful/relevant parts of the kodi log.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2020-02-27, 08:34)Milhouse Wrote: Can you zip the entire kodi log? Just zip both logs together and upload a single archive file. The texturecache script log starts at 18:32:44, your kodi.log ends at 18:28:09, so you've cut off all the useful/relevant parts of the kodi log.

Thanks @Milhouse, this must be like pullingteeth, sorry......

........the zip of both is here
Reply
@littlejeem: I've had a look, and there's a problem with the webserver requests as all pre-caching download requests are failing.

It's as if Kodi doesn't have access to the artwork (but it's not logging anything useful).

This is the Files.PrepareDownload method that is failing - this must succeed before any artwork can be downloaded (and thus cached) by Kodi, but it fails every item of artwork:
text:

2020-02-26 18:32:46.763537:Thread-1 : preparedl.JSON WEB REQUEST: [POST] [{"jsonrpc": "2.0", "params": {"path": "image://smb%3a%2f%2f192.168.0.2%2fSD%20Films%2f(500)%20Days%20of%20Summer%20(2009)%2f(500)%20Days%20of%20Summer.DVD-fanart.jpg/"}, "method": "Files.PrepareDownload", "id": "preparedl"}]
2020-02-26 18:32:47.231921:Thread-1 : preparedl.RECEIVED WEB DATA: 200, OK, {"error":{"code":-32602,"data":{"details":null},"message":"Invalid params."},"id":"preparedl","jsonrpc":"2.0"}

Not sure if it's available in Kodi 18, but try enabling "JSON-RPC", "Webserver" and "Database" component debug logging then run the following test:
text:

texturecache.py c movies "Days of Summer"

and upload both kodi and texturecache logs. Earliest I can look at it will be Sunday.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2020-02-28, 07:02)Milhouse Wrote: @littlejeem: I've had a look, and there's a problem with the webserver requests as all pre-caching download requests are failing.

It's as if Kodi doesn't have access to the artwork (but it's not logging anything useful).

This is the Files.PrepareDownload method that is failing - this must succeed before any artwork can be downloaded (and thus cached) by Kodi, but it fails every item of artwork:
text:

2020-02-26 18:32:46.763537:Thread-1 : preparedl.JSON WEB REQUEST: [POST] [{"jsonrpc": "2.0", "params": {"path": "image://smb%3a%2f%2f192.168.0.2%2fSD%20Films%2f(500)%20Days%20of%20Summer%20(2009)%2f(500)%20Days%20of%20Summer.DVD-fanart.jpg/"}, "method": "Files.PrepareDownload", "id": "preparedl"}]
2020-02-26 18:32:47.231921:Thread-1 : preparedl.RECEIVED WEB DATA: 200, OK, {"error":{"code":-32602,"data":{"details":null},"message":"Invalid params."},"id":"preparedl","jsonrpc":"2.0"}

Not sure if it's available in Kodi 18, but try enabling "JSON-RPC", "Webserver" and "Database" component debug logging then run the following test:
text:

texturecache.py c movies "Days of Summer"

and upload both kodi and texturecache logs. Earliest I can look at it will be Sunday.

thanks so much for looking at this @Milhouse.

I've gone into kodi->system->logging->Turned on Debug, Selected "specifiy component-specific logging"->"JSON-RPC", "Webserver" and "Database", clicked "OK".

Ran script command as requested and zipped both logs here

I also ran a manual refresh of the movie "Batman Returns" from within kodi which works and the log is here in case it helps.
Reply
@littlejeem I may have to admit defeat, here. I really appreciate all the logs, but unfortunately Kodi is telling me _nothing_ about why it can't process the artwork it has been requested to download - it just returns a totally irrelevant error message and doesn't log anything useful about the artwork file it is being instructed to access.

Are you able to re-cache the Batman artwork? Based on the log in your second link it appears you actually scanned "The Dark Knight", and are currently using remote artwork for the poster and fanart, so does texturecache.py C movies "The Dark Knight" successfully re-cache those posters and fanart?

I'm wondering if your media-library has become messed up which is now causing problems for Kodi (such a shame it's keeping the details of those problems to itself). A full rescan may be the best solution (you can use texturecache.py to backup then restore watched statuses!)
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2020-03-01, 17:29)Milhouse Wrote: texturecache.py C movies "The Dark Knight"

Hey @Milhouse hugely appreciate you taking the time in the first place........seems I'm becoming an expert in finding problems for experts....apologies!

running 
bash:
texturecache.py C movies "The Dark Knight"

gives something different...

bash:
jlivin25@kodibuntu:~$ /home/jlivin25/bin/myscripts/kodi_texturecache/texturecache.py C movies "The Dark Knight"
                                                                                  
Caching artwork: 0 items remaining of 4 (qs: 0, qm: 0), 2 errors, 0 threads active (02.00 downloads per second, ETA: 00:00:00)

The following items could not be downloaded:

[set.fanart] [The Dark Knight                         ] smb://192.168.0.2/HD Films/Dark Knight, The (2008)/The Dark Knight (2008)-fanart.jpg
[set.poster] [The Dark Knight                         ] smb://192.168.0.2/HD Films/Dark Knight, The (2008)/The Dark Knight (2008)-poster.jpg

Cache pre-load activity summary for "movies", filtered by "The Dark Knight":

              |    fanart   |    poster   |  set.fanart |  set.poster |    TOTAL    
--------------+-------------+-------------+-------------+-------------+-------------
Cached        |      1      |      1      |      -      |      -      |      2      
Deleted       |      1      |      1      |      -      |      -      |      2      
Duplicate     |      -      |      -      |      -      |      -      |      0      
Error         |      -      |      -      |      1      |      1      |      2      
Ignored       |      -      |      -      |      -      |      -      |      0      
Skipped       |      -      |      -      |      -      |      -      |      0      
Undefined     |      -      |      -      |      -      |      -      |      0      
====================================================================================
TOTAL         |      2      |      2      |      1      |      1      |      6      
Download Time | 00:00:00.73 | 00:00:00.77 | 00:00:01.53 | 00:00:01.52 | 00:00:02.29 

  Threads Used: 2
   Min/Avg/Max: 01.12 / 01.74 / 02.00 downloads per second
   Min/Avg/Max: 00.74 / 01.14 / 01.53 seconds per download

       Loading: 00:00:00.05
       Parsing: 00:00:00.00
     Comparing: 00:00:00.04
   Downloading: 00:00:02.30
 TOTAL RUNTIME: 00:00:02.39

In case of any use, the logs are here

I don't see any mileage in you putting in more time if it's not obvious that I'm screwing up! Smile

I'll back up the watch status as you suggest, make a note of the most recently added TV and Movie shows, delete the source from kodi, drop the database in mysql and start again.......Big Grin

........can't help thinking that somethings messed up in the shared sql database and that the paths are not what they should be.

To you, as to all the Kodi devs, I say again....thank you! Rofl
Reply
@Milhouse 

A few problems that I can't get it solved, hope you can help. Beats scraping all my anime all over again, rip. I delete Thumbnails folder AND Textures13.db to start fresh. All the database and source files are all same and location between devices.
  1. All of my anime have custom nfo, png, and -thumb.jpg. Courtesy of TMM. Means S01E01.mkv, S01E01.nfo, S01E01-thumb.jpg.
  2. Having considered local, 
    Code:
    texturecache.py c tvshows
    will not work.
  3. Tried 
    Code:
    texturecache.py qax tvshows "Beyond" @qa.nfo.refresh=0
    , this will NOT work without nfo and it takes too long, also bad as the database is already perfect.
  4. All I need are the jpg and png for all the tvshows. After they have been generated to the Thumbnails folder, I will delete nfo/jpg to save space. Its kinda counter intuitive to have both thumbnails folders and -thumb.jpg, double storage. Bad for my firesticks.
  5. I can generate them to thumbnails folder if i painstakingly scroll through the gui. Hope you have a command to imitate this.
  6. Can we specify the show that we wanna texturecache.py c. e.g. 
    Code:
    texturecache.py c tvshows "Batman Beyond" 
    will scrape both batman beyond AND batman animated series AND new batman adventures. the quotes are not doing its thing here.
 
Please help.
Reply
1. OK

2. It won't work if you run that command on a PC while trying to update a remote client such as a Firestick - you also need to tell the script the IP address of the remote client ie. Firestick, for example texturecache.py c tvshows @kodi.host=192.168.0.101 (assuming you have a Firestick listening on IP address 192.168.0.101).

3. This will potentially refresh any tvshows that include the characters "Beyond" - obviously it will only work correctly if the .nfo files are present as @qax.nfo.refresh causes it to query the modification timestamp of your .nfo files. And if it does refresh a tvshow (or or episode) then Kodi will need the .nfo files - "refresh" in this context is "delete tv show or episode, and rescan the same").

4. Not something I'd advise as you'll likely regret it in future, but OK

5. Yes, that's what this script is designed to do on your behalf.

6. Yes - your example would cache artwork for any tvshows that includes the name "Batman Beyond". Note that the c command will cache artwork that is not yet already cached, while the C command will forcibly re-cache artwork (remove existing cached artwork then re-cache it - useful if the existing cached artwork is incorrect).

Obviously if you are deleting your source artwork in #5 then using C would be highly inadvisable. Should you ever lose the Textures13 database or Thumbnails folder on one of your clients then you'd obviously be pretty much stuffed if you've already deleted the source artwork.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2020-03-12, 21:33)Milhouse Wrote: 5. Yes, that's what this script is designed to do on your behalf.
 
Thanks for answering the rest. But 5 is most important to me. I'd like to scrape the JPG/PNG only.

The horror of scraping anime shows. Most scrapers do not have thumb.jpg on every episodes. I diy thumbnails using FFMPEG script. What command do I just scrape the thumb.jpg> Thumbnails folder. That's all I'm asking, thanks!
Reply
(2020-03-13, 07:41)clockwerk Wrote: What command do I just scrape the thumb.jpg> Thumbnails folder. That's all I'm asking, thanks!

text:

texturecache.py c tvshows @cache.artwork=thumb
should work, I think.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2020-03-13, 21:05)Milhouse Wrote:
text:

texturecache.py c tvshows @cache.artwork=thumb
should work, I think. 

This is exactly what I'm looking for! I added ,poster to test and it works!
text:

texturecache.py c tvshows @cache.artwork=thumb,poster
 

Also regarding my question 6., you didn't get what I was asking.
Its mainly, How do you separate "Batman Beyond" and "Batman Animated Series" and "Batman Adventures". More like precise exact pinpoint tvshow. like
ONLY Batman Beyond,
text:
texturecache.py c tvshows "Batman Beyond" @cache.artwork=thumb,poster

ONLY Batman Animated Series,
text:
texturecache.py c tvshows "Batman Animated Series" @cache.artwork=thumb,poster

ONLY Batman Adventures,
text:
texturecache.py c tvshows "Batman Adventures" @cache.artwork=thumb,poster

Better yet, can you choose a specific source path location to be cached? E.g.

text:
texturecache.py c tvshows "/storage/emulated/0/Anime" @cache.artwork=thumb,poster

or

text:
texturecache.py c tvshows "source=Anime" @cache.artwork=thumb,poster

as per sources.xml
text:
<sources>
    <video>
        <default pathversion="1"></default>
        <source>
            <name>Anime</name>
            <path pathversion="1">/storage/emulated/0/Anime/</path>
            <allowsharing>true</allowsharing>
        </source>

</sources>
I've tried this, doesn't work.
Reply
(2020-03-14, 09:40)clockwerk Wrote: Its mainly, How do you separate "Batman Beyond" and "Batman Animated Series" and "Batman Adventures". More like precise exact pinpoint tvshow. like

The default filter is title and the default filter operator is contains, so "Batman Beyond" would match any tvshow title that contains the phrase "Batman Beyond" (so it shouldn't match "Batman Animated Series" or "Batman Adventures"), whereas texturecache.py c tvshows "Batman" would match all 3 shows.

However if you want to be precise, you can try:
text:

texturecache.py c tvshows @cache.artwork=thumb,poster @filter=title @filter.operator=is "Batman Beyond"

and now it will only match tvshows where the title *exactly* matches "Batman Beyond".

Assuming your Anime tvshows have the "Anime" genre then you could try matching by genre:
text:

texturecache.py c tvshows @cache.artwork=thumb,poster @filter=genre "Anime"

You can view your tvshow genres with:
text:

texturecache.py Jd tvshows @query.seasons=no @extrajson.tvshows.tvshow=genre
However it's only possible to match a single genre - the script doesn't currently support combinations of genre (ie. not possible to match both "Horror" and "Comedy").
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2020-03-14, 09:40)clockwerk Wrote: Better yet, can you choose a specific source path location to be cached? E.g.

Not really possible - a tvshow doesn't always have a single path, you could have season1 on one share, and the remaining seasons on another disk.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2020-03-14, 20:41)Milhouse Wrote:
(2020-03-14, 09:40)clockwerk Wrote: Better yet, can you choose a specific source path location to be cached? E.g.

Not really possible - a tvshow doesn't always have a single path, you could have season1 on one share, and the remaining seasons on another disk. 

What about specific tvshows? Like the example i gave above.
Reply
  • 1
  • 181
  • 182
  • 183(current)
  • 184
  • 185
  • 197

Logout Mark Read Team Forum Stats Members Help
[RELEASE] Texture Cache Maintenance utility17