• 1
  • 188
  • 189
  • 190(current)
  • 191
  • 192
  • 197
[RELEASE] Texture Cache Maintenance utility
(2020-06-02, 16:48)J_E_F_F Wrote: Artwork beef will not install on my LibreELEC (Milhouse): devel-20200527213954-#0527-ga7939cf (RPi2.arm), at least one of the dependencies will not install. (xbmc.python version 2.19.0 could not be satisfied)

Hmmm.... you'll need to contact the author (he's very active) or check the Artwork Beef thread as there may be some info there (maybe a new Matrix repo is needed?) I've never used or installed Artwork Beef myself...
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2020-06-02, 17:24)Milhouse Wrote:
(2020-06-02, 11:59)Raytestrak Wrote: When using the c parameter, does it first check for local fanarts / thumbs before looking online?

No, the path for each artwork item is in your media library - this may be the path of a local file, or it may be a remote artwork url - so there's no need for the script to search, it only has to ask Kodi to download the relevant path or url based on the location of the artwork you have scraped into your media library. Of course if the artwork is remote, and the artwork no longer exists at the configured url, then Kodi won't be able to download anything, and you will need to scrape a new url for the poster.
(2020-06-02, 11:59)Raytestrak Wrote: Also, but this probably has nothing to do with this utility: if I remove stuff from my library, I (sometimes) still see references in the path and/or art tables in the database.

That's what the P option is for - if you remove an item (movie, tvshow, album, etc.) from your media library then the texture cache on each of your Kodi clients will continue to store the old cached artwork (posters, fanart, cover art etc. etc.), which is just wasting space. The P (prune) option cross references your media library artwork with the cached artwork and removes the cached artwork that is not present (or no longer present) in your media library.

Could it be a 'problem' with the clean library part of Kodi then? I'm assuming the vclean parameter calls the default video library cleanup? I just deleted a season of a certain show (the folder is definitely no longer on disk and I don't see the season in Kodi anymore), but in in the path table my database I still see an entry for the path of the deleted season and in the art table I still see antries for the poster of the deleted season and the seasson before that.

Edit: just did a check, I finished season 4 of aanother show. Ran cleanup from with Kodi and pruned my texturecache afterwards. I still see entries to the folder in the path table and art tables. I even see entries for all the episode thumbs of season 3, not for season 4. Artwork for season 3 and 4 were pruned though. Not related to texturecache, but interesting nonetheless.
CoreElec on a tn95 (s905x). Onkyo NR-656. Canton Movie CD-1000. LG 55B6V.

If it ain't broke: break it, fix it, repeat
Reply
(2020-06-02, 12:16)FXB78 Wrote:
(2020-06-02, 12:10)peque Wrote:
(2020-06-02, 09:19)peque Wrote: This is why fanart.tv requests keeps returning 503 error every now and then:

https://twitter.com/fanartdottv/status/1...6035563520

The question is: could the script be modified somehow to respect this max speed of 3 requests a second? I assumed that 1 thread max could slow down the process... but I see it's not the case. Could it just add a config option to limit requests at 1/second for example?

I see it was an error to delete all my thumbnails... Sad 
Ok. I know nothing about python, but I managed to find the place to add a "time.sleep(1)" and thus control download speed... BUT, I've tested some sleep values, and all of them end with a "ban" from fanart.tv. If I cancel script execution and restart it, it works again for some downloads, so I assume I should find a way to close and open the thread every few seconds... or... uglyer but also possible, I should find a way to automate that script "cancel/restart"... 
Which thumbnails are you trying to import? Did you try disabling fanart.tv and using an alternative. For Music you can use Universal Scraper, not sure about Video content but isn't there another option like tvdb or tmdb? 

That was an option, but I'm still in my way to learn about Kodi, and once "scraped" I don't know if disabling fanart.tv would do anything...

Finally I found a way to automatically cancel/restart script everytime I and an error (running it through a vbscript in SecureCRT), so my problem is solved.

Thanks anyway.
Reply
(2020-06-02, 17:10)Milhouse Wrote:
(2020-06-02, 09:19)peque Wrote:
(2020-06-02, 05:05)Milhouse Wrote: Unfortunately not - this is another of the problems when using remote content. Sad  

This is why fanart.tv requests keeps returning 503 error every now and then:

https://twitter.com/fanartdottv/status/1...6035563520

The question is: could the script be modified somehow to respect this max speed of 3 requests a second? I assumed that 1 thread max could slow down the process... but I see it's not the case. Could it just add a config option to limit requests at 1/second for example?

I see it was an error to delete all my thumbnails... Sad  

I'll have a think, but yes a 1/second limit option seems like the quickest fix. Since every artwork hosting web site is likely to impose different rate limiting rules the ideal would be for the script to be configured with those rules but that isn't likely to happen (at least, I have no intention of implementing that) as it would become a maintenance chore (as rules change etc.). Hopefully the fanart rate limit is only temporary as I imagine that's going to screw up Kodi as well.  

As I said somewhere below that, I found a way to add a second sleep time between requests to your script... but fanart.tv still banned me every few requests. I had to cancel the script and restart it, so I could download artwork again.

You're probably right. It has no much sense to implement site by site rules inside the script. May be you could implement some config options so users can "adapt" script behaviour to each site limitation. "cadence" and "reconnect" options maybe? I don't know. You're the master here. Smile

Thanks a lot for your support.
Reply
(2020-06-03, 09:27)Raytestrak Wrote: Could it be a 'problem' with the clean library part of Kodi then? I'm assuming the vclean parameter calls the default video library cleanup? I just deleted a season of a certain show (the folder is definitely no longer on disk and I don't see the season in Kodi anymore), but in in the path table my database I still see an entry for the path of the deleted season and in the art table I still see antries for the poster of the deleted season and the seasson before that.

Yes, vclean just executes the same video library cleanup process that you would run from the Kodi GUI (the only difference being the script version has control over whether the clean runs "silently" without GUI updates or not).

It seems that the tables you are referring to are those from your MyVideos database (the Textures13.db also has a path table)? If there are artefacts that remain in the MyVideos database after removing a season and running the video cleanup, then you might want to open an issue on the Kodi github where it can be investigated/confirmed as a bug.

(2020-06-03, 09:27)Raytestrak Wrote: Edit: just did a check, I finished season 4 of aanother show. Ran cleanup from with Kodi and pruned my texturecache afterwards. I still see entries to the folder in the path table and art tables. I even see entries for all the episode thumbs of season 3, not for season 4. Artwork for season 3 and 4 were pruned though. Not related to texturecache, but interesting nonetheless.

Yes, the metadata for the tv show would reflect that it no longer has a Season 4 so as far as the texturecache.py script is concerned that season HAS been deleted (and the cached artwork successfully pruned away). The only question is why references to the deleted season path and artwork remain in the video library when in theory this should have been removed (either immediately when removing the season, or eventually following a library clean) - maybe something is broken there.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2020-06-03, 10:50)peque Wrote: As I said somewhere below that, I found a way to add a second sleep time between requests to your script... but fanart.tv still banned me every few requests. I had to cancel the script and restart it, so I could download artwork again.

You're probably right. It has no much sense to implement site by site rules inside the script. May be you could implement some config options so users can "adapt" script behaviour to each site limitation. "cadence" and "reconnect" options maybe? I don't know. You're the master here. Smile

Thanks a lot for your support.

I haven't had time to look at this yet, and assuming the fanart.tv restriction is only temporary I may never get around to it (what they have done is going to screw up Kodi as well, so it's not really a practical solution for the majority of their users such as Kodi).

It's interesting that adding time.sleep(1.0) to force 1 request per second wasn't successful, although I have an idea why that could be. Can you re-try your modified script with the option @Download.prime=no as that will stop the script checking if the artwork exists before it asks kodi to perform the download ("priming" the download seemed to help improve reliability of imdb.com artwork downloads, although it could be a placebo...). Also, the script will ask kodi to download artwork up to 3 times by default, so this could also be blowing the fanart.tv rate limit (it depends where you added the delay, I suppose - between artwork items, or between web requests etc.). You could try @Download.retry=0 to avoid retrying failed downloads, and (in combination with the previous option) see if that helps avoid restarting the script (I've no idea why restarting the script helps, other than to introduce a sufficiently long enough delay to avoid immediately hitting the fanart.tv rate limits again).

The problem with implementing per-site rate limiting is really to do with the multi-threading as that makes it all a bit more complicated - per-site limits could be implemented, but it may mean enforcing a single thread just to simplify things. And when sites do implement rate limits they don't usually make public what those limits actually are - fanart.tv is quite unusual in providing the limits they have implemented. I'm fairly sure imdb.com implement some sort of rate limiting, but the exact details are unknown (at least to me). So configuring per-site limits would just be a guessing game, and crossing fingers, and I'm really not sure that there is a good solution for this issue other than the sledgehammer "1 request per second using a single thread".

Edit: Those options are @ download.prime / @ download.retry (lower case d - use the @ on the command line, leave it out when adding to texturecache.cfg). Unfortunately there's a @Download user (capital D) and the forum software in it's infinite wisdom decides to rewrite the message. Bah.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Ok. I though something similar (more than one requests asked to Kodi).

Unfortunately now I've got cached all my artwork and I wouldn't like to contribute to stress fanart.tv site again. But I'll test that option when I have some more artwork to cache.

Thank you so much.
Reply
(2020-06-05, 09:02)peque Wrote: Ok. I though something similar (more than one requests asked to Kodi).

Unfortunately now I've got cached all my artwork and I wouldn't like to contribute to stress fanart.tv site again. But I'll test that option when I have some more artwork to cache.

Thank you so much.
You could get your artwork on your media folders and set cache from local artwork
Image Image
Reply
[mention]Milhouse [/mention] thank you for the great tool which is texturecache script. I’m mostly using it to clean and update my library. And I’m more than happy to have it. There’s one thing that bother me for some time. Is it normal that there’s two lines indicating new episode or movie? Or maybe it’s by design? To illustrate what I mean, here is a screenshot form my last vacan.

Image

Thank you!
Reply
(2020-06-06, 15:08)JeDaj Wrote: There’s one thing that bother me for some time. Is it normal that there’s two lines indicating new episode or movie? Or maybe it’s by design? To illustrate what I mean, here is a screenshot form my last vacan.

Image

Thank you!

Can you upload a texturecache log (created using @logfile=/tmp/tc.log) for a vscan when "duplicate" episodes (or movies) are processed?

I'm pretty sure this is a bug in older versions of kodi, and I don't see it with Kodi 19 (when using MySQL, in case that's another variable). The episodes/movies are logged by the script in response to JSON-RPC notifications received from the Kodi client, so if an episode (or movie) is logged twice that would mean 2 notifications were received from/emitted by the kodi client when normally only one notification should have been received.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2020-06-10, 00:03)Milhouse Wrote:
(2020-06-06, 15:08)JeDaj Wrote: There’s one thing that bother me for some time. Is it normal that there’s two lines indicating new episode or movie? Or maybe it’s by design? To illustrate what I mean, here is a screenshot form my last vacan.

Image

Thank you!

Can you upload a texturecache log (created using @logfile=/tmp/tc.log) for a vscan when "duplicate" episodes (or movies) are processed?

I'm pretty sure this is a bug in older versions of kodi, and I don't see it with Kodi 19 (when using MySQL, in case that's another variable). The episodes/movies are logged by the script in response to JSON-RPC notifications received from the Kodi client, so if an episode (or movie) is logged twice that would mean 2 notifications were received from/emitted by the kodi client when normally only one notification should have been received.

Hi. I'm sorry for the wait. 
https://pastebin.com/0MWezJB5

My setup:
RPi3, OSMC, shared SQL database on my Synology NAS. My texture database is hasted locally on RPi. If you need some other informations jus let me know.

Here is my Kodi version from OSMC
Code:

osmc@salon:~$ grep Kodi .kodi/temp/kodi.log | head -3
2020-06-11 20:46:31.146 T:1915446832  NOTICE: Starting Kodi (18.7). Platform: Linux ARM (Thumb) 32-bit
2020-06-11 20:46:31.146 T:1915446832  NOTICE: Using Release Kodi x32 build (version for Raspberry Pi)
2020-06-11 20:46:31.146 T:1915446832  NOTICE: Kodi compiled 2020-06-03 by GCC 6.3.0 for Linux ARM (Thumb) 32-bit version 4.9.110 (264558)
osmc@salon:~$

Cheers! And Thank You!
Reply
Hello, 

I'm trying to reinstall this script on my new PC.  I didn't get the chance to move the files from my old PC due to hard drive failure.  So I reinstalled Python (version 3.8.3) and downloaded the script files from Github as directed in the 1st post.  When I try to run a test (run script without parameters) to see if the script is working, I get the error "Requested Python version () is not installed".  From what I've read in this thread (lots of reading), this should work with both V2 and V3 of python.  So the error is throwing me off.  Any ideas as to why I am getting this error?

Thank you for any help in advance,

Mychale
Reply
For the attached log, you would have seen the following:
text:

Rescanning library...
Updating library: New episodeid 7349 [Marvel's Agents of S.H.I.E.L.D. S07E02 (Know Your Onions)]
Updating library: New episodeid 7349 [Marvel's Agents of S.H.I.E.L.D. S07E02 (Know Your Onions)]
Updating library: New episodeid 7350 [Blindspot S05E04 (And My Axe!)]
Updating library: New episodeid 7350 [Blindspot S05E04 (And My Axe!)]

For episodeid 7349, there are 2 update notifications output by Kodi (and present in the log):

text:

2020-06-11 20:50:37.442386:MainThread: libRescan.PARSING JSON DATA: {"jsonrpc":"2.0","method":"VideoLibrary.OnUpdate","params":{"data":{"item":{"id":7349,"type":"episode"},"transaction":true},"sender":"xbmc"}}
2020-06-11 20:50:37.500667:MainThread: libRescan.PARSING JSON DATA: {"jsonrpc":"2.0","method":"VideoLibrary.OnUpdate","params":{"data":{"added":true,"item":{"id":7349,"type":"episode"},"transaction":true},"sender":"xbmc"}}

which confirms that Kodi 18 is sending a second VideoLibrary.OnUpdate notification - the only difference between the 2 notifications is that the second notification includes "added":true. This is meant to represent that the notification corresponds with a "newly added item", which is clearly not the case.

This doesn't happen with Kodi 19 on my system (MySQL) - only a single VideoLibrary.OnUpdate notification is output by Kodi:
text:

2020-06-17 18:19:18.044178:MainThread: libRescan.PARSING JSON DATA: {"jsonrpc":"2.0","method":"VideoLibrary.OnUpdate","params":{"data":{"added":true,"item":{"id":32491,"type":"episode"},"transaction":true},"sender":"xbmc"}}

which is correct, as per the intention of the "added" property.

So my suspicion is that this is/was a bug in Kodi 18, but has been fixed in Kodi 19 (can't find the actual fix - it may have been fixed accidentally...).

I could possibly add a fix to the script that ignores one of the notifications for pre-Kodi 19 clients but that's a bit of an ugly fix for what is a cosmetic issue caused by the client and that is (hopefully) already fixed.

Are you able to test with Kodi 19? Be aware of the Python3 change, so don't test on a system you care about! Smile
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2020-06-17, 12:45)Mychale Wrote: When I try to run a test (run script without parameters) to see if the script is working, I get the error "Requested Python version () is not installed". 

How are you running the script? I'm not familiar with that error - it's not an error that is coded into the script but may be being generated by something else?

If the script runs at all then Python is working, and there's nothing in the script that checks for a specific Python version (other than detecting Python2 or Python3).
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2020-06-17, 19:46)Milhouse Wrote: Are you able to test with Kodi 19? Be aware of the Python3 change, so don't test on a system you care about! Smile

Well, I don’t think I have the strength to test osmc with Kodi 19 on my Pi. I will just wait for an official update to osmc from Sam when Kodi 19 is released. It’s not (this two times output) a big deal for me. Just a strange thing that I wanted to mention here. I can live with that. If it was adding those episodes twice to my library than I would like to fix that. Right now I will just wait. Thanks for looking into that anyway Smile

I’m trying to recall when this two time output started for me and I think it was always like that. So it must have been since Kodi 16 or maybe even earlier. I don’t remember when I started to use texturecache script.

When Kodi 19 will be released I’ll get back here to report if that has changed Smile

Cheers!
Reply
  • 1
  • 188
  • 189
  • 190(current)
  • 191
  • 192
  • 197

Logout Mark Read Team Forum Stats Members Help
[RELEASE] Texture Cache Maintenance utility17