[RELEASE] Texture Cache Maintenance utility
(2020-06-03, 10:50)peque Wrote: As I said somewhere below that, I found a way to add a second sleep time between requests to your script... but fanart.tv still banned me every few requests. I had to cancel the script and restart it, so I could download artwork again.

You're probably right. It has no much sense to implement site by site rules inside the script. May be you could implement some config options so users can "adapt" script behaviour to each site limitation. "cadence" and "reconnect" options maybe? I don't know. You're the master here. Smile

Thanks a lot for your support.

I haven't had time to look at this yet, and assuming the fanart.tv restriction is only temporary I may never get around to it (what they have done is going to screw up Kodi as well, so it's not really a practical solution for the majority of their users such as Kodi).

It's interesting that adding time.sleep(1.0) to force 1 request per second wasn't successful, although I have an idea why that could be. Can you re-try your modified script with the option @Download.prime=no as that will stop the script checking if the artwork exists before it asks kodi to perform the download ("priming" the download seemed to help improve reliability of imdb.com artwork downloads, although it could be a placebo...). Also, the script will ask kodi to download artwork up to 3 times by default, so this could also be blowing the fanart.tv rate limit (it depends where you added the delay, I suppose - between artwork items, or between web requests etc.). You could try @Download.retry=0 to avoid retrying failed downloads, and (in combination with the previous option) see if that helps avoid restarting the script (I've no idea why restarting the script helps, other than to introduce a sufficiently long enough delay to avoid immediately hitting the fanart.tv rate limits again).

The problem with implementing per-site rate limiting is really to do with the multi-threading as that makes it all a bit more complicated - per-site limits could be implemented, but it may mean enforcing a single thread just to simplify things. And when sites do implement rate limits they don't usually make public what those limits actually are - fanart.tv is quite unusual in providing the limits they have implemented. I'm fairly sure imdb.com implement some sort of rate limiting, but the exact details are unknown (at least to me). So configuring per-site limits would just be a guessing game, and crossing fingers, and I'm really not sure that there is a good solution for this issue other than the sledgehammer "1 request per second using a single thread".

Edit: Those options are @ download.prime / @ download.retry (lower case d - use the @ on the command line, leave it out when adding to texturecache.cfg). Unfortunately there's a @Download user (capital D) and the forum software in it's infinite wisdom decides to rewrite the message. Bah.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply


Messages In This Thread
Crash on Gotham on OS X - by desepticon - 2014-05-29, 17:57
Cleaning - by AleisterHH - 2018-05-28, 22:03
RE: Cleaning - by Milhouse - 2018-05-28, 22:16
qax genre not updated - by Just-Me_A-User - 2018-06-12, 22:06
RE: qax genre not updated - by Milhouse - 2018-06-12, 23:40
RE: [RELEASE] Texture Cache Maintenance utility - by Milhouse - 2020-06-05, 03:38
Logout Mark Read Team Forum Stats Members Help
[RELEASE] Texture Cache Maintenance utility17