18 Nov 2019- TVDB are still in the process of fixing a number of bugs that have broken the TVDB scraper and any add-on that relies on TVDB data. TVDB are still working to rectify the problems.

  •   
  • 1
  • 116
  • 117
  • 118(current)
  • 119
  • 120
  • 179
  •   
[RELEASE] Texture Cache Maintenance utility
(2015-11-19, 09:12)MANswers Wrote: what about the old thumbnails then?

You said you wanted to delete them - so delete them, they're in the Thumbnails folder. Just remember to delete Textures13.db also.

(2015-11-19, 09:12)MANswers Wrote: also, i forgot to mention earlier, when i go to http://localhost:8080/ amazon echo's ha bridge webpage opens up. I have configured amazon echo HA bridge on this port. is it cos of this?

Yes.

You'll need to configure Kodi so that the webserver is listening on a port other than 8080 (and avoid port 80 if you're on Windows 10, as Microsoft already run something else on port 80), then add "webserver.port = <port>" to texturecache.cfg so that the script knows which port Kodi is now using.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2015-11-18, 22:46)Milhouse Wrote: Sorry no idea about Android.

You can always run the script on the Windows PC and provide the IP address of your Android device (@kodi.host), that will work for a lot of the options.

working great, many thanks
Reply
(2015-11-18, 22:59)Milhouse Wrote:
(2015-11-18, 18:13)Nepomuk Wrote: I found "allow.recacheall = " in this thread here but sorry I can't remember for what it is. Can you explane it for me again please?

It allows you to forcibly recache all items for a particular media type (ie. "C movies") which isn't something you would normally want to do, or might only do by mistake, but if you know what you're doing then you can enable this option.

Reading the source code will probably answer a lot of questions.

Yes, for sure you are right. And yes I can read the source code but I understand nothing really nothing. Confused

Quote:
(2015-11-18, 18:13)Nepomuk Wrote: I have to run "texturecache.py Xd" on my PC in my home office. This PC have a data rights and connected by a 1G Lan network. Is that nevertheless OK?

No idea what the question is, sorry.
This answer or question refers to your post here:
Quote:I'm assuming you're running the script on the Kodi client, as "texturecache.py Xd" will only work if it has direct access to the Thumbnails folder - all the other commands you've listed should work remotely across the network, if you add "kodi.host=<ip_address>" where <ip_address> is the ip address of the Kodi client you wish to update.
And "Xd" had no log output that makes me uncertain.

Quote:
(2015-11-18, 21:04)Nepomuk Wrote: I don't understand why so much movie items are ignored or skipped. Huh
Why are the duplicate arts in tv-show not deleted automatically- because I see it at the second run that this 1749 files are available.

There's a link at the bottom of the first post. Or read the source code.
Ok, I will study this link.
Thanks for your help and your patience and your great script. Love
Reply
(2015-11-19, 19:51)Nepomuk Wrote: And "Xd" had no log output that makes me uncertain.

If you see no log output, then it didn't find any problems (which is good).

What "Xd" is designed to do is remove from the Textures13.db any row that refers to a non-existent thumbnail file.

If you use the "X" option it will perform the same function, but without deleting any rows (ie. it will just list the bogus database entries).
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Hi,

Kodi is crashing lately when i'm trying to use Texturecache with the qax option (after updating ALL nfo files of my movies - about 1500 files).

The kodi.log isn't really helpful - the last lines:

Quote:17:24:59 475.778198 T:1625822272 DEBUG: Mysql Start transaction
17:24:59 475.778595 T:1625822272 DEBUG: Mysql execute: DELETE FROM streamdetails WHERE idFile = 5151
17:24:59 475.780243 T:1625822272 DEBUG: Mysql execute: INSERT INTO streamdetails (idFile, iStreamType, strVideoCodec, fVideoAspect, iVideoWidth, iVideoHeight, iVideoDuration, strStereoMode) VALUES (5151,0,'h264',2.000000,1280,536,8315,'')
17:24:59 475.781677 T:1625822272 DEBUG: Mysql execute: INSERT INTO streamdetails (idFile, iStreamType, strAudioCodec, iAudioChannels, strAudioLanguage) VALUES (5151,1,'dts',6,'english')
17:24:59 475.782806 T:1625822272 DEBUG: Mysql execute: INSERT INTO streamdetails (idFile, iStreamType, strSubtitleLanguage) VALUES (5151,2,'english')
17:24:59 475.784637 T:1625822272 DEBUG: Previous line repeats 1 times.
17:24:59 475.784821 T:1625822272 DEBUG: Mysql execute: update movie set c11=8315 where idFile=5151 and c11=''
17:24:59 475.786255 T:1625822272 DEBUG: Mysql execute: update episode set c09=8315 where idFile=5151 and c09=''

The last lines of the texturecache log file:

Quote:2015-11-22 17:24:58.495905:MainThread: Rescanning directory: nfs://192.168.48.31/volume2/filme/Flight (2012)...
2015-11-22 17:24:58.496901:MainThread: libRescan.JSON SOCKET REQUEST: [{"jsonrpc": "2.0", "params": {"directory": "nfs://192.168.48.31/volume2/filme/Flight (2012)", "showdialogs": false}, "method": "VideoLibrary.Scan", "id": "libRescan"}]
2015-11-22 17:24:58.499562:MainThread: libRescan.BUFFER RECEIVED (len 48)
2015-11-22 17:24:58.500551:MainThread: libRescan.PARSING JSON DATA: {"id":"libRescan","jsonrpc":"2.0","result":"OK"}
2015-11-22 17:24:58.501379:MainThread: libRescan.PARSING COMPLETE, elapsed time: 0.000927 seconds
2015-11-22 17:24:58.502135:MainThread: libRescan.PERFORMING CALLBACK: Name [jsonWaitForScanFinished], with Id [libRescan], Method [OK], Params [None]
2015-11-22 17:24:58.502927:MainThread: libRescan.CALLBACK RESULT: [False] Name [jsonWaitForScanFinished], Id [libRescan], Method [OK], Params [None]
2015-11-22 17:24:58.503518:MainThread: libRescan.READING SOCKET UNTIL CALLBACK SUCCEEDS...
2015-11-22 17:24:58.681696:MainThread: libRescan.BUFFER RECEIVED (len 94)
2015-11-22 17:24:58.683496:MainThread: libRescan.PARSING JSON DATA: {"jsonrpc":"2.0","method":"VideoLibrary.OnScanStarted","params":{"data":null,"sender":"xbmc"}}
2015-11-22 17:24:58.684944:MainThread: libRescan.PARSING COMPLETE, elapsed time: 0.001558 seconds
2015-11-22 17:24:58.686483:MainThread: libRescan.PERFORMING CALLBACK: Name [jsonWaitForScanFinished], with Id [None], Method [VideoLibrary.OnScanStarted], Params [{u'data': None, u'sender': u'xbmc'}]
2015-11-22 17:24:58.687828:MainThread: libRescan.CALLBACK RESULT: [False] Name [jsonWaitForScanFinished], Id [None], Method [VideoLibrary.OnScanStarted], Params [{u'data': None, u'sender': u'xbmc'}]
2015-11-22 17:24:58.689007:MainThread: libRescan.READING SOCKET UNTIL CALLBACK SUCCEEDS...
2015-11-22 17:24:59.601251:MainThread: libRescan.BUFFER RECEIVED (len 0)
2015-11-22 17:24:59.601813:MainThread: libRescan.PERFORMING CALLBACK: Name [jsonWaitForScanFinished], with Id [None], Method [System.OnQuit], Params [{'data': -1, 'sender': 'xbmc'}]
2015-11-22 17:24:59.602141:MainThread: libRescan.CALLBACK RESULT: [False] Name [jsonWaitForScanFinished], Id [None], Method [System.OnQuit], Params [{'data': -1, 'sender': 'xbmc'}]
2015-11-22 17:24:59.602655:MainThread: Removing movieid 4922 from media library.
2015-11-22 17:24:59.603051:MainThread: libRemove.JSON SOCKET REQUEST: [{"jsonrpc": "2.0", "params": {"movieid": 4922}, "method": "VideoLibrary.RemoveMovie", "id": "libRemove"}]
2015-11-22 17:24:59.603669:MainThread: libRemove.BUFFER RECEIVED (len 0)
2015-11-22 17:24:59.604227:MainThread: ERROR: Socket closed prematurely - exiting

I tried texturecache from my NAS device and from my Raspberry Pi itself. The effect is the same. Most of the time, the crash happens after the same amount of files - more or less. It is not always the same file/folder, but the moviename starts with F - you get the idea.

Sometimes - not always - there is another error message in the texturecache.log:

Quote:2015-11-22 16:51:34.087888:MainThread: Removing movieid 3337 from media library.
2015-11-22 16:51:34.088012:MainThread: libRemove.JSON SOCKET REQUEST: [{"jsonrpc": "2.0", "params": {"movieid": 3337}, "method": "VideoLibrary.RemoveMovie", "id": "libRemove"}]
2015-11-22 16:51:34.119663:MainThread: libRemove.BUFFER RECEIVED (len 111)
2015-11-22 16:51:34.119957:MainThread: libRemove.PARSING JSON DATA: {"jsonrpc":"2.0","method":"VideoLibrary.OnRemove","params":{"data":{"id":3337,"type":"movie"},"sender":"xbmc"}}
2015-11-22 16:51:34.120105:MainThread: libRemove.PARSING COMPLETE, elapsed time: 0.000231 seconds
2015-11-22 16:51:34.120234:MainThread: libRemove.IGNORING NOTIFICATION [{"params": {"data": {"type": "movie", "id": 3337}, "sender": "xbmc"}, "jsonrpc": "2.0", "method": "VideoLibrary.OnRemove"}]
2015-11-22 16:51:34.120322:MainThread: libRemove.READING SOCKET FOR A RESPONSE...
2015-11-22 16:51:34.159240:MainThread: libRemove.BUFFER RECEIVED (len 48)
2015-11-22 16:51:34.159413:MainThread: libRemove.PARSING JSON DATA: {"id":"libRemove","jsonrpc":"2.0","result":"OK"}
2015-11-22 16:51:34.159530:MainThread: libRemove.PARSING COMPLETE, elapsed time: 0.000144 seconds
2015-11-22 16:51:34.159627:MainThread: libRemove.FINISHED, elapsed time: 0.071526 seconds
2015-11-22 16:51:34.159782:MainThread: Rescanning directory: nfs://192.168.48.31/volume2/filme/Foreign Correspondent (1940)...
2015-11-22 16:51:34.159912:MainThread: libRescan.JSON SOCKET REQUEST: [{"jsonrpc": "2.0", "params": {"directory": "nfs://192.168.48.31/volume2/filme/Foreign Correspondent (1940)", "showdialogs": false}, "method": "VideoLibrary.Scan", "id": "libRescan"}]
2015-11-22 16:51:34.186590:MainThread: libRescan.PERFORMING CALLBACK: Name [jsonWaitForScanFinished], with Id [None], Method [System.OnQuit], Params [{'data': -1, 'sender': 'xbmc'}]
2015-11-22 16:51:34.186751:MainThread: libRescan.CALLBACK RESULT: [False] Name [jsonWaitForScanFinished], Id [None], Method [System.OnQuit], Params [{'data': -1, 'sender': 'xbmc'}]
2015-11-22 16:51:34.186952:MainThread: Removing movieid 3338 from media library.
2015-11-22 16:51:34.187080:MainThread: libRemove.JSON SOCKET REQUEST: [{"jsonrpc": "2.0", "params": {"movieid": 3338}, "method": "VideoLibrary.RemoveMovie", "id": "libRemove"}]
ERROR:root:** Terminating due to unexpected exception **
Traceback (most recent call last):
File "./texturecache.py", line 8413, in <module>
main(sys.argv[1:])
File "./texturecache.py", line 8225, in main
extraFields=_extraFields, query=_query)
File "./texturecache.py", line 4568, in jsonQuery
qaData(mediatype, jcomms, database, data, title_name, id_name, rescan)
File "./texturecache.py", line 5257, in qaData
jcomms.rescanDirectories(workItems)
File "./texturecache.py", line 2531, in rescanDirectories
self.sendJSON(REQUEST, "lib%s" % method)
File "./texturecache.py", line 2191, in sendJSON
jsocket.send(json.dumps(request))
error: [Errno 32] Broken pipe
Reply
If Kodi is crashing you'll need to upload a crashlog to have any idea of what is causing the crash - depending on your OS and build this may or may not be available (it's available in OpenELEC test builds, but not release builds).

If you're using (or able to use) one of my OpenELEC test builds I can upload a debug-enabled test build later this evening which will provide a detailed crashlog.

Also, you should start a separate thread to continue the discussion as although TextureCache is provoking the crash, it's not doing anything out of the ordinary so this is really an issue with Kodi core functionality.

Presumably running "texturecache.py vscan 'nfs://192.168.48.31/volume2/filme/Foreign Correspondent (1940)'" doesn't produce a crash, which would suggest it's an accumulated error. One possible reason for the crash might be out of memory, due to memory leak - try running "bcmstat.sh AD" (included in my builds) to keep an eye on memory usage during the qax operation, perhaps all memory is exhausted by the time you get to "F". "journalctl --no-pager | paste" (on OpenELEC) would include details of any out-of-memory activity.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Hi there, first of all thanks for the great script. I have a small problem, maybe you can help:

I had some orphaned/wrong entries in the art-table of the MyMovies93.db which path/url was at "/var/tmp/..."which is after an upgrade from 14.x to 15.x is no longer there. That were the posters and fanart of various entries. I used sqllitebrowser to load the db and remove all entries in the art table.

After that I ran texturecache.py qax (and edited before the cfg file to scan for entries up to 9999 days old) but the missing fanart and posters are just "WARN" and not "FAIL" and get therefore not corrected/re-scanned.

I also tried variants of c, C, p, P but none of that worked

Any advice how to automatically rescan those movies/tvshows which have a WARN? Or a way to make them a FAIL so they get re-scanned?
(e.g.
WARN (missing fanart; missing poster)
WARN (missing fanart, local not found; missing poster, local not found; missing banner, local not found)
WARN (missing poster)
WARN (missing thumb, local not found)
)

The relevant entries in the cfg-file are:
allow.recacheall=yes
qa.file = yes
qaperiod = 9999

Many thanks in advance...
Reply
Artwork will "fail" if there is a suitable replacement local artwork file available (.

When there is no local artwork file available, the missing artwork will only be flagged as a warning (as there is no point in failing the item and refreshing it for no reason).

If you want items - movies, tvshows/episodes etc. - with missing artwork to be refreshed whether local artwork is present or not, then add "@qa.fail.missinglocalart=yes". This may allow you to find new remote artwork, but it will also continually refresh a movie or tvshow/episode with any item of missing artwork, each time you run the command. But the choice is yours...

By the way, you could use mklocal.py to automatically remove garbage or otherwise non-existent artwork by using the --nokeep option.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Windows user. Recently installed cygwin. Do you recommend running your scripts under bash rather than cmd or maybe powershell?

scott s.
.
Reply
Plain cmd with Python works (as per the installation instructions), there's no need for bash/cygwin./etc.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Hi, this may be a stupid question but... when I initially scanned my DB, there were a few local fanarts available to kodi (a few = several hundred..) which have been deleted in the interim, so all these files are missing posters etc on new machines which try to build their texture cache... is there any way to get these to download the artwork from the internet instead using the script (or another method) without doing it one by one? I understand that the locations are in the kodi database, not textures.db... so I'm unsure how to purge without rem,oving and rescanning the libary (which has MANY entrys that had to be manually fixed, so not an option right now...)

thanks for any ideas and nudges in the right direction!
Reply
You can use mklocal.py with the "--nokeep --readonly" options - this will remove artwork that is no longer accessible from the media library. Afterwards you can use something like Artwork Downloader to re-associate the movies with internet-based artwork.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
I'm having an issue where I need to reload all the tvshow.nfo for my whole library and wipe the old cached version. I removed the source, cleaned and re-added, but it didn't fix my problem. I need to manually remove and re-add each show.

What command would I do to execute this rather than spending hours removing and re-scraping?

I already used P,p,C, & c & aclean/vclean to clear up any garbage, but it seems that nothing happened to reload/wipe my nfos.
--------------------
Also, when I get The following items are present in the texture cache but not the media library: from a (p) ...why would it show .jpg and .avi files that I have in the source? The directory is perfect.
(Its mostly all extra fanart and 1 set of .avi Extras that I manually added)
Reply
tvshow.nfo can only be refreshed by removing the entire tv show, which isn't something this script supports (episodes, yes, via qax) so I'm afraid if you need to reload all your shows then removing the tv source (or removing the shows) then re-scraping should work.

When pruning the cache (P) if you want to keep extrafanart then add @cache.extrafanart=yes (also @cache.extrathumbs to keep extrathumbs, @cache.videoextras=yes for video extras, or @cache.extra=yes to enable all three).
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Excellent. Odd that source removal didn't fix it. I'll just do manual rescrapes. Looks like only about 30 were having an issue. 20 minute fix. Thanks much!
Reply
  •   
  • 1
  • 116
  • 117
  • 118(current)
  • 119
  • 120
  • 179
  •   
 
Thread Rating:
  • 17 Vote(s) - 4.76 Average



Logout Mark Read Team Forum Stats Members Help
[RELEASE] Texture Cache Maintenance utility4.7617