[RELEASE] Texture Cache Maintenance utility - Printable Version +- Kodi Community Forum (https://forum.kodi.tv) +-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33) +--- Forum: Supplementary Tools for Kodi (https://forum.kodi.tv/forumdisplay.php?fid=116) +--- Thread: [RELEASE] Texture Cache Maintenance utility (/showthread.php?tid=158373) Pages:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
|
RE: [RELEASE] Texture Cache Maintenance utility - gjwAudio - 2016-01-23 Hi Milhouse Thanks for the new version of the script. A timely question: where should the scripts & config file be kept and run from in OE ? Mine are currently sitting in "\\192.168.1.170\Root File System\storage\", but is this the ideal location for this code ? Thanks. Supplementary (n00b) Q: is there a complete list of CFG options I can download for reference ? EDIT: found it (!) on github. RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2016-01-23 (2016-01-23, 08:35)gjwAudio Wrote: where should the scripts & config file be kept and run from in OE ? Not sure I really understand, you can store the code wherever you like/wherever makes sense. The config should ideally be in the same directory as the script, or in your current directory when running the script. The config search order is (first is #1):
The last one means a folder called .config in your users home directory. Alternatively you can specify the configuration filename with @config=<filename> (2016-01-23, 08:35)gjwAudio Wrote: Supplementary (n00b) Q: is there a complete list of CFG options I can download for reference ? Or: Code: texturecache.py config which will list your current configuration (most of them will be defaults). It will also tell you the location of the configuration file that is being used. The sample on github is out of date - I kinda regret creating it as people think it's a good idea to use it as their cfg file. Just add what properties you need, don't copy the sample file. RE: [RELEASE] Texture Cache Maintenance utility - gjwAudio - 2016-01-26 Hi Milhouse (2016-01-20, 07:03)Milhouse Wrote: So when you re-scrape/refresh a movie you're going to lose all your non-standard artwork types, ie. your banner, whether it's in your NFO or not.Yeah, I HATE when this happens (...quite regularly ). (2016-01-20, 07:03)Milhouse Wrote: This is where you need to... re-associate the banner artwork manually... - knock up a small script that calls "texturecache.py vscan" then mklocal.py followed by "texturecache.py c movies" and you shouldn't have any more problems with non-standard artwork. Indeed. I've been looking at the post announcing mklocal.py, and can't quite discern the minimum number of parameters required to make it do what I need . I desperately need that knocked-up script, to fix/restore the non-standard art every time a scan wipes it out ! The chicken-and-egg thing is making me crazy... how can I create a process that will recognize these non-standard guys and "automatically" re-associate them with their respective movies ? Also, can it be run repeatedly without messing up any fixes done previously ? OK, I presume it's possible . Curiosity (on my part) wants to know how the three steps you outline work together. What does the vscan do/create, that mklocal then acts upon ? (BTW, you might check the 5th paragraph in that mklocal post - I think a typo has crept into the example of mapping a local file to a URL) Oh, and... I read most of the AD thread as you suggested... man, I need a shower after going through all that ! You deserve uber-points for patience . ps: I'm having no luck understanding what fields can be queried with the "extrajson.* = fieldname" parameters in the config file. I thought the info here would show me a list, but no. RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2016-01-26 (2016-01-26, 08:17)gjwAudio Wrote: Indeed. I've been looking at the post announcing mklocal.py, and can't quite discern the minimum number of parameters required to make it do what I need . I desperately need that knocked-up script, to fix/restore the non-standard art every time a scan wipes it out ! The example in the linked post is pretty minimal, you'd just change the artwork types to "--artwork fanart poster banner" and it will automatically download any remote fanart/poster/banner artwork to your local filesystem alongside your movies, and then associate any new local fanart/poster/banner artwork with your movies. If you don't want remote artwork downloaded, add "--readonly". Change the --local and --prefix arguments to suit your configuration. (2016-01-26, 08:17)gjwAudio Wrote: The chicken-and-egg thing is making me crazy... how can I create a process that will recognize these non-standard guys and "automatically" re-associate them with their respective movies ? That's what we're discussing. (2016-01-26, 08:17)gjwAudio Wrote: Also, can it be run repeatedly without messing up any fixes done previously ? Yep. (2016-01-26, 08:17)gjwAudio Wrote: OK, I presume it's possible . Curiosity (on my part) wants to know how the three steps you outline work together. What does the vscan do/create, that mklocal then acts upon ? "texturecache.py vscan" initiates a video library scan (use ascan for audio/music scans) - this will add any new movies (or tvshows/episodes) to your library. It's basically the same as clicking "Update library" in the GUI. After this any new movies will have poster and fanart - possibly remote if nothing found locally - but no banners. You then run mklocal.py to download any remote artwork that is being used (although this is optional), and mklocal.py will also automatically associate movies that don't already have associations with any matching local artwork, including banners. If a movie already has an association with a local artwork file it won't be changed, unless you add the option --nokeep as this will enable removal of any association where the artwork file (remote or local) no longer exists - handy when cleaning up garbage/stale/non-existent associations. Once you've created new associations in your media library you might want to pre-load the texture cache of every client by executing "texturecache.py c movies" (or sets, or tvshows) against each client in turn. I run a process similar to this whenever new movies/episodes are detected on my NAS. Or you could just run it on a regular schedule, say every morning at 3am, or just run it whenever you feel like. (2016-01-26, 08:17)gjwAudio Wrote: (BTW, you might check the 5th paragraph in that mklocal post - I think a typo has crept into the example of mapping a local file to a URL) Thanks, fixed. (2016-01-26, 08:17)gjwAudio Wrote: ps: I'm having no luck understanding what fields can be queried with the "extrajson.* = fieldname" parameters in the config file. I thought the info here would show me a list, but no. For extrajson.movies: Video.Fields.Movie - click on "[show]": Code: "items": { eg. Code: $ texturecache.py Jd movies avatar @extrajson.movies=rating,votes,dateadded RE: [RELEASE] Texture Cache Maintenance utility - zeroday - 2016-01-29 Found my way into the tool Rescanning directory: smb://SHARK/Usenet/TV/Ballers/Season 1.. it's possible that it keeps stalling here for minutes (+15 min) but no error message ./texturecache.py qax tvshows "Ballers" gives no output and when starting: qax again .. it will not use above example anymore and goes to the next Rescanning directory: smb://SHARK/Usenet/TV/Bitten/Season 2... my config userdata = /media/kodi/ dbfile = Database/Textures13.db thumbnails = Thumbnails/ xbmc.host = 192.168.1.154 webserver.port = 80 #rpc.port = 9090 download.threads = 2 #extrajson.albums = #extrajson.artists = #extrajson.songs = #extrajson.movies = #extrajson.sets = extrajson.tvshows.tvshow = plot, rating, cast, file, imdbnumber extrajson.tvshows.episode = streamdetails, file, dateadded, rating, plot, cast # extrajson.tvshows.tvshow = # extrajson.tvshows.season = # extrajson.tvshows.episode= #qaperiod = 30 #qa.file = no qa.fail.missinglocalart=yes qaperiod=9999 cache.extra = yes cache.castthumb = yes allow.recacheall=yes logfile = /home/dennis/thumbs/texturecache.log logfile.verbose = yes #network.mac = aa:bb:cc:dd:ee:ff log tells me 2016-01-29 21:34:59.905087:MainThread: libRescan.PERFORMING CALLBACK: Name [jsonWaitForScanFinished], with Id [None], Method [VideoLibrary.OnCleanFinished], Params [{u'data': None, u'sender': u'xbmc'}] 2016-01-29 21:34:59.905142:MainThread: libRescan.CALLBACK RESULT: [False] Name [jsonWaitForScanFinished], Id [None], Method [VideoLibrary.OnCleanFinished], Params [{u'data': None, u'sender': u'xbmc'}] 2016-01-29 21:34:59.905186:MainThread: libRescan.READING SOCKET UNTIL CALLBACK SUCCEEDS... and than it stalls .. RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2016-01-30 Send me the whole logfile. RE: [RELEASE] Texture Cache Maintenance utility - zeroday - 2016-01-30 Thanks for such quick reply http://we.tl/G60T1HbY3H started with empty logfile started with ./texturecache.py qax tvshows logfile 5MB .. (sorry I have too much tvshows) ps. When I use ./texturecache.py qax tvshows "Baby Daddy" than the tool is finishing its job in above log baby daddy as example is stalling on the callback part and in the 2nd log (much smaller btw) you see it is finishing ok http://we.tl/9aYzHwj4WY RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2016-01-30 (2016-01-30, 09:21)zeroday Wrote: ps. This isn't doing a scan, which is why it succeeds - from what I can tell it found no QA issues that needed fixing. (2016-01-30, 09:21)zeroday Wrote: Thanks for such quick reply It looks like you've got some other process that is starting a video library clean while the script is running, which is conflicting with the subsequent video library scan started by texturecache.py. In the following log detail, while texturecache.py is removing a Baby Daddy Season 4 episode, a library clean is started. The texturecache.py script sees the "OnCleanStarted" notification, which it simply ignores as it's not relevant to it's current processing. The library clean is not being started by texturecache.py - the library clean must be being started from the GUI, or an addon or some other external script/cron: Code: 2016-01-30 09:09:10.367642:MainThread: Removing episodeid 6858 from media library. Subsequently, when the texturecache.py script tries to start a library scan - when the library clean is still running - the scan request is silently ignored by Kodi because there is a library clean in progress: Code: 2016-01-30 09:09:10.790270:MainThread: Rescanning directory: smb://SHARK/Usenet/TV/Baby Daddy/Season 4... Consequently, the texturecache.py script never sees a "VideoLibrary.OnScanFinished" notification and is left hanging, waiting forever for a notification that is never going to come (unless you manually start another library scan). 18 seconds after the library scan request (which didn't start, despite the "OK" result), the library clean finishes but again this notification is not relevant as the texturecache.py script is waiting for a scan to finish, not a clean... so it continues to wait. Perhaps I could add some sort of "state" so that the script knows if a library clean has been started (or queries the clean/scan status before trying to kick off a scan, and polls/waits for it to complete)... nah, just stop running whatever process is automatically starting the library clean. If you want to clean your library you can use the texturecache.py script (aclean/vclean), but to avoid hanging the script while it is trying to run a library scan simply stop cleaning your library at the same time! RE: [RELEASE] Texture Cache Maintenance utility - zeroday - 2016-01-30 ok, this helped me a lot The issue is the xbmc auto updater plugin When we do the scan ./texturecache qax tvshows I see that kodi.log notices that there is an incoming scan but .. in my config fo the xbmc auto updater plugin I have stated: clean after scan This is done with a purpose: make sure that there are no unleft references of removed tvshows after the database have been scanned (ie. HDTV replaced by WEB-DL) After changing this setting to daily .. the issue was resolved. So if I am correct I coudl do following: disable xbmc auto updater as plugin in Kodi create simple shell script (anyone?) with a few ./texturecache commands like scan than ./texturecache c | ./texturecache qax than clean .. Or something like this? RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2016-01-31 (2016-01-30, 19:34)zeroday Wrote: So if I am correct I coudl do following: Yep, should work - you can automate pretty much everything with texturecache.py. Run the script on demand (click from a desktop), on a schedule (ie. daily) or run the script constantly (daemon) and then vscan/qax/c/vclean whenever new content becomes available. How creative you want to be is up to you - I do a combination of the last two options, vscan/qax/c as new movies/episodes appear in the NAS filesystem, with a regular daily qax/c to process any new thumbnails/artwork or updated NFOs. RE: [RELEASE] Texture Cache Maintenance utility - lstar337 - 2016-02-01 Can I use this: RunScript(script[,args]*) to create a button for this in my kodi home menu? Like say, Action = RunScript(special://profile/texturecache.py[,c]) ? Or am I way off base with this? RE: [RELEASE] Texture Cache Maintenance utility - MANswers - 2016-02-01 Sir, i have an issue here, and i hope texture cache can help me with this. Recently I deleted a few duplicate movies manually. The movies would show up in Kodi two times, anywas, will this utility help me remove those duplicate entires from Kodi? I have already run the command using the switch "c", "p", "P" ran all one after another (without the double quote) It ran did a lot of things, but still the entries remain. Please guide, or do i need to delete the movies database files and re-update kodi? RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2016-02-02 (2016-02-01, 17:24)lstar337 Wrote: Can I use this: RunScript(script[,args]*) to create a button for this in my kodi home menu? Should be possible. Note the square brackets denote optionality, so you don't include the [] in your actual command. RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2016-02-02 (2016-02-01, 23:30)MANswers Wrote: Recently I deleted a few duplicate movies manually. The movies would show up in Kodi two times, anywas, will this utility help me remove those duplicate entires from Kodi? The script won't automatically remove duplicate movies, but it will identify duplicate movies based on imdbnumber ("texturecache.py duplicates"). What you do with them is up to you - you can remove them based on the movie id ("texturecache.py remove movie 12345" where 12345 is the movie id). If you're having problems with duplicate movies you probably need to work out what is causing that and delete or rename the media files that are causing the duplicates to be created by Kodi. RE: [RELEASE] Texture Cache Maintenance utility - lstar337 - 2016-02-02 (2016-02-02, 01:46)Milhouse Wrote:Thanks.(2016-02-01, 17:24)lstar337 Wrote: Can I use this: RunScript(script[,args]*) to create a button for this in my kodi home menu? I gave it a go, but I get an addon error. Looking in the log it says this: Code: 10:27:01 T:2640049056 WARNING: CPythonInvoker(49): Script invoked without an addon. Adding all addon modules installed to python path as fallback. This behaviour will be removed in future version. This is really reaching the limit of my understanding. Here is what I am putting in: |