[RELEASE] Texture Cache Maintenance utility - Printable Version +- Kodi Community Forum (https://forum.kodi.tv) +-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33) +--- Forum: Supplementary Tools for Kodi (https://forum.kodi.tv/forumdisplay.php?fid=116) +--- Thread: [RELEASE] Texture Cache Maintenance utility (/showthread.php?tid=158373) Pages:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
|
RE: [RELEASE] Texture Cache Maintenance utility - akya - 2014-07-01 Cool No problem.. I updated from 1.6.4 to 1.6.6 successfully and everything is fine now cheers ! RE: [RELEASE] Texture Cache Maintenance utility - SirCmpwn - 2014-07-08 Hi there! I screwed up my cache and I thought I'd try fixing it by force with `./texturecache.py C` (with the config set appropriately). Weirdly enough, it goes through everything and errors out on every one (the following items could not be downloaded...). However, each of the items it tells me it couldn't download includes a URL that leads to a perfectly downloadable file. Help? RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2014-07-08 Check xbmc.log for errors. Turn on debugging, particularly curl logging. Some websites may reject multiple concurrent requests, try reducing the number of download threads to 1. RE: [RELEASE] Texture Cache Maintenance utility - gargamon - 2014-07-13 I've been using your very useful utility as needed for a while now and it's worked very well. I now have an issue with some addon icons and fanart. The cached images appear to be missing as the icons are displayed as the jigsaw puzzle and the backgrounds are the skin default. I checked the logs and noticed an error that a particular fanart was missing so I manually copied it from the addon folder to the appropriate cache location and that resolved the fanart problem for that addon. The icon is still the jigsaw puzzle and no error log was produced. I've tried a number of things to figure out what the issue is. ./texturecache.py C addons "TMZ" gave an error on the thumbnail. With debug on there was nothing in the log to indicate the issue. python ./texturecache.py j addons gave a list of addons as expected with the correct icons and fanart in the addons directory. So I have a couple of questions. 1) How can I fix this with the script? 2) How can I find the location of the offending cached icon? I have no problem fixing the few broken ones I have by hand if necessary, as I did with the missing fanart. Thanks RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2014-07-13 If you run "./texturecache.py jd addons tmz" you should see the artwork urls associated with the addon - check that these artwork items exist, and are accessible to XBMC (as long as they're in the .xbmc/addons folder, they should be accessible, but if you're doing something like path substitution then all bets are off!) You can search for the cached TMZ artwork with "./texturecache.py s tmz" - this will list the database rows with matching urls (urls containing tmz - use the same urls from the above "jd" call if you want to be more precise). You can then delete individual rows and associated cached artwork files with "./texturecache.py d #" where # is the rowid (left-most column in the list). After removing any cached artwork items you can then perform a normal cache pre-load with "./texturecache.py c addons tmz", that should pre-load any missing (ie. recently deleted) artwork items. If you still have errors then enable debug log (wiki) in XBMC, ideally with curl debugging enabled, and upload it to pastebin after trying to preload the cache for this addon. A logfile from texturecache.py would also be useful (add @logfile=/tmp/tc.log when pre-loading the cache and upload tc.log to pastebin.com). RE: [RELEASE] Texture Cache Maintenance utility - gargamon - 2014-07-13 Thanks for the quick response. I don't use path substitution. In all my cases the addons were installed correctly and had tha jpg's and png's in the proper location. The files in cache indicated in the "s tmz", etc queries did not exist. I simply copied the files (and renamed them) to the proper place in cache and full joy. I would have thought that "addons c tmz" should have fixed this, but for sure "addons C tmz" would write the missing image file, but they did not. Anyway, thanks again for the quick accurate response. RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2014-07-13 (2014-07-13, 05:36)gargamon Wrote: In all my cases the addons were installed correctly and had tha jpg's and png's in the proper location. The files in cache indicated in the "s tmz", etc queries did not exist. I simply copied the files (and renamed them) to the proper place in cache and full joy. OK, so what you're saying is that you have rows in the database but the corresponding files (cachedurl) are not in the cache? This would make sense if there are problems writing the artwork files into the cache (Thumbnails folder), and would suggest the image conversion (resizing etc.) is failing for some reason, after the database row is inserted. Maybe there's something peculiar about the artwork being used by this addon? By the way, "./texturecache.py X" will identify any Textures13 database rows that do not have a corresponding cache file. And "./texturecache.py Xd" will automatically remove these rows from the database. To identify cached artwork that is no longer referenced by the database , use "./texturecache.py r", and to automatically remove this artwork use "./texturecache.py R". (2014-07-13, 05:36)gargamon Wrote: I would have thought that "addons c tmz" should have fixed this, but for sure "addons C tmz" would write the missing image file, but they did not. It should have, but if XBMC can't write the file to the cache then that's a problem and not one the script can solve. As a test, remove the artwork from the cache (remove from both Textures13.db and the Thumbnails folder), restart XBMC, enable debug logging (and CURL component debugging, if available in your XBMC build) and then browse to the TMZ addon icon/fanart in the GUI. Can you confirm if you still have the same problem, ie. no artwork displayed for the addon? If so, there should be errors present in the XBMC log. It would be useful to know why XBMC is failing to cache these artwork files, it could be a jpeg/png format issue, and/or could be dependent on the device you are using for XBMC (R-Pi, Android, x86 etc.). If you could upload a full debug log (wiki) that would be great, although it's almost certainly an XBMC-related issue (which would be confirmed if XBMC fails to update the cache via the GUI) and should probably be addressed in the general help section. RE: [RELEASE] Texture Cache Maintenance utility - gargamon - 2014-07-13 Ok, here we go. I used raspbmc.browser instead of tmz to do my tests. Raspbmc.browser is in the same state tmz was before I "fixed" it. I saw nothing in the logs about that addon, but here they are: http://pastebin.com/tTJVnEDQ Because I saw nothing there I thought it might be useful to grab the texturecache logs. I ran 2 scenarios, with "c addons browser" http://pastebin.com/dbqCbyGQ and with "C addons browser" http://pastebin.com/wubP9Sea I hope that's enough to give a clue as to the problem. Some historical info that may be helpful. I used to run R-Pi openelec and recently converted to Raspbmc. Originally the TMZ icon worked in Openelec. Some time before I migrated to Raspbmc I needed to rebuild my cache database in openelec. I deleted all the thumbnails and the texturesXX.db and ran the texturecache utility to repopulate the cache. I think sometime around then I lost the TMZ and other icons. On the change from openelec to raspbmc I migrated from nfs to fstab mounting of my media. This also necessitated a rebuild of the texture cache as above. Please let me know if you find anything. RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2014-07-13 (2014-07-13, 10:10)gargamon Wrote: Ok, here we go. I used raspbmc.browser instead of tmz to do my tests. Raspbmc.browser is in the same state tmz was before I "fixed" it. I saw nothing in the logs about that addon, but here they are: Hmmm... as you say, there's no evidence of XBMC loading the icon.png or fanart.jpg artwork for raspbmc.browser at all in your debug log... What I can see from the texturecache logs is that the fanart.jpg has been cached successfully, but the icon.png fails - why, I don't know, it's XBMC that is failing here. When I re-cache artwork for an addon (eg. "C addons artwork.downloader") I'm seeing confirmation that the artwork is successfully cached (written) to the file system - note the "Fast Caching image" and "cached image" entries here. Can you upload the XBMC debug log when running "C addons raspbmc.browser @Download.threads=1" - using a single download thread will serialise the webserver requests and make it easier to follow. If there are still no errors in your debug log, then I would suggest deleting Textures13.db in case it is corrupt, and if that still doesn't help then try re-installing... RE: [RELEASE] Texture Cache Maintenance utility - gargamon - 2014-07-13 Here's the XBMC debug logs you requested. http://pastebin.com/rHXuhLc8 I'm happy with the way my XBMC is running now. The icon's I care about have all been fixed and they were the last thing I needed to get right. The later debugging stuff was just to assist in tracking down a potential bug. Thanks again for all your help. RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2014-07-13 This is the only debug written for the icon.png requests: Code: 18:14:48 T:2903000128 DEBUG: webserver: request received for /jsonrpc texturecache.py will retry 3 times when communicating with the web server (4 failed requests in total). So what we see here is the web server failing to complete the Files.PrepareDownload method for this file - it doesn't appear to get as far as actually loading the file, but without more detail it's hard to say why it is failing, though I'd double check file permissions. Enabling CURL debug might reveal more detail regarding the failure, but if you're happy I'm happy. RE: [RELEASE] Texture Cache Maintenance utility - gargamon - 2014-07-13 Interestingly enough, after my last test I decided to clean up the textures db and ran the "Xd" and "R" commands and then the browser icon magically appeared. Are you sure the code checks if there is an actual file in the cache or only if the db says there is one? RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2014-07-13 (2014-07-13, 12:48)gargamon Wrote: Interestingly enough, after my last test I decided to clean up the textures db and ran the "Xd" and "R" commands and then the browser icon magically appeared. The db is king. The caching code looks only at the db for a few reasons: 1) performance - hitting the filesystem would be very slow (plus, for remote users, there is no JSON API for that), 2) it's basically the same approach used by XBMC, and 3) the db is supposed to be correct! However we do know from time to time that the db lies, which is when problems start - and that's why the Xd/R options exist to help recover from these problems and synchronise the db (Textures13.db) and filesystem (Thumbnails) once more. Having a file in the Thumbnails folder that isn't referenced by the db - fixed by R - isn't generally a problem. When XBMC tries to display the artwork and fails to find a row in the db it will create a new db row and then overwrite the existing Thumbnails file. At worst it wastes a little file space but shouldn't ever result in any visual/display issues. However a row in the db with no underlying Thumbnails file is a different matter entirely. In this case XBMC will find the row in the db, then look in the filesystem but fail to find the file, the net result being that no artwork is displayed (you'll see only the standard placeholders). In theory XBMC should log an error when it can't find the file but as you've seen that doesn't always happen so it can be a very perplexing problem. The only way for XBMC to create the missing file is for the offending db row to be removed. Fortunately "Xd" will do this automatically. Anyway, glad you got it sorted. RE: [RELEASE] Texture Cache Maintenance utility - theowiesengrund - 2014-07-22 Hi Milhouse, just a quick question. I was browsing my library and looking for "obscure/rare" movies, therefore i would love to sort/list my movies according to the imdb vote numbers. Is this somehow possibile with your script, for example with a json query? RE: [RELEASE] Texture Cache Maintenance utility - gargamon - 2014-07-22 (2014-07-13, 13:10)Milhouse Wrote:(2014-07-13, 12:48)gargamon Wrote: Interestingly enough, after my last test I decided to clean up the textures db and ran the "Xd" and "R" commands and then the browser icon magically appeared. I was thinking about this again. It seems to me that when you're doing the "C" option, you really want to overwrite whatever is there, whether it's nothing, a corrupted file, or who knows what. Wouldn't that imply that you no longer trust the database and want to rewrite that row anyway? I know it would be slower than now, but the added functionality of being able to do "C tvshows some_show_name" is much more versatile than doing "Xd"on a particular row when you may want to do 20 or more rows. Maybe a new command could handle this. |