[RELEASE] Texture Cache Maintenance utility - Printable Version +- Kodi Community Forum (https://forum.kodi.tv) +-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33) +--- Forum: Supplementary Tools for Kodi (https://forum.kodi.tv/forumdisplay.php?fid=116) +--- Thread: [RELEASE] Texture Cache Maintenance utility (/showthread.php?tid=158373) Pages:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
|
RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2013-08-05 Argghhh... I'm totally confused why your artwork isn't caching - it can't be because of the backslashes, because even those files with only forward slashes are not caching. You shouldn't, in theory, have had any artwork to cache for the "nc" run, and the run above should have deleted all 82 items, not just 5! Can you do me a favour... run the following commands in the following sequence (top to bottom) - do not access the XBMC GUI until all commands have been run: Code: texturecache.py x > C:\Temp\x0.log then zip up the seven log files (x0.log, tc1.log, out1.log, x1.log, tc2.log, out2.log and x2.log) and PM me a link - that would be much appreciated. Maybe I can spot what is happening. Also, can you check your XBMC log to make sure there are no errors there (filesystem errors, or database errors?) Maybe include xbmc.log in the zip... Cheers. RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2013-08-05 Thanks to CaptainKen for helping debug his problem, which is being caused by those backslashes. It turns out that XBMC is automatically converting the erroneous backslashes into forward slashes when creating entries in Textures13.db, so reconciling the media library paths (with backslashes) against the texture cache (without backslahes) was problematic... The script will now inspect the path and replace the slashes whenever it can't find a match, so the overhead should be fairly minimal. Version 0.8.9
RE: [RELEASE] Texture Cache Maintenance utility - teeedubb - 2013-08-06 Hi MilhouseVH I'm getting the following error when tryig to use your script: Code: ./texturecache.py P May it be because GitHub is down? EDIT: Github is back up and the script is now working. RE: [RELEASE] Texture Cache Maintenance utility - teeedubb - 2013-08-07 I'm trying to clean up my 12gb userdata/9gb thumbnails directory, but I'm not not having much luck using p/P. The same library scanned into a fresh install of xbmc weighs in at 1.1gb. texturecache a few hundred megabytes on first run and has removed small amounts in certain runs since (like in the log below). If it matters, I have removed all the old *.db files in the ~/.xbmc/userdata/Database directory and currently on have one of each. ./texturecache.py p @logfile=/media/storage/tc.log @logfile.verbose=yes https://dl.dropboxusercontent.com/u/22410524/xbmc/logs/tc.log I have been thinking that I may need to delete the thumbnails folder and use the c/C switch to re-cache everything? This way I wouldn't lose any additional data thats stored in the xbmc database. RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2013-08-07 According to the log, a very large number of files (71,051 to be precise) are being detected as "orphans" - these are files that exist in the Thumbnails folder, but do not have an entry in the Textures13.db database, ie. they were created and referenced by the cache at some point in the past, but are now no longer referenced. These are most likely cached files left over from a previous pre-Frodo installation. I would suggest running "texturecache.py r" to see details of the orphan files, then "texturecache.py R" to automatically remove them. If there are "too many" files identified as orphans (ie. more than 5% of your entire Thumbnails folder) you will need to enable "orphan.limit.check = no" in the properties file for the script to proceed with removal - only set this property if you think removing this number of thumbnails is reasonable for your system! Since you only have 9,328 rows in your Textures13.db, it would seem entirely reasonable to set/enable the property - the orphaned files really do look like genuine orphaned files and not some error in the script. Once you have removed the orphaned files, what remains should be files that are indexed within the texture cache. Pruning (p/P) should then identify these files that are known to the texture cache, but have no matching entry in your media library (and should therefore be safe to delete). I suppose it would be a good idea if the "prune" options notified the user if orphaned files are detected. Not sure what you mean by (2013-08-07, 05:59)teeedubb Wrote: a few hundred megabytes on first run and has removed small amounts in certain runs sinceas the files identified by "texturecache.py p" should be consistent from one run to the next (unless you access new artwork in the GUI which would create more "prunable" files). If you were to run "texturecache.py P", and then follow it up with "texturecache.py p" there should be no files listed as a result of the latter run. If you did see files listed for "p" it would suggest a problem. Keeping the logs for any runs would be useful just in case! RE: [RELEASE] Texture Cache Maintenance utility - teeedubb - 2013-08-07 (2013-08-07, 06:21)MilhouseVH Wrote: According to the log, a very large number of files (71,051 to be precise) are being detected as "orphans" - these are files that exist in the Thumbnails folder, but do not have an entry in the Textures13.db database, ie. they were created and referenced by the cache at some point in the past, but are now no longer referenced. These are most likely cached files left over from a previous pre-Frodo installation. This worked nicely, now the thumbnails folder is 2.9gb, thanks! Quote:I suppose it would be a good idea if the "prune" options notified the user if orphaned files are detected. Sorry, I should have been clearer - I was using a mix of p and P and the small amounts of deleted files would have been new artwork. No problems with files being listed from p after P. Thanks for your help Milhouse! RE: [RELEASE] Texture Cache Maintenance utility - teeedubb - 2013-08-07 I've also disabled dds fanart which was enabled a long time ago and isn't needed with my current setup, after deleting all *.dds fanart the Thumbnails folder is now 810mb RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2013-08-07 Yes... after pruning the artwork items, the corresponding dds files would have been left as orphans - will see if I can clobber the dds files whenever the original artwork is pruned away. RE: [RELEASE] Texture Cache Maintenance utility - Martijn - 2013-08-07 iirc it is doubtful if the dds files still have any form of performance improvement will all the code improvements done of the time. RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2013-08-07 dds is certainly disabled by default on the Pi, but could still be enabled on an x86 system (or the old dds files could be left lying around after upgrading an x86 system to a more recent version which has dds disabled) so they still need to be processed. RE: [RELEASE] Texture Cache Maintenance utility - Maetrik - 2013-08-07 When using option "c" i always get this message after movies where checked: Code: Caching artwork: 0 items remaining of 320 (qs: 0, qm: 0), 0 errors, 0 threads active (06.10 downloads per second, ETA: 00:00:00) Should i be worried about that? Music and TVShows dont show that error message. RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2013-08-07 What media class are you using to generate this error - "video"? I guess you have no movie Sets defined... will try and suppress that error, somehow. RE: [RELEASE] Texture Cache Maintenance utility - Maetrik - 2013-08-07 Yes, i do not have any sets defined. What do you mean with media class? RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2013-08-07 (2013-08-07, 19:04)Maetrik Wrote: Yes, i do not have any sets defined. What command are you using to run the script and generate this error? (2013-08-07, 19:04)Maetrik Wrote: What do you mean with media class? A "media class" is one of artists, albums, songs, movies, sets, tags, tvshows, addons, genres. There are also three "meta media classes": "music" (artists + albums + songs), "video" (movies + sets + tvshows) and all (basically, all media classes). So, running "texturecache.py c video" would cache movies + sets + tvshows, but since you don't have any Sets defined you are seeing this error. RE: [RELEASE] Texture Cache Maintenance utility - Maetrik - 2013-08-07 Well, i just ran the c option, ./texturecache.py c I deleted Textures13.db and ran your script again, so i can post a full "log": http://pastebin.com/yZc6BqBB (error is in line 283 and 284) My setup: Win7 HTPC, MySQL on a Synology NAS, ATV2 and MacBook also connected through MySQL. My .cfg file: Code: sep = | I've mounted the HTPCs XBMC folder on my MacBook so that i can do everthing like scraping and running the script from the OS X machine. |