[RELEASE] Texture Cache Maintenance utility - Printable Version +- Kodi Community Forum (https://forum.kodi.tv) +-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33) +--- Forum: Supplementary Tools for Kodi (https://forum.kodi.tv/forumdisplay.php?fid=116) +--- Thread: [RELEASE] Texture Cache Maintenance utility (/showthread.php?tid=158373) Pages:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
|
RE: [RELEASE] Texture Cache Maintenance utility - rainman74 - 2019-10-15 @millhouse When cleaning, the castthumbs are always deleted, although I have activated them in the setting. What am I doing wrong? Code: python texturecache.py P @profile.name=Parents texturecache.cfg Code: #userdata = ~/.xbmc/userdata/ Code: Command line args: ['texturecache.py', 'c', '@profile.name=Parents'] RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2019-10-15 I don't know how you've populated the "Parents" media library, or corresponding texture cache. Is the "Parent" profile separate or shared? If separate, were the "Parent" libraries copied from "master" (default) or created from scratch? If you copied from master, maybe the cached cast artwork isn't present in the "Parents" media library. Check you have enabled "Download actor thumbnails" in Settings > Media > Videos for the "Parents" profile. Failing that, a logfile ( @logfile=tc.log ) for the p prune command should help explain what is occurring (lowercase p won't remove anything, but will instead list what it would have deleted). I'll need the entire log file, so probably best to zip it and upload it somewhere.Note that the cache.castthumb option only dictates whether the script will cache cast artwork (if there are references to the cast artwork in your media library). In the texture cache there's no way to tell a cast artwork from a poster - all cached artwork is just a "thumbnail" without any specific type (ie. poster, fanart, cast etc.) - so there's no way to prevent removal of a thumbnail that happens to be a cast artwork, apart from adding a regex to prune.retain.types - if all your cast artwork is hosted on imdb.com then adding prune.retain.types=http://ia.media-imdb.com might stop unreferenced cast artwork from being removed.But adding the missing references (ie. fixing the underlying issue) might be easier. RE: [RELEASE] Texture Cache Maintenance utility - rainman74 - 2019-10-15 (2019-10-15, 12:24)Milhouse Wrote: I don't know how you've populated the "Parents" media library, or corresponding texture cache. Is the "Parent" profile separate or shared? If separate, were the "Parent" libraries copied from "master" (default) or created from scratch?The parent profile is my master profile ("Eltern" in German), I just renamed it. Strangely enough, the cast thumbs are not deleted from the second "Children" ("Kinder" in German) profile, which I created as an additional standalone separated profile (python texturecache.py p @profile.name=Children). Quote:If you copied from master, maybe the cached cast artwork isn't present in the "Parents" media library. Check you have enabled "Download actor thumbnails" in Settings > Media > Videos for the "Parents" profile.I had already activated this option. Quote:Failing that, a logfile (Ok, there is my log file: log.zip Quote:Note that theIf the root cause can't be found, I should try that, shouldn't I? prune.retain.types=/.actors/.* Update: I have just seen that with selected "Parent" profile also thumbnails of children medias (e.g. episode thumbnails) should also be deleted, but they are not present in the parent profile at all, these media files were never added by me to the library in the parent profile! Is it possible that Kodi gets confused internally when using profiles, or was it because I cached the textures before? But also here I strictly separated the profiles: First -> python texturecache.py c @profile.name=Parents Second -> python texturecache.py c @profile.name=Children RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2019-10-19 Sorry for taking so long to reply. I've just downloaded your log.zip, but it looks like you've reformatted it (pretty printed the JSON data) which has kind of made it a bit useless to me as I need it in the original raw format - the texturecache.py script has a @replayfile option which takes the log file as input and allows me to reproduce and debug your data as though it were my data (assuming the log hasn't been tampered with...).Any chance you can run the command again, and upload the unmodified log file? RE: [RELEASE] Texture Cache Maintenance utility - rainman74 - 2019-10-21 (2019-10-19, 07:18)Milhouse Wrote: Sorry for taking so long to reply. I've just downloaded your log.zip, but it looks like you've reformatted it (pretty printed the JSON data) which has kind of made it a bit useless to me as I need it in the original raw format - the texturecache.py script has a ok, here it is ... link RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2019-10-25 (2019-10-21, 12:15)rainman74 Wrote:(2019-10-19, 07:18)Milhouse Wrote: Sorry for taking so long to reply. I've just downloaded your log.zip, but it looks like you've reformatted it (pretty printed the JSON data) which has kind of made it a bit useless to me as I need it in the original raw format - the texturecache.py script has a Sorry for the delay (again) - been really busy. OK, I've figured out why your log looks the way it does - you've configured: in your Kodi advancedsettings.xml, which means the JSON responses from Kodi are being written to the log in a "pretty" format that I can't currently parse (and replay). Can you disable this compactoutput setting in your as.xml (or set it to true ), restart kodi, reproduce the issue and then re-upload your log (for a third time - sorry!)Maybe one day I'll store the responses from Kodi with all tabs and newlines stripped so that this as.xml setting doesn't cause a problem, but I'm not likely to get around to it for a while. RE: [RELEASE] Texture Cache Maintenance utility - rainman74 - 2019-10-26 (2019-10-25, 01:30)Milhouse Wrote:That's no problem at all! Okay, I'll do that. But it will take a little longer, because in desperation I deleted my whole library & textures db and have it scrapened and cached again. This will take a little longer, because the scraping over SMB is hellish slow...(2019-10-21, 12:15)rainman74 Wrote:(2019-10-19, 07:18)Milhouse Wrote: Sorry for taking so long to reply. I've just downloaded your log.zip, but it looks like you've reformatted it (pretty printed the JSON data) which has kind of made it a bit useless to me as I need it in the original raw format - the texturecache.py script has a But maybe in the meantime you can tell me why I get this error message when caching music videos: Code: Traceback (most recent call last):0, multi thread 2288 In texturecache.cfg I use: Code: download.threads = 4 RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2019-10-28 (2019-10-26, 23:26)rainman74 Wrote: But maybe in the meantime you can tell me why I get this error message when caching music videos: Ah, that's a bug - I don't have any musicvideos so have done minimal testing with that media type. I'll push a fix shortly. RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2019-10-28 I've pushed v2.5.2 of the script, it adds the missing musicvideos key which hopefully fixes the issue you found.I've also add rudimentary support for pretty-formatted JSON responses so you should be able to upload a log I can use without disabling <compactoutput>false</compactoutput> .
RE: [RELEASE] Texture Cache Maintenance utility - rainman74 - 2019-10-28 (2019-10-28, 17:13)Milhouse Wrote: I've pushed v2.5.2 of the script, it adds the missingThank you! I was about to create a new log, but noticed that this time no cast thumbs were cached because I used the default config, damn! So I have to run this again with "cache.castthumb = yes" ... ;-) RE: [RELEASE] Texture Cache Maintenance utility - rainman74 - 2019-10-29 (2019-10-28, 17:13)Milhouse Wrote: I've pushed v2.5.2 of the script, it adds the missing So, after 2 days now all castthumbs were cached over SMB again, I started a test prune of the texture cache. Code: python texturecache.py c @profile.name=Eltern The program wants to delete 22,196 files after the test run (python texturecache.py p @profile.name=Eltern), although I read everything completely new and then cached. That can therefore definitely not be that these are all files that are no longer used. You can find the log here. What am I doing wrong? P.S. Caching of musicvideos works now! ;-() Update: Could it be because I renamed the "Master User" profile to "Eltern" and it doesn't find the correct path? "PROFILE_DIRECTORY": "profiles\\Eltern" <- does not exist and this is generated automatically. But if I start the program without the "@profile.name=Eltern", it doesn't work: --> ERROR: Profile "Master user" is not valid! Update II: I tested it once, but even this command doesn't work and there are still 22,196 files to delete: Code: python texturecache.py p @profile.name=Eltern @profile.directory=C:\Users\Karlheinz\AppData\Roaming\Kodi\userdata RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2019-11-02 Thanks for your patience. I've spent a few hours looking at this today, and I've found two bugs in my code, which I think I've fixed in this version (not yet merged): http://ix.io/20DH - you can try testing it, but I don't think it will make much difference for you (the first issue failed to process Extras correctly for your folder Der 90. Geburtstag oder Dinner for One (1963) , while the second issue related to incorrect handling of Windows paths on Linux which is only an issue when replaying a log file generated on a different platform - you're using Windows, I'm replaying your file on Linux etc.).With this updated version I've been able to replay your log, and it mostly wants to remove textures with paths starting P:\Kinder\ (Filme2, Serien, etc.) - the first 50 can be see here: http://ix.io/20DLI can also confirm that it wants to remove 22,196 items:
This all looks to be pretty much entirely correct, as these are cached textures for one of your other profiles ( Kinder )), as you are pruning while using the profile Eltern .I can't see how you can share a single texture cache (database + Thumbnails folder) for two profiles and avoid pruning the textures for the inactive profiles - each profile should be using it's own dedicated texture cache (ie. Textures13.db database and Thumbnails folder) which is synced to the media library (MyMusic72/MyMovies116 etc) that is used by the profile. It also wants to remove 1430 other textures, mostly starting with the path P:\Musik\ , these look like album cover art (folder.jpg, for albums you no longer have in your MyMusic media library). So pruning these textures would appear to be correct as well.
RE: [RELEASE] Texture Cache Maintenance utility - rainman74 - 2019-11-03 (2019-11-02, 23:59)Milhouse Wrote: ... Thank you very much for your research. First of all, the 1430 music files are all still there, but the program still wants to delete the textures. This still seems to be a bug to me. But now to the main problem. Now my guess seems to be correct with the error in the profile handling of Kodi! I had already observed this several times, but first I thought it couldn't be. But there is no error in your script! Here my observation:
What do you think, could that phenomenon be the cause? But that would then be a very unpleasant error in Kodi's caching! RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2019-11-03 (2019-11-03, 00:42)rainman74 Wrote: First of all, the 1430 music files are all still there, but the program still wants to delete the textures. This still seems to be a bug to me.There don't appear to be any media library items that reference those P:\Musik textures, unless it's some sort of assumed relationship. For example - these textures don't have matching media library entries:
so try texturecache.py jd albums Passenger and see if anything is found, as there's nothing in the media library for the Eltern profile. It might be an album in the media library of a different profile, which again would explain why the script wants to remove these textures when you are pruning using the Eltern profile.(2019-11-03, 00:42)rainman74 Wrote: What do you think, could that phenomenon be the cause? But that would then be a very unpleasant error in Kodi's caching!What you suggest is entirely possible - unfortunately the profile handling in Kodi has many problems and the plan is to rewrite/reimplement it from scratch, as it's considered beyond fixing in any sane way. Given that profile handling in Kodi is flaky at best, it might be necessary to avoid using the script to change profiles. Or try this (last throw of the dice): change the profile using the script then quit Kodi (something like texturecache.py @profile.name=Eltern power exit might work on Windows - ie. switch to the Eltern profile, and then exit/shutdown Kodi). Start kodi again (making sure it is configured to use the "Last used profile" on startup), and *then* cache or prune artwork for the active profile without changing the profile again, ie. texturecache.py @profile.enabled=no c movies would cache movie artwork for the currently active profile (ie. Eltern in this example).You can automate all of this using a Windows command script. You can also view the currently active profile with texturecache.py @profile.enabled=no status .
RE: [RELEASE] Texture Cache Maintenance utility - rainman74 - 2019-11-03 (2019-11-03, 01:38)Milhouse Wrote: ... Thanks for the tips. With the musik artworks you were right, since I had read all media folders again, I had forgotten the music folder with my parent master profile and had not read it yet (only with the children), therefore the deletion is correct, however, it is also so, that no music files exist in the parent profile, but nevertheless after the profile switch briefly erroneously in the master profile are cached! Since the erroneous behavior does not only happen with your script, but also quite normally when you switch the profiles in Kodi, it doesn't help me to pay attention to it when caching via texturecache.py, because at the latest with the next manual switch the artworks are again wrongly cached in the wrong profile. So I can only hope that this bug will soon be fixed in one of the upcoming Kodi 19 versions. |