Kodi Community Forum
[RELEASE] Texture Cache Maintenance utility - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for Kodi (https://forum.kodi.tv/forumdisplay.php?fid=116)
+--- Thread: [RELEASE] Texture Cache Maintenance utility (/showthread.php?tid=158373)



RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2019-02-15

(2019-02-15, 13:29)Yuren Wrote: However I have again many problems
When I run  texturecache.py c movies no actor thumb is downloaded

After many attemps and investigations I understand that Kodi, maybe, isn't able to locate the files (even if I can play them)

I ran this command texturecache.py qa and these are some lines of the report
Movie    [Tropic Thunder                                    ]: WARN (missing cast, local not found; missing file)
Episode  [X-Files, Stagione 11 Episode 11x10                ]: WARN (missing file)
TVShow   [X-Files                                           ]: WARN (missing cast, local not found)



I don't know how to fix my library

Yes, it seems like Kodi can't access the files, you should check if the files exist in your file system and assuming they do, check that you have a suitable source defined in Kodi (sources.xml).

The missing cast information is a scraper issue, and not something the script can help with other than to tell you the information is missing (not scraped). You'll need to re-scrape the movie with cast information.

You can view the path being used for one of the problem movies with texturecache.py jd movies "Tropic Thunder" - look at the file property.

If you want to view more information, such as cast information, use texturecache.py @extrajson.movies=cast Jd movies "Tropic Thunder".

(2019-02-15, 13:29)Yuren Wrote: ps:   Before you responded to me I have created a new Kodi user profile directory and I have imported the library. After that, I have run  texturecache.py c movies  and this time your script downloaded all actor thumbs and then again freeze on  W:\\torrent\\[TV Rip ITA}Alla Ricerca Della Valle Incantata 2 - Le Avventure Della Grande Vallata

You probably have that path still in your library - without another tc.log I can't know for sure.

You could try running the following query which will list all movies with } in the media (file) path:
text:

texturecache.py query movies "file contains }"
Rename the folder in your file system, remove the movie from your library then rescan it back in with the new path.


RE: [RELEASE] Texture Cache Maintenance utility - bob77 - 2019-02-15

(2019-02-15, 01:38)Milhouse Wrote:
(2019-02-15, 00:56)bob77 Wrote: hey man,
I have a bunch of missing thumbnails. Ive been trying to fix it with "texturecache.py c" but it just gives me an error on every download.
I dont know where to find the tc.log but I did find the kodi log and all it says is
00:44:16.868 T:7276   DEBUG: CUtil::GetMatchingSource: no matching source found for [E:\-=Movies=-\.....
and so for every entry.
any ideas?
Thanks!

See messages above yours about where to put the tc.log.

As for your errors, can't really help much without a log but I assume you don't have the paths for your movies in your sources.xml so Kodi can't/won't read the directory contents for security reasons. Or the path is no longer accessible to Kodi because you've deleted/moved/renamed/offlined the storage.
[/quote]
hey man,
I read posts further back and I finally understood that the log file location must be set in the config file.
so here it is:
https://drive.google.com/file/d/1alBj7UtiElIil84puz9RCQBBnfE4sKOC/view?usp=sharing
its pretty big but I hope you can make something out of it and tell me why Im getting the errors on the "c" command.
thanks!! Smile


RE: [RELEASE] Texture Cache Maintenance utility - Yuren - 2019-02-15

Thanks again,
Yes, I solved with the re-scrape of the library.

Do you know the addon Extended Info?
Is it possible use your script to scan the library with this addon and download all extra informations?


RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2019-02-16

(2019-02-15, 21:21)Yuren Wrote: Do you know the addon Extended Info?
Is it possible use your script to scan the library with this addon and download all extra informations?

You might have more luck with the Artwork Beef add-on.


RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2019-02-16

@bob77

So looking at your log I see a lot of these errors:
text:

2019-02-15 20:51:19.342492:MainThread: libDirectory.JSON SOCKET REQUEST: [{"method": "Files.GetDirectory", "params": {"directory": "E:\\-=Movies=-\\-=Other=-\\2 Guns (2013)1080p.BluRay.x265.HEVC.10bit.DTS.(xxxpav69)\\", "media": "files", "properties": ["file", "lastmodified"]}, "jsonrpc": "2.0", "id": "libDirectory"}]
2019-02-15 20:51:19.343532:MainThread: libDirectory.BUFFER RECEIVED (len 89)
2019-02-15 20:51:19.343532:MainThread: libDirectory.PARSING JSON DATA: {"error":{"code":-32602,"message":"Invalid params."},"id":"libDirectory","jsonrpc":"2.0"}
...
2019-02-15 20:51:19.351470:MainThread: libDirectory.JSON SOCKET REQUEST: [{"method": "Files.GetDirectory", "params": {"directory": "E:\\-=Movies=-\\-=Comedy=-\\30.Minutes.or.Less.2011.1080p.BluRay.x265.HEVC.10bit.DTS(xxxpav69)\\", "media": "files", "properties": ["file", "lastmodified"]}, "jsonrpc": "2.0", "id": "libDirectory"}]
2019-02-15 20:51:19.351470:MainThread: libDirectory.BUFFER RECEIVED (len 89)
2019-02-15 20:51:19.351470:MainThread: libDirectory.PARSING JSON DATA: {"error":{"code":-32602,"message":"Invalid params."},"id":"libDirectory","jsonrpc":"2.0"}

This is the script asking Kodi to traverse the folders E:\-=Movies=-\-=Other=- and E:\-=Movies=-\-=Comedy=- which Kodi is unable to traverse, possibly because you don't have the E:\-=Movies=- path in your sources.xml which is a new security feature in Kodi 18, so add the missing movie source if you want to allow external programs access to the directory and sub-directories.

Or possibly the path no longer exists or is no longer accessible to Kodi, in which case... you'll need to fix that.

It looks like the script is able to access F:\-=Video=-\Movies\-=Other=- (etc.) so either that path is in your sources.xml, or it is available/accessible to Kodi - it's just the E:\-=Movies=- path that Kodi is not allowed/not able to access.

Your poster artwork:
text:

C:\Users\Dr3adLoX\Desktop\kodi backup\kodi_videodb_2019-02-08\movies\2_Guns_2013-fanart.jpg
Check your kodi.log for errors, as Kodi is not able to access this path - missing, deleted, offline, not in sources.xml, I can't tell.


RE: [RELEASE] Texture Cache Maintenance utility - Yuren - 2019-02-16

I never had movies in E:\  nor in F:\  They are stored only in W:\
There is the library backup in E:\


Before I imported my library I manually edited the sources.xml to add the movies paths. However this seems not having effect since when I start Kodi in the library preferences there isn't any movies path


In texturecache.cfg  what's mean cache.videoextras?


RE: [RELEASE] Texture Cache Maintenance utility - Yuren - 2019-02-16

(2019-02-16, 03:26)Milhouse Wrote: Your poster artwork:
text:

C:\Users\Dr3adLoX\Desktop\kodi backup\kodi_videodb_2019-02-08\movies\2_Guns_2013-fanart.jpg
Check your kodi.log for errors, as Kodi is not able to access this path - missing, deleted, offline, not in sources.xml, I can't tell. 
 I checked lthe debug log and now there isn't that error anymore


RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2019-02-16

@Yuren sorry that reply is to @bob77 - I've edited the post.


RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2019-02-16

(2019-02-16, 08:47)Yuren Wrote: Before I imported my library I manually edited the sources.xml to add the movies paths. However this seems not having effect since when I start Kodi in the library preferences there isn't any movies path

Not really sure what you mean.

(2019-02-16, 08:47)Yuren Wrote: In texturecache.cfg  what's mean cache.videoextras?

https://github.com/robwebset/script.videoextras/wiki

Artwork found in the "extras" subdirectory will be cached.


RE: [RELEASE] Texture Cache Maintenance utility - bob77 - 2019-02-16

(2019-02-16, 03:26)Milhouse Wrote: @bob77

So looking at your log I see a lot of these errors:
text:

2019-02-15 20:51:19.342492:MainThread: libDirectory.JSON SOCKET REQUEST: [{"method": "Files.GetDirectory", "params": {"directory": "E:\\-=Movies=-\\-=Other=-\\2 Guns (2013)1080p.BluRay.x265.HEVC.10bit.DTS.(xxxpav69)\\", "media": "files", "properties": ["file", "lastmodified"]}, "jsonrpc": "2.0", "id": "libDirectory"}]
2019-02-15 20:51:19.343532:MainThread: libDirectory.BUFFER RECEIVED (len 89)
2019-02-15 20:51:19.343532:MainThread: libDirectory.PARSING JSON DATA: {"error":{"code":-32602,"message":"Invalid params."},"id":"libDirectory","jsonrpc":"2.0"}
...
2019-02-15 20:51:19.351470:MainThread: libDirectory.JSON SOCKET REQUEST: [{"method": "Files.GetDirectory", "params": {"directory": "E:\\-=Movies=-\\-=Comedy=-\\30.Minutes.or.Less.2011.1080p.BluRay.x265.HEVC.10bit.DTS(xxxpav69)\\", "media": "files", "properties": ["file", "lastmodified"]}, "jsonrpc": "2.0", "id": "libDirectory"}]
2019-02-15 20:51:19.351470:MainThread: libDirectory.BUFFER RECEIVED (len 89)
2019-02-15 20:51:19.351470:MainThread: libDirectory.PARSING JSON DATA: {"error":{"code":-32602,"message":"Invalid params."},"id":"libDirectory","jsonrpc":"2.0"}

This is the script asking Kodi to traverse the folders E:\-=Movies=-\-=Other=- and E:\-=Movies=-\-=Comedy=- which Kodi is unable to traverse, possibly because you don't have the E:\-=Movies=- path in your sources.xml which is a new security feature in Kodi 18, so add the missing movie source if you want to allow external programs access to the directory and sub-directories.

Or possibly the path no longer exists or is no longer accessible to Kodi, in which case... you'll need to fix that.

It looks like the script is able to access F:\-=Video=-\Movies\-=Other=- (etc.) so either that path is in your sources.xml, or it is available/accessible to Kodi - it's just the E:\-=Movies=- path that Kodi is not allowed/not able to access.

Your poster artwork:
text:

C:\Users\Dr3adLoX\Desktop\kodi backup\kodi_videodb_2019-02-08\movies\2_Guns_2013-fanart.jpg
Check your kodi.log for errors, as Kodi is not able to access this path - missing, deleted, offline, not in sources.xml, I can't tell.

Milhouse,
thank you very much for trying to help me.
let me clarify what I did.
I had E: 2TB, F: 2TB, and G: 500GB
I bought a 4TB and copied both F: and G: to that drive
the new 4TB is now designated as F:
So basically the only change occurred for G:
I created a Kodi backup and edited the .xml (if Im not mistaken the name was library.xml) where I replaced every G: with F:
so everything should match
I also simply removed the entry of G:\-=Movies=- from sources.xml because it no longer exists and F:\-=Movies entry should be sufficient
and I restored the Kodi backup
here is how source.xml looks right now:
Quote:<sources>
    <programs>
        <default pathversion="1"></default>
    </programs>
    <video>
        <default pathversion="1"></default>
        <source>
            <name>Drive 2 - TV Docu</name>
            <path pathversion="1">F:\-=Video=-\Docu\-=TV=-\</path>
            <allowsharing>true</allowsharing>
        </source>
        <source>
            <name>Drive 2 - Docu</name>
            <path pathversion="1">F:\-=Video=-\Docu\</path>
            <allowsharing>true</allowsharing>
        </source>
        <source>
            <name>Drive 2 - TV</name>
            <path pathversion="1">F:\-=Video=-\TV\</path>
            <allowsharing>true</allowsharing>
        </source>
        <source>
            <name>Drive 1 - Movies</name>
            <path pathversion="1">E:\-=Movies=-\</path>
            <allowsharing>true</allowsharing>
        </source>
        <source>
            <name>Drive 2 - Movies</name>
            <path pathversion="1">F:\-=Video=-\Movies\</path>
            <allowsharing>true</allowsharing>
        </source>

    </video>
    <music>
        <default pathversion="1"></default>
    </music>
    <pictures>
        <default pathversion="1"></default>
    </pictures>
    <files>
        <default pathversion="1"></default>
    </files>
    <games>
        <default pathversion="1"></default>
    </games>
</sources>

Both "Movies" paths are completely accurate and working
I just tried playing the 2 Guns movie through Kodi and it does find the source and play it with no problem.
I can also click on the choose art button and add new art to it. thats working also.

initially when I restored the backup I did have all my posters but I used the "P" command to do a cleanup and that removed my actor images which I understand now is because I did not have "cache.castthumb = yes" in the cfg, and so I tried to get those actor images back by using the "C" command. Thats when I lost the posters on half the movies and can no longer get the texturecache.py to re-download them.

I have no idea what could be wrong. only thing Im noticing is that the log that I posted has a double slash in the path "E:\\-=Movies=-\\-=Comedy=-\\" dont know if thats normal.

I hope all this can give you an idea of what went wrong and come up with some suggestions...
Thanks again!


[RELEASE] Texture Cache Maintenance utility - Milhouse - 2019-02-16

@bob77 have a look in your kodi.log for errors as it's Kodi that is having the problem accessing the E:\ path - the script is trying to access the path "through" Kodi but Kodi isn't working and there's no way for the script to know what the problem is as this isn't reported by Kodi to the script - all the script gets from Kodi is a pretty useless generic "invalid params" error.

Double backslashes are normal when dealing with Windows file names.


RE: [RELEASE] Texture Cache Maintenance utility - bob77 - 2019-02-16

(2019-02-16, 11:24)Milhouse Wrote: @bob77 have a look in your kodi.log for errors as it's Kodi that is having the problem accessing the E:\ path - the script is trying to access the path "through" Kodi but Kodi isn't working and there's no way for the script to know what the problem is as this isn't reported by Kodi to the script - all the script gets from Kodi is a pretty useless generic "invalid params" error.

Double backslashes are normal when dealing with Windows file names.
here is the full kodi log:

https://drive.google.com/file/d/1zwpRILAOk4N7YUt51UNX0YC-yo1_xuyO/view?usp=sharing

and the texturecache.log

https://drive.google.com/file/d/1t0MvJ0rZXPAvAWT8luqkqfMt3NxJRuFb/view?usp=sharing

of my latest run of the "c" command.
All I see in kodi log is the "no matching source found" and all those paths that are listed I can copy and paste straight into win explorer and it opens right up completely working.


RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2019-02-16

@bob77 the problem is definitely the fact that Kodi 18 is not matching the requested Files.GetDirectory path with a source in your sources.xml.

Did you restart Kodi after adding the new source to sources.xml, as sources.xml is only read once at startup.

If you have restarted kodi then try adding:
text:

<source>
<name>Drive 1 - Movies</name>
<path pathversion="1">E:\\-=Movies=-\\</path>
<allowsharing>true</allowsharing>
</source>
and see if that helps (this is just a reach, as I have no other ideas).

I've tested adding a new source to sources.xml and this is working on Linux.

Before I added it, a Files.GetDirectory query would fail:
text:

$ texturecache.py directory /var
Directory "/var" not found.
[/syntaxt]

and checking the texturecache log:

[syntax=text]
2019-02-16 16:15:18.904981:MainThread: libDirectory.JSON SOCKET REQUEST: [{"jsonrpc": "2.0", "params": {"directory": "/var", "media": "files", "properties": ["file", "lastmodified"]}, "method": "Files.GetDirectory", "id": "libDirectory"}]
2019-02-16 16:15:18.905551:MainThread: RPC connection established with IPv4
2019-02-16 16:15:18.932971:MainThread: libDirectory.BUFFER RECEIVED (len 89)
2019-02-16 16:15:18.933073:MainThread: libDirectory.PARSING JSON DATA: {"error":{"code":-32602,"message":"Invalid params."},"id":"libDirectory","jsonrpc":"2.0"}

the expected Invalid params. response is seen. And in kodi.log:

text:

2019-02-16 16:15:18.907 T:1617810288 DEBUG: CUtil::GetMatchingSource: no matching source found for [/var]

Next, I edited sources.xml and added the new video source:
text:

<sources>
<video>
...
<source>
<name>Var</name>
<path pathversion="1">/var/</path>
<allowsharing>true</allowsharing>
</source>
...
</video>
</sources>

and after restarting Kodi, the Files.GetDirectory query works as expected:
text:

$ texturecache.py directory /var
DIR : /var/cache
DIR : /var/empty
DIR : /var/lib
DIR : /var/lock
DIR : /var/log
DIR : /var/media
DIR : /var/run
DIR : /var/spool
DIR : /var/tmp

If this process isn't working for you on Windows, then maybe there is a problem with Windows and it should be reported as a Kodi bug.


RE: [RELEASE] Texture Cache Maintenance utility - bob77 - 2019-02-16

I looked around and I am certain I just found the problem. Although I have no idea how this happened but it must be related to the algorithm of the "C" command.
The problem is this:
Image

there is a "\...\" in the middle of the path of every movie which definitely should not be there.
or at least thats what it shows in Kodi.
which seems to be ignored by Kodi itself but your script doesnt ignore it.
I will try to repair it by exporting, editing library.xml, and importing.



EDIT:

So after exporting I examined the videodb.xml (actual name of library.xml) and didnt find any "\...\"
so I decided to just try importing again.
that fixed it and posters started to update on their own. I ran the "c" command and thats working great also.
however, some of the paths which had \...\ no longer have it but some still do. some even have a double or triple one:
Image

and even so they still play straight away with no problems.
so I am willing to leave this one for the X files but if you have any ideas of the cause and how to clean it up I would be interested to find out.
either way, thanks a lot for your help and the great script you wrote. Smile


RE: [RELEASE] Texture Cache Maintenance utility - Milhouse - 2019-02-16

(2019-02-16, 20:33)bob77 Wrote: there is a "\...\" in the middle of the path of every movie which definitely should not be there.

The "\...\" is known as ellipsis and is shown when the path is very long and a portion of the path in the middle has been omitted so that the start and end may be shown.

It's a bit odd of Kodi to show three ellipsis as just one would suffice (maybe the path shortening algorithm is replacing each sub-dir with one "\...\") but the use of the ellipsis affects only the displayed path and Kodi will be using the full path (without any "\...\") behind the scenes.