Kodi Community Forum
[RELEASE] Texture Cache Maintenance utility - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for Kodi (https://forum.kodi.tv/forumdisplay.php?fid=116)
+--- Thread: [RELEASE] Texture Cache Maintenance utility (/showthread.php?tid=158373)



RE: [RELEASE] Texture Cache maintenance utility - nsviper - 2013-04-19

Quote:root@xbian:~# ./texturecache.py jd movies Cloverfield
[
{
"movieid": 48,
"title": "Cloverfield",
"art": {
"fanart": "image://nfs://192.168.0.100/MediaShare/Movies/HD/Cloverfield/Cloverfield-fanart.jpg/",
"discart": "image://http://assets.fanart.tv/fanart/movies/7191/moviedisc/cloverfield-512206b21f9dc.png/",
"clearlogo": "image://http://assets.fanart.tv/fanart/movies/7191/hdmovielogo/cloverfield-505d704506228.png/",
"clearart": "image://http://assets.fanart.tv/fanart/movies/7191/hdmovieclearart/cloverfield-505859697da10.png/",
"poster": "image://nfs://192.168.0.100/MediaShare/Movies/HD/Cloverfield/Cloverfield-poster.jpg/",
"banner": "image://http://assets.fanart.tv/fanart/movies/7191/moviebanner/cloverfield-50eb6a84a6246.jpg/",
"landscape": "image://http://assets.fanart.tv/fanart/movies/7191/moviethumb/cloverfield-505d9ec87bf6b.jpg/"
},
"label": "Cloverfield"
}
]

Ok so if I want to limit the scan so that only poster and fanart are checked for / queued / download (frmo local), are there settings in the .cfg file I need? Smile


RE: [RELEASE] Texture Cache maintenance utility - Milhouse - 2013-04-19

(2013-04-19, 19:44)nsviper Wrote: Ok so if I want to limit the scan so that only poster and fanart are checked for / queued / download (frmo local), are there settings in the .cfg file I need? Smile

If you only want to pre-load local content then the following added to the properties file should do the trick:
Code:
cache.ignore.types = image://video, image://music, image://http



RE: [RELEASE] Texture Cache maintenance utility - Milhouse - 2013-04-19

(2013-04-19, 17:26)3dfx Wrote: That would be nice, I really want to clean up the mess in the Thumbnails folder Smile

@3dfx: Can you enable PM, I'm trying to send you a PM and it's telling me you have it disabled... Smile


RE: [RELEASE] Texture Cache maintenance utility - 3dfx - 2013-04-20

(2013-04-19, 21:38)MilhouseVH Wrote:
(2013-04-19, 17:26)3dfx Wrote: That would be nice, I really want to clean up the mess in the Thumbnails folder Smile

@3dfx: Can you enable PM, I'm trying to send you a PM and it's telling me you have it disabled... Smile

Why, why did I do that?
I didn't..... I guess Smile

It's enabled now Blush


RE: [RELEASE] Texture Cache maintenance utility - charrua - 2013-04-20

Trying the last version (0.5.3) here.
Is it supposed to work with Python 3.3.1?
Cause it's returning this error:
Code:
texturecache.py version
Traceback (most recent call last):
  File "texturecache.py", line 2896, in <module>
    main(sys.argv[1:])
  File "texturecache.py", line 2722, in main
    loadConfig()
  File "texturecache.py", line 2540, in loadConfig
    gConfig = MyConfiguration()
  File "texturecache.py", line 185, in __init__
    if (web_user and web_pass): self.WEB_AUTH_TOKEN = base64.encodestring('%s:%s' % (web_user, web_pass)).replace('\n', '')
  File "C:\Python\lib\base64.py", line 355, in encodestring
    return encodebytes(s)
  File "C:\Python\lib\base64.py", line 343, in encodebytes
    raise TypeError("expected bytes, not %s" % s.__class__.__name__)
TypeError: expected bytes, not str

THX!


RE: [RELEASE] Texture Cache maintenance utility - Milhouse - 2013-04-20

(2013-04-20, 23:24)charrua Wrote: Trying the last version (0.5.3) here.
Is it supposed to work with Python 3.3.1?
Cause it's returning this error:

Good spot... Smile Fixed in v0.5.4 (just pushed to github).

v0.5.4
* Fixed sqlite3 characterset decode issue

* Default value for singlethread.urls is now assets\.fanart\.tv, to avoid hammering fanart.tv (logos, clearart, discart, etc.) as this site appears to reject multiple concurrent requests

* Write command line args and current version to logfile (if logfile is enabled)

* Added orphan.limit.check property to allow disabling of safety check when removing orphan files - default value is yes (safety check enabled).

* Updated directory traversing code to traverse an arbitrary number of directory levels (r, R, p, P) - previously limited to 2-3 directory levels.

* Fix base64.encodestring() in Python3

@3dfx - you can auto-update using the "fupdate" option (force update - ignores current version). Many thanks for your help with testing!


RE: [RELEASE] Texture Cache maintenance utility - charrua - 2013-04-21

(2013-04-20, 23:48)MilhouseVH Wrote:
(2013-04-20, 23:24)charrua Wrote: Trying the last version (0.5.3) here.
Is it supposed to work with Python 3.3.1?
Cause it's returning this error:
Good spot... Smile Fixed in v0.5.4 (just pushed to github).
Great job as always.


RE: [RELEASE] Texture Cache maintenance utility - Milhouse - 2013-04-21

FYI, v0.5.4 included a bug in the nc option that prevented the "Need to cache" message from appearing... fixed in latest version.

Version 0.5.5
* Fix: "Need to cache" message not appearing (lnc, nc)
* Fix: Not processing all seasons correctly during lnc and lc
* Modified lnc/lc to discard tvshow seasons that do not have a new episode - previously considered for caching all seasons of a tvshow with at least one new episode
* Show summary of recently added movies/tvshows (lnc, lc)
* Added lastrunfile date/time to stats summary information (lnc, lc)


RE: [RELEASE] Texture Cache maintenance utility - zestef - 2013-04-27

Hi,

Great script, but no success for me in trying it (cleaning sets)
Platform MacOs 10.8 / Python 2.7.2 (is 2.7.3 mandatory ?) / specific texturecache.cfg file.

Code:
Zephyr:downloads zestef$ ./texturecache.py C sets "Ace Ventura"
Traceback (most recent call last):
  File "./texturecache.py", line 2968, in <module>
    main(sys.argv[1:])
  File "./texturecache.py", line 2873, in main
    jsonQuery("cache", argv[1], argv[2], force=True)
  File "./texturecache.py", line 1670, in jsonQuery
    (section_name, title_name, id_name, data) = jcomms.getData(action, mediatype, filter, extraFields, lastRun = lastRun)
  File "./texturecache.py", line 1339, in getData
    return (SECTION, TITLE, IDENTIFIER, self.sendJSON(REQUEST, "lib%s" % mediatype.capitalize()))
  File "./texturecache.py", line 892, in sendJSON
    raise socket.error("Socket IO timeout exceeded")
socket.error: Socket IO timeout exceeded
Zephyr:downloads zestef$

LogFile :
Code:
2013-04-27 22:51:41.237794:MainThread: Command line args: ['./texturecache.py', 'C', 'sets', 'Ace Ventura']
2013-04-27 22:51:41.237832:MainThread: Current version #: 0.5.6
2013-04-27 22:51:41.240558:MainThread: libPing.JSON SOCKET REQUEST: [{"jsonrpc": "2.0", "method": "JSONRPC.Ping", "id": "libPing"}]
2013-04-27 22:51:41.240732:MainThread: libPing.BUFFER RECEIVED (len 48)
2013-04-27 22:51:41.240750:MainThread: libPing.PARSING JSON DATA: {"id":"libPing","jsonrpc":"2.0","result":"pong"}
2013-04-27 22:51:41.240791:MainThread: libPing.PARSING COMPLETE, elapsed time: 0.000019 seconds
2013-04-27 22:51:41.240815:MainThread: libPing.FINISHED, elapsed time: 0.000233 seconds
2013-04-27 22:51:41.240845:MainThread: libVersion.JSON SOCKET REQUEST: [{"jsonrpc": "2.0", "method": "JSONRPC.Version", "id": "libVersion"}]
2013-04-27 22:51:41.241005:MainThread: libVersion.BUFFER RECEIVED (len 88)
2013-04-27 22:51:41.241022:MainThread: libVersion.PARSING JSON DATA: {"id":"libVersion","jsonrpc":"2.0","result":{"version":{"major":6,"minor":0,"patch":3}}}
2013-04-27 22:51:41.241061:MainThread: libVersion.PARSING COMPLETE, elapsed time: 0.000022 seconds
2013-04-27 22:51:41.241080:MainThread: libVersion.FINISHED, elapsed time: 0.000218 seconds
2013-04-27 22:51:41.241346:MainThread: EXECUTING SQL: SELECT idVersion FROM version
2013-04-27 22:51:41.939389:MainThread: libSets.JSON SOCKET REQUEST: [{"jsonrpc": "2.0", "params": {"sort": {"order": "ascending", "method": "title"}, "properties": ["title", "art"]}, "method": "VideoLibrary.GetMovieSets", "id": "libSets"}]
2013-04-27 22:51:42.086453:MainThread: libSets.BUFFER RECEIVED (len 32664)
2013-04-27 22:51:42.086502:MainThread: libSets.BUFFER RECEIVED (len 18696)
2013-04-27 22:51:42.086593:MainThread: libSets.PARSING JSON DATA: {"id":"libSets","jsonrpc":"2.0","result":{"limits":{"end":124,"start":0,"total":124},"sets":[{"art":{"fanart":"image://smb%3a%2f%2fSRV-MEDIA%2fVideos%2fBoxset%2fAce%20Ventura%2fAce%20Ventura%20en%20Afrique%20%5b1995%5d%2fAce%20Ve (truncated)
2013-04-27 22:51:42.086925:MainThread: libSets.Incomplete JSON data - continue reading socket

Any idea ?
(Tried to increase timeout on line 772 without success)

Regards.
Zestef.


RE: [RELEASE] Texture Cache maintenance utility - Milhouse - 2013-04-27

Is this repeatable? If you are not using MySQL, could you send me your MyVideos75.db (PM me a download link etc.)?

Another option is to enable "logfile.verbose=yes" in the properties file and send me a link to the resulting log file which will contain a LOT more information (don't paste it here!), although sending me your MyVideos75.db would be preferable, particularly if this is repeatable.

Increasing the timeout won't do any good as it's a JSON data conversion issue - it thinks it's received a partial/incomplete JSON response and is trying to read the socket for more data which never comes. I need to understand why the JSON response is not considered valid.


RE: [RELEASE] Texture Cache maintenance utility - Milhouse - 2013-05-02

Version 0.5.6

* Change: Use local time and not UTC for lastrunfile timestamp (media library dateadded appears to be using local time, so this is more consistent)

Version 0.5.7
* Don't check for error during title lookup - if an OnRemove notification is received, the item being removed may already have been removed from the media library before the lookup is executed (but sometimes not). Return None for the title whenever an item no longer exists.
* Add power option, supporting states of suspend, hibernate, reboot or shutdown. This allows the XBMC client to be rebooted, shutdown etc.
* Add albums, artists and songs support to qa option (but no qax support as music items can't be removed from the media library).
* Add QA checks for artwork urls during qa/qax, failing/warning QA if found. Default fail urls are "image://video, image://music", there is no default "warn" url. Specify alternative urls using qa.fail.urls and qa.warn.urls properties (comma delimited patterns). Rescan will be triggered only for fail, not warn.
* Ignore JSON encode errors (should fix problem encountered by zestef - many thanks for helping out with testing).


RE: [RELEASE] Texture Cache maintenance utility - charrua - 2013-05-02

Thanks for the update.
I found this error when using option qa for either albums, artists or songs:
Code:
texturecache.py qa songs
Song ['a Tazza 'e Cafe'                                 ]: Missing fanart3bc\u03b1 (Part 2)]...
Song ['Na Musica - Dicitencello Vule                    ]: Missing fanart
.
.
.
Song [Africa                                            ]: Missing fanart
Traceback (most recent call last):
  File "D:\programas\multimedia\XBMC\utilidades\Texture Cache maintenance utility\texturecache.py", line 3019, in <module>
    main(sys.argv[1:])
  File "D:\programas\multimedia\XBMC\utilidades\Texture Cache maintenance utility\texturecache.py", line 2945, in main
    jsonQuery("qa", argv[1])
  File "D:\programas\multimedia\XBMC\utilidades\Texture Cache maintenance utility\texturecache.py", line 1758, in jsonQuery
    qaData(mediatype, jcomms, database, data, title_name, id_name, rescan)
  File "D:\programas\multimedia\XBMC\utilidades\Texture Cache maintenance utility\texturecache.py", line 2135, in qaData
    for m in mediaitems: gLogger.out("%s\n" % m)
  File "D:\programas\multimedia\XBMC\utilidades\Texture Cache maintenance utility\texturecache.py", line 407, in out
    sys.stdout.write("%-s" % udata)
  File "C:\Python\lib\encodings\cp850.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_map)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u02dc' in position 17: character maps to <undefined>



RE: [RELEASE] Texture Cache maintenance utility - Milhouse - 2013-05-02

/Milhouse gently weeps...

Is this Python 3.3 by any chance?

Can you provide me a link to your MyMusic32.db as although I have some foreign characters in my Music library, none of them are triggering this problem in Windows with either Python 2.7.x or Python 3.3.


Re: RE: [RELEASE] Texture Cache maintenance utility - charrua - 2013-05-03

(2013-05-02, 20:45)MilhouseVH Wrote: /Milhouse gently weeps...
Is this Python 3.3 by any chance?
yes, it's Python 3.3.1 as stated in a previous post.

(2013-05-02, 20:45)MilhouseVH Wrote: Can you provide me a link to your MyMusic32.db as although I have some foreign characters in my Music library, none of them are triggering this problem in Windows with either Python 2.7.x or Python 3.3.
I use MySQL for the DB so I don't have MyMusic32.db and I have a big music library so the music DB dump from MySQL is about 60Mb.
I'll try to pinpoint the albums, artists and songs that trigger this error and let you know.
Thanks


RE: [RELEASE] Texture Cache maintenance utility - Milhouse - 2013-05-03

(2013-05-03, 12:09)charrua Wrote: I use MySQL for the DB so I don't have MyMusic32.db and I have a big music library so the music DB dump from MySQL is about 60Mb.
I'll try to pinpoint the albums, artists and songs that trigger this error and let you know.
Thanks

Much appreciated.

You can supply a filter to qa, eg. "qa songs africa" (although it's the song after Africa that is causing the problem) which should speed things up. Partial filter matches also work. If you enable the logfile and logfile.verbose properties, the logfile information might help with any diagnosis/solution.

With Python2 I've been able to replace stdout and stderr with Unicode-savvy codecs, but have so far failed to get this to work in Python3 which explains why you're hitting this - I'll keep looking for a solution though...