[CrazyCanuck]

ok, I opened my db in SQLite browser and almost all urls are encoded. My album art is mostly embedded in music files and I noticed that URL entries for these were encoded like this: image://music@smb%3a%2f%2fNAS%2fMusic%2fThe%20Beatles%2fHelp!%2f01%20Help!.mp3/

Whereas any image files like .jpg had URL entries with no encoding like this:

smb://NAS/Music/Elton John/The One/Folder.jpg

Hope this helps.

[Milhouse]

ok, I opened my db in SQLite browser and almost all urls are encoded. My album art is mostly embedded in music files and I noticed that URL entries for these were encoded like this: image://music@smb%3a%2f%2fNAS%2fMusic%2fThe%20Beatles%2fHelp!%2f01%20Help!.mp3/

Whereas any image files like .jpg had URL entries with no encoding like this:

smb://NAS/Music/Elton John/The One/Folder.jpg

Hope this helps.

Ah right, yes, that's an "embedded" artwork item (music@, video@), and it's true that their urls are stored in an encoded form (why? no idea, for the sake of consistency they shouldn't be). However it's not possible for texturecache.py to pre-load the cache with such items so after you've deleted such an item you won't be able to re-cache it - you need to browse the GUI in order for the image to be extracted from the media item.

As you can see, with some urls encoded and the vast majority not, encoding the search url will make the situation worse - the correct solution is for urls to always be stored unencoded (will mention it to jmarsall).

[Milhouse]

I've opened Trac ticket #14905 to report the encoding discrepancy between embedded and non-embedded urls.

In the meantime, I've added a property @urlencode=yes|no (default no), which will encode the search term on the "s" option, matching against any encoded urls you may have. This may help you a bit, but the correct fix is to stop the urls from being encoded in the first place. The new property will be in the next release.

[GoogleRocks!]

Hi MilhouseVH,

Admittedly, I'm somewhat new to this and have spent most of my energy on the Video aspects of XBMC over the past year. Recently I've started to divert some of my time towards the Music side, although I did do an initial Library setup in the beginning, importing and scanning my iTunes directory structure into the Media Center as well. Somewhere along the way, possibly through acts of commission, my music library has become mucked-up. The Albums node seems to be okay, however in the Artists node, I have hundreds of artists whose thumbnail is defaulting to a soundtrack album thumb from "Bad Boys II - featuring Nelly, P.Diddy and Murphy Lee" and some others defaulting to "Skyfall". However most of the other fanarts seem okay. Looking into the MySql 5.5 - MyMusic44.db, I see hundreds if not more, of a "folder.jpg" image referenced in the Art and other related tables. I've searched the entire HTPC for the physical presence of such a file but have come up empty-handed. So, obviously (to me) it (they) must be cached in the Textures16.db cache file.

Came across your thread and and started reading, from the beginning until my eyes refused to go on, nearing the half way point, lol. Anyway, wanted to start to dig in and get my feet wet - I'm a visual kind of guy. Installed Python 3.3.3 and your script today, launched it without adding any command line parameters and right out of the gate, I got this error:

Code:

C:\Users\Gordon\AppData\Roaming\XBMC\system>python texturecache.py
Traceback (most recent call last):
  File "texturecache.py", line 6710, in <module>
    main(sys.argv[1:])
  File "texturecache.py", line 6458, in main
    loadConfig(argv)
  File "texturecache.py", line 6047, in loadConfig
    gConfig = MyConfiguration(argv)
  File "texturecache.py", line 124, in __init__
    config.readfp(cfg)
  File "C:\Python33\lib\configparser.py", line 753, in readfp
    self.read_file(fp, source=filename)
  File "C:\Python33\lib\configparser.py", line 708, in read_file
    self._read(f, source)
  File "C:\Python33\lib\configparser.py", line 1087, in _read
    fpname, lineno)
configparser.DuplicateOptionError: While reading from '<???>' [line 43]: option '<meta content' in section 'global' already exists

...same error with added parameter: python texturecache.py x "order by usecount desc" 2>%appdata%/xbmc/system/output/null

& @crazycanuck's post, I too have hundreds of embedded image url's within the music db as well, fyi.

I'm running Gotham (13 Alpha 12, Nightly) which was upgraded from Frodo 12.3/12.2/12.1 along the way, Python v3.3.3 on a Windows 8.1 64-bit HTPC, a second W8.1 client and several Android handheld devices all sharing the MySql database.

[Milhouse]

...same error with added parameter: python texturecache.py x "order by usecount desc" 2>%appdata%/xbmc/system/output/null

It looks like you have downloaded the texturecache.cfg file as a web page (if you view the texturecache.cfg file it should be a plan text file, yours is likely to be full of angled brackets).

I'd recommend deleting the texturecache.cfg file as you should only need it if/when you need to change a property - in many cases you should be able to get the script running using just the default settings (assuming you're running the script on the same machine as XBMC).

& @crazycanuck's post, I too have hundreds of embedded image url's within the music db as well, fyi.

I'm running Gotham (13 Alpha 12, Nightly) which was upgraded from Frodo 12.3/12.2/12.1 along the way, Python v3.3.3 on a Windows 8.1 64-bit HTPC, a second W8.1 client and several Android handheld devices all sharing the MySql database.

Yeah, it's an "issue" that continues to this day - with a bit of luck it may be possible to eliminate the encoded urls in a future Gotham release, but what you can do with these embedded artwork urls is severely limited as far as the script is concerned - there's no way to pre-load the cache with these items, unfortunately (by default any attempt to cache such an item will be ignored).

I just make sure I have cover.jpg artwork in each album folder, which is another way of eliminating the problem (while also meaning you are able to pre-load the cache with album artwork).

[GoogleRocks!]

Thanks for the speedy reply.

Yep, you're right, I wasn't paying attention on the right click. It surely is an html file!

python texturecache.py --loaded ok to the help list v1.4.4

python texturecache.py x "order by usecount desc" 2>%appdata%/xbmc/system/output/null
Stopped on a sql syntax error...

Code:

Loading database items...
Traceback (most recent call last):
  File "texturecache.py", line 6710, in <module>
    main(sys.argv[1:])
  File "texturecache.py", line 6484, in main
    sqlExtract("NONE", filter=argv[1])
  File "texturecache.py", line 4858, in sqlExtract
    dbrows = database.getRows(filter=SQL, order="ORDER BY t.cachedurl ASC", allfields=True)
  File "texturecache.py", line 1292, in getRows
    return self._transform(self._getAllColumns(filter, order))
  File "texturecache.py", line 1312, in _getAllColumns
    return self.execute(SQL).fetchall()
  File "texturecache.py", line 1262, in execute
    self.cursor.execute(SQL)
sqlite3.OperationalError: near "ORDER": syntax error

Thanks for the tip on Cover.jpg to the folder. If I wasn't clear previously, I did not create a Folder.jpg anywhere that I can recall, unless I did it way back in the beginning under an influence of confusion during early setup, lol.

I will continue to wade through this and get acquainted with your script. Presents itself as a vary powerful toolkit!

[Milhouse]

python texturecache.py x "order by usecount desc" 2>%appdata%/xbmc/system/output/null
Stopped on a sql syntax error...

Oops that's a bug that crept in some time ago and has gone unnoticed... I'll fix it in the next release. For now you can just edit texturecache.py and replace line 4858 as follows:

Code:

dbrows = database.getRows(filter=SQL, allfields=True)

[Milhouse]

In the meantime, I've added a property @urlencode=yes|no (default no), which will encode the search term on the "s" option, matching against any encoded urls you may have. This may help you a bit, but the correct fix is to stop the urls from being encoded in the first place. The new property will be in the next release.

I've come up with a workaround that doesn't require the @urlencode property - it will search the database twice, once for the non-encoded search term and a second time with the url encoded search term, then combine the results. Obviously it won't perform both searches if the search term is unchanged after being url encoded.

[trevez]

I ended up using SQLite browser to export the db to a .sql file and using that to create a new clone of the db. Xbmc and your script were both happy after that

Hello there!
I too am having this issue on my Atv2 running the db over a LAN. Any chance you can elaborate on what the fix is? Tried repairing tables in Mysql workbench but get 'myvideos75.actorlinkmovie', 'repair', 'note', 'The storage engine for the table doesn''t support repair'

cheers

[ghost832]

Hi, looks like this is going to be a great script to use, thanks for the time and the effort you have put in,

1 question ( probably a dumb one) but how are you finding out which art/backgrounds are corrupt ( black) ? other than going through each one individually. is there an automatic way of finding this info?

[Milhouse]

how are you finding out which art/backgrounds are corrupt ( black) ? other than going through each one individually. is there an automatic way of finding this info?

If the artwork is "black" it may be missing from the cache, which could be fixed by running the "c" option, however if the original artwork files are inaccessible (which is why they're not in the cache, and thus appearing as "black") then you need to fix that problem first - check your xbmc.log for errors.

If the artwork really is corrupt, ie. appears as multi-coloured garbage rather than just black, then no - you need eyes to detect this situation. However if you know the movie (or tvshow, or artist) associated with the artwork you can force the artwork to be re-cached eg. "C movies <moviename>".

[Ruschi]

I tried this on a raspberry running openelec 13.x nightly build and i get only this.

OpenELEC:~ # ./texturecache.py c
Traceback (most recent call last):
File "./texturecache.py", line 6710, in <module>
main(sys.argv[1:])
File "./texturecache.py", line 6462, in main
if not checkConfig(argv[0]): sys.exit(2)
File "./texturecache.py", line 6120, in checkConfig
data = jcomms.sendJSON(REQUEST, "libPing", checkResult=False, useWebServer=True)
File "./texturecache.py", line 1529, in sendJSON
data = self.sendWeb("POST", "/jsonrpc", json.dumps(request), {"Content-Type": "application/json"}, timeout=timeout)
File "./texturecache.py", line 1497, in sendWeb
raise httplib.HTTPException("Remote web host requires webserver.username/webserver.password properties")
httplib.HTTPException: Remote web host requires webserver.username/webserver.password properties

[Milhouse]

The answer is in the message... "Remote web host requires webserver.username/webserver.password properties"

Add your username and password to your properties file (texturecache.cfg) using the webserver.username/webserver.password properties.

[Milhouse]

texturecache.py (Version 1.4.5)

• Fix: Improve memory efficiency of JSON GetDirectory processing by limiting the directory cache to a fixed size - thanks @theowiesengrund for helping with testing
  
• Fix: Significantly improve memory efficiency of cast thumbnail processing - thanks @theowiesengrund for helping with testing
  
  
• Chg: Make @chunked=yes the default setting
  
• Chg: Enable HDMI with tvservice --preferred whenever xbmc quits, either normally or abnormally - this will ensure that HDMI is correctly re-enabled even after killall xbmc.bin etc.
  
• Chg: Alter missing behaviour to now consider only file types matching known audio or video file extensions (those supported by XBMC). This is to say the behaviour has changed from an exclusion to inclusion approach. Specify additional file extensions that should be included by adding comma-delimited values to audio.filetypes or video.filetypes properties. All standard XBMC audio and video file extensions are supported by default.
  
• Chg: Remove nonmedia.filetypes property as no longer has a purpose
  
• Chg: Display warning when a tv show with invalid season (season identifier < 0) is detected
  
• Chg: Search the texture cache database (s option) up to two 2 times, first with the non-encoded search term, and the second time after encoding the search term (and only then if the encoded term is different to the unencoded search term), then combining the results. This is necessary to find encoded urls used for embedded artwork (music@, video@ and also picture thumbnails)
  
  
• Rev: Revert the change that made @dbjson=no the default setting on localhost, as this should no longer be necessary when using chunked queries. When available, the TextureDB JSON API should always be used by default (whether remote or local).
  
  
• Add: When filtering media library results, the default filter field is usually title (eg. movies avatar or albums mothership). However should you wish to filter on an alternate field then specify the field name in the @filter property, eg. movies cameron @filter=director or albums "the beatles" @filter=artist
  
• Add: Specify alternate filter operator, the default opertator being contains. Use @filter.operator=<operator> where <operator> is any one of the standard filter operators. For example, jd movies 21 @filter.operator=is would return details only for the movie named "21", and not also "21 Jump Street" as would be the case when using the default contains operator.
  

[piotrasd]

sorry for stupid question, but somebody could give me config with settings for cache all posible things ?

im used something like this

sep = |
userdata = /storage/.xbmc/userdata
dbfile = Database/Textures13.db
thumbnails = Thumbnails
xbmc.host = localhost
webserver.port = 80
webserver.username = xbmc
webserver.password =
rpc.port = 9090
download.threads = 5
download.predelete = no
download.payload = yes
extrajson.albums = None
extrajson.artists = None
extrajson.songs = None
extrajson.movies = trailer, streamdetails, file
extrajson.sets = None
#extrajson.tvshows.tvshow = None
#extrajson.tvshows.season = None
#extrajson.tvshows.episode = None
qaperiod = 30
qa.rating = yes
qa.file = yes
qa.art.movies = thumb, fanart, clearlogo, poster, landscape, extrafanart, cast.thumb, characterart
cache.artwork = thumb, fanart, clearlogo, poster, landscape, extrafanart, cast.thumb, characterart
cache.videoextras = yes
cache.extra = yes
cache.castthumb = yes
cache.ignore.types = none
cache.extrafanart = yes
cache.extrathumbs = yes
logfile = /storage/test.log

but i get almost 90% ignored even if i use "C all" with small c also (after delate textures.db also )
(the most strage thing is, even if finish caching once and try one more time then start from 0 - when im use small "c" )

my log (2 min of work)
https://copy.com/7xqVZRn3oFu0DpFA