[K20evo]

That leaves your router as the possible culprit, and did you try a different ethernet cable? Also, make sure you don't also have a WiFi dongle installed, even if you're not using it.

Tried a different cable, no different. No WiFi dongle either.

I've just played around a bit and now it outputs:

pi@raspbmc:/$ ./texturecache.py c tvshows
Traceback (most recent call last):
File "./texturecache.py", line 3916, in <module>
main(sys.argv[1:])
File "./texturecache.py", line 3825, in main
rescan=_rescan, decode=_decode, extraFields=_extraFields, query=_query)
File "./texturecache.py", line 2082, in jsonQuery
cacheImages(mediatype, jcomms, database, data, title_name, id_name, force, nodownload)
File "./texturecache.py", line 2131, in cacheImages
with database:
File "./texturecache.py", line 644, in __enter__
self.getDB()
File "./texturecache.py", line 657, in getDB
self.DBVERSION = self.execute("SELECT idVersion FROM version").fetchone()[0]
File "./texturecache.py", line 669, in execute
self.logger.log("EXCEPTION SQL: %s - retrying attempt #%d" % (e.value, self.RETRY))
AttributeError: 'OperationalError' object has no attribute 'value'

I get the feeling thats because im using a shared database though? Although when ran again, im hit by the same issues

[Milhouse]

Now that's a strange one... I'd suggest changing line 666 to the following:

Code:

except lite.OperationalError as e:
      raise
      if str(e) == "database is locked" and self.RETRY <= self.RETRY_MAX:

and then paste here whatever the real error (stacktrace) is - it looks like the database is locked, although I only ever saw this error once and haven't been able to reproduce it so the "fix" (try again after a short delay) may need some tweaking...

[Milhouse]

@K20evo - I've pushed a new version (v0.8.5) which corrects the locked database detection problem, and will now display a suitable "database locked" error if the lock remains in place for too long (which would suggest a long running database update is taking place elsewhere, most likely XBMC itself). Thanks for reporting it.

[K20evo]

@K20evo - I've pushed a new version (v0.8.5) which corrects the locked database detection problem, and will now display a suitable "database locked" error if the lock remains in place for too long (which would suggest a long running database update is taking place elsewhere, most likely XBMC itself). Thanks for reporting it.

Awesome. Just deleted and redid wget. Had an odd permission issue but just ran them as sudo, hopefully that doesn't mess up the chmod permissions..

Running it now and don't get the errors as above. Still get the download errors though

Code:

Cache pre-load activity summary for "movies":

              |    fanart   |    poster   |    TOTAL    
--------------+-------------+-------------+-------------
Cached        |      -      |      -      |      0      
Deleted       |      -      |      -      |      0      
Duplicate     |      -      |      -      |      0      
Error         |     184     |     184     |     368    
Ignored       |      -      |      -      |      0      
Skipped       |      11     |      12     |      23    
Undefined     |      -      |      -      |      0      
========================================================
TOTAL         |     195     |     196     |     391    
Download Time | 00:07:43.11 | 00:07:49.55 | 00:15:32.65

Tried to do a wget on one of the images

Code:

pi@raspbmc:/$ wget http://cf2.imgobject.com/t/p/original/8RkLHFm2PK4PO5W0WZA5LMXLZBz.jpg
--2013-07-01 10:44:57--  http://cf2.imgobject.com/t/p/original/8RkLHFm2PK4PO5W0WZA5LMXLZBz.jpg
Resolving cf2.imgobject.com (cf2.imgobject.com)... 54.230.0.218, 54.230.2.16, 54.230.3.102, ...
Connecting to cf2.imgobject.com (cf2.imgobject.com)|54.230.0.218|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 538637 (526K) [image/jpeg]
8RkLHFm2PK4PO5W0WZA5LMXLZBz.jpg: Permission denied

Cannot write to `8RkLHFm2PK4PO5W0WZA5LMXLZBz.jpg' (Permission denied).

Tried to do it as sudo

Code:

pi@raspbmc:/$ sudo wget http://cf2.imgobject.com/t/p/original/8RkLHFm2PK4PO5W0WZA5LMXLZBz.jpg
--2013-07-01 10:47:14--  http://cf2.imgobject.com/t/p/original/8RkLHFm2PK4PO5W0WZA5LMXLZBz.jpg
Resolving cf2.imgobject.com (cf2.imgobject.com)... 54.230.1.134, 54.230.1.188, 54.230.3.133, ...
Connecting to cf2.imgobject.com (cf2.imgobject.com)|54.230.1.134|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 538637 (526K) [image/jpeg]
Saving to: `8RkLHFm2PK4PO5W0WZA5LMXLZBz.jpg'

100%[================================================================================================================================================>] 538,637      351K/s   in 1.5s    

2013-07-01 10:47:15 (351 KB/s) - `8RkLHFm2PK4PO5W0WZA5LMXLZBz.jpg' saved [538637/538637]

Tried to run the whole script as sudo

Code:

pi@raspbmc:/$ sudo ./texturecache.py c movies
FATAL: The task you wish to perform requires read/write file
       access to the XBMC sqlite3 Texture Cache database.

       The following sqlite3 database could not be opened:
       /root/.xbmc/userdata/Database/Textures13.db

       Check settings in properties file texturecache.cfg

[nickr]

Why are you running everything as root in /?

Not surprising an ordinary user cannot download a file to /

[K20evo]

Why are you running everything as root in /?

Not surprising an ordinary user cannot download a file to /

Where should the texturecache.py file be downloaded to? The default location when you ssh in? If so, still having the issues as just reverted to a clean OpenElec

[piotrasd]

@MilhouseVH
Many skins in last time support Laguage Flags from Video Files over script http://forum.xbmc.org/showthread.php?tid=155693

but flags first time extracted/showed if we enter to Detail about movie, or scrap info about new movie - after that will be even in main movie menu.(after reboot etc, will work)

could you add to your script some option - to get/re-get flags for all movies in library ?
im start use this script from 2 months - and for new movies flags are automatical added when we scrap new position in library,
but i have something arround 800 old movies, where i must one by one open window detail about movie for get flag

[Milhouse]

I have no idea why you are running the script as root, that should be totally unnecessary on Raspbmc - all the XBMC files will be owned by the pi user. If you downloaded the script as root, just change the file owner back to pi (or delete the script as root, and re-download as pi).

As for why you are getting download errors, you need to look at the XBMC log for precise details.

The way the script works when pre-loading artwork is this: the script asks XBMC for an artwork file, if XBMC has it already cached then XBMC returns the cached version immediately, if not XBMC goes out and retrieves the original artwork file (from your local LAN or from the internet etc.), XBMC caches the retrieved file, and finally XBMC returns the cached file to the script (which then moves on to the next file as the script isn't interested in the file data itself). If the script cannot "download" a file it means XBMC was not able to download the original artwork file.

As we have already established, your network is not 100% reliable and if you look at a debug log (wiki) for XBMC you'll likely see errors whenever XBMC attempts to download artwork from these remote websites.

You need to get to the bottom of your network problem and discussing it in this thread is not the best way to get eyeballs on your problem. The script, curl and XBMC are just symptoms of your wider network problem, they are not the cause.

---

Where should the texturecache.py file be downloaded to? The default location when you ssh in? If so, still having the issues as just reverted to a clean OpenElec

Yes, that should be fine.

On Raspbmc ssh in as pi and create the script in your home location as user pi. On OpenELEC things are a little different, you do ssh in as root (everything runs as root), and your home location is /storage so create the script there.

You want to run the script as the same user that runs XBMC, as you'll need read/write access to files owned/created by XBMC.

[Milhouse]

@MilhouseVH
Many skins in last time support Laguage Flags from Video Files over script http://forum.xbmc.org/showthread.php?tid=155693

but flags first time extracted/showed if we enter to Detail about movie, or scrap info about new movie - after that will be even in main movie menu.(after reboot etc, will work)

could you add to your script some option - to get/re-get flags for all movies in library ?
im start use this script from 2 months - and for new movies flags are automatical added when we scrap new position in library,
but i have something arround 800 old movies, where i must one by one open window detail about movie for get flag

OK I see how that script is working, but where are the flags? My guess is that some other skin you are using is taking the language code provided by the Video Language script, then retrieving artwork from it's own personal stash of language flags (presumably local, but could be on the internet). Unless I can programmatically work out the name and location of these language flags it's going to be difficult to request XBMC to cache them. What skin are you using, can you find the flags?

[AnotherPhil]

Hi,

I'd like to run this most excellent script from another computer on my Windows network. Can I have some suggestions on how best (simplest) to do this

On Windows, you'll need to install Python 2.7.3 (or 3.3) on the PC running the script - most Linux PCs have it already or can install it with apt-get install python.

You'll need to specify the IP address (or hostname) of the remote client, this is the property xbmc.host. And the MAC address (network.mac) if you want to wake the remote client.

Depending on what action you wish to perform, you may require read/write access to the Userdata folder, the location of which is specified by the userdata property. For remote access, you may need to mount the location of the Userdata folder on the local PC. If you don't intend to perform any cache related operations (ie. you will just be querying the media library or making JSON calls) then you don't need to worry about the Userdata folder and can leave it undefined.

Assuming you configure the XBMC webserver to use port 8080 with no username/password, and don't require access to the Userdata folder, then the minimum information required by the script for remote access to a client is just the xbmc.host, ie:

Code:

texturecache.py jd movies avatar @xbmc.host=192.168.0.1

or create a file called texturecache.cfg in the same directory as the script, and specify:

Code:

xbmc.host=192.168.0.1

then run:

Code:

texturecache.py jd movies avatar

See the README for more details.

and preferably be notified when the script has completed.

Can't really help you here - maybe send yourself an email or something? I'm sure there's software for Windows that can help you do that.

Most helpful my Good Man

One more nube question if I may...Is possible to restrict the Prune option to a particular movie e.g. python texturecache.py P movies "Dr. No" ?

[Milhouse]

One more nube question if I may...Is possible to restrict the Prune option to a particular movie e.g. python texturecache.py P movies "Dr. No" ?

No, that would make no real sense - the prune process is matching cached images with media library items, and then pruning all those images that cannot be matched to a media library item. If you were to restrict the media library items (ie. to just "Dr. No") then you'd have an awful lot of cached images that would remain unmatched and would become candidates for pruning.

[AnotherPhil]

One more nube question if I may...Is possible to restrict the Prune option to a particular movie e.g. python texturecache.py P movies "Dr. No" ?

No, that would make no real sense - the prune process is matching cached images with media library items, and then pruning all those images that cannot be matched to a media library item. If you were to restrict the media library items (ie. to just "Dr. No") then you'd have an awful lot of cached images that would remain unmatched and would become candidates for pruning.

Ok, my understanding is ...

The [P]rune command deletes all orphaned images from the cache.

The [C]ache command issued for a particular movie, re-downloads & caches the movies' poster and fanart images from the existing URLs.

What I'm after is a quick way to remove whatever remnants are left after issuing a JSON "VideoLibrary.RemoveMovie" command. i.e. Delete all the cached images & database records relating to the removed movie. Is this already possible with your script or is the logic there, but needs rejigging into another command?

As always your expert advice is very much appreciated

[wishie]

So it seems I have a database (MySQL) full of incorrect URLs for thumbnails and other artwork.. so when I exported my library to NFOs, they now too contain invalid/incorrect URLs..

Is there a way I can simply tell your script to fetch everything new from the internet, ignoring the URLs in the NFO files?

Also, how to remove all the invalid entries from my database?

[Milhouse]

Ok, my understanding is ...

The [P]rune command deletes all orphaned images from the cache.

It's the [R]everse query option that locates "orphaned" files and removes them.

The role of Textures13.db is to act as an "index" for files in the Thumbnails folder using a "hash" identifier which is also the filename of each thumbnail file (eg. 2/276c1eb0.jpg). The "[R]everse lookup" option will scan the Thumbnails folder retrieving all of the hashes and then try to find the corresponding rows in the Textures13.db - hence the "reverse" nature of this option. Any thumbnail file for which a row can't be found will be considered an "orphan" and removed - without a row in Textures13.db such files are just wasting file space.

Orphaned files are rare, but most likely to appear following an upgrade (eg. Eden to Frodo) when old Eden thumbnails are left unreferenced by the updated Textures13.db used by Frodo.

The [P]rune option on the other hand will match cached files (ie. a row in Textures13.db) with the original artwork in the Media Library (MySQL etc.). Any cached file that cannot be matched to an artwork file in the Media Library will be removed.

Typically the files to be pruned are "preview" thumbnails and other transient artwork that may be downloaded by plugins (eg. BBC iPlayer episode thumbnails), but can also include artwork associated with movies and TV shows that have been removed from the media library.

The [C]ache command issued for a particular movie, re-downloads & caches the movies' poster and fanart images from the existing URLs.

Correct.

What I'm after is a quick way to remove whatever remnants are left after issuing a JSON "VideoLibrary.RemoveMovie" command. i.e. Delete all the cached images & database records relating to the removed movie. Is this already possible with your script or is the logic there, but needs rejigging into another command?

If you know what artwork needs to be deleted, you could try the [s]earch option and then [d]elete cached items based on the row ids found by [s]earch, but it's unlikely you will know all of the artwork references to search for, in which case the current [P]rune option is still the best method - while slow, this will identify ALL of the artwork that is no longer referenced by the Media Library, including previously removed movies.

So it seems I have a database (MySQL) full of incorrect URLs for thumbnails and other artwork.. so when I exported my library to NFOs, they now too contain invalid/incorrect URLs..

Is there a way I can simply tell your script to fetch everything new from the internet, ignoring the URLs in the NFO files?

Also, how to remove all the invalid entries from my database?

No, sorry - that's not how it works.

The URLs you have in your media library determine what artwork is cached and displayed by XBMC, so even if it were possible for this script to cache alternative URLs, XBMC would still be using the URLs specified by your media library and you'd just end up with both artwork URLs being cached and the artwork from the "incorrect" URLs still appearing on the display. In short, it would be a waste of time and disk space caching "alternative" URLs, as XBMC (driven by your media library) would know nothing about them.

Correcting the URLs in the Media Library is the only option, which you can of course do via the Info screen and "Choose Art" button, but assuming we're talking about more than just a handful of movies/tv shows this quickly becomes an impractical solution.

My advice would be to fix the URLs in the NFO files using a media manager such as Ember or alternatively delete all the NFO files if they're not worth keeping and you're happy with internet-sourced artwork; drop your MySQL databases (MyMusic and MyVideo); restart XBMC so that it re-creates empty databases; and then finally re-scrape your library.

[wishie]

I feared this was the case.. Looks like im dumping the DB and rescraping all my content.. Last time I did this it was a nightmare, as the scraper fetched the wrong movie info for a lot of movies.. I guess I can write some sort of script to remove the thumbnail URLs from the NFOs. Atleast that way, XBMC will get the correct result for each movie.. but what happens with artwork in that case?