• 1
  • 112
  • 113
  • 114(current)
  • 115
  • 116
  • 197
[RELEASE] Texture Cache Maintenance utility
@Milhouse this is my config file. I added some values for testing sake, don't know what values should go there.

Code:
userdata = ~/.xbmc/userdata/
dbfile = Database/Textures13.db
thumbnails = Thumbnails/
xbmc.host = localhost
webserver.port = 8080
webserver.username = username
webserver.password = password
rpc.port = 9090
download.threads = 2
extrajson.albums  =
extrajson.artists =
extrajson.songs =
extrajson.movies = cast
extrajson.sets =
extrajson.tvshows.tvshow = cast
extrajson.tvshows.season = cast
extrajson.tvshows.episode= cast
qaperiod = 30
qa.file = no
cache.castthumb = true
logfile =
logfile.verbose = no
network.mac = aa:bb:cc:dd:ee:ff
allow.recacheall = yes

Can you give some examples of the
Code:
extrajson.*
usage?

Thanks
Laptop: Dell Inspiron i7559-2512BLK | Windows 10 Pro x64 | Intel Core i7 6700HQ (2.60 GHz) | 1 TB + 8 GB SSHD
Kodi 18.3
MiniPC: LibreELEC 8.2.1
Sharing Media: SMB (HDD: 2 TB, 1 TB) | Media Manager/Organizer - Media Companion
Reply
Here's a few examples:
Code:
extrajson.songs = file, artist, album, duration
extrajson.movies = trailer, streamdetails, file, mpaa, rating, plot, dateadded, playcount, cast, tag, studio, runtime, country, lastplayed, imdbnumber, votes
extrajson.tvshows.tvshow = plot, rating, cast, file, imdbnumber
extrajson.tvshows.episode = streamdetails, file, dateadded, rating, plot, cast

Any valid field (Kodi JSON RPC API) should work, and the relevant fields will be output on the "Jd" or "J" options (eg. "Jd songs" will include file, artist, album and duration in addition to the basic fields output when using "jd" or "j").
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
@Milhouse,
Thank you so much for this invaluable script. I run it from Windows 7 to talk to my AFTVv1 running Kodi 15.2, and regularly use the 'p' and 'P' options to control usage of the limited space on my FireTV.

Recently, I tried using the 'r' and 'R' options but receive the following errors (yes, different errors running twice in succession):
Code:
C:\> c:\Python27\python.exe texturecache.py r
FATAL: The task you wish to perform requires read/write file
       access to the Kodi SQLite3 Texture Cache database.

       The following SQLite3 database could not be opened:
       \sdcard\Android\data\org.xbmc.kodi\files\.kodi\userdata\Database\Textures13.db

       Check settings in properties file texturecache.cfg,
       or upgrade your Kodi client to use a more recent
       version that supports Textures JSON API.

C:\> c:\Python27\python.exe texturecache.py r
FATAL: The task you wish to perform requires read/write file
       access to the Thumbnails folder, which is inaccessible.

       Specify the location of this folder using the thumbnails property
       as an absolute path or relative to the userdata property.

       The currently configured Thumbnails path is:
       \sdcard\Android\data\org.xbmc.kodi\files\.kodi\userdata\Thumbnails\

       Check userdata and thumbnails settings in properties file texturecache.cfg

As I mentioned, I can successfully run using the 'p' or 'P' options, and my config file is simple:
Code:
userdata = /sdcard/Android/data/org.xbmc.kodi/files/.kodi/userdata/
xbmc.host = <my-ipv4>
webserver.username = <my-user>
webserver.password = <my-pass>
logfile = texturecache.log
logfile.verbose = yes
The verbose log file from the last run is here: texturecache.log

I'd appreciate any suggestion you might have about the problem, and forgive me if there's an obvious mistake here.

Many thanks and take care!
Reply
As per the error message, the option requires read/write access to the Textures13.db file and Thumbnails folder, ie. direct file system access.

The options "f", "F", "r", "R", "S", "X", "Xd" all require direct filesystem access as there is currently no JSON functionality available for the actions these options perform (ie. deleting files from the Thumbnails folder when there is no matching entry in Textures13.db, and retrieving file size/date information of files from the Thumbnails folder).

There are two solutions - run the texturecache.py script directly on the Kodi client, or mount the userdata folder of the Kodi client on the PC that is running the script.

Note that specifying the Android userdata location in your texturecache config when running the script on a PC is not going to achieve anything as the location you have configured is not accessible to the PC. You need to mount the Kodi client userdata partition on the PC and then configure the userdata mount point on the PC as the userdata path, allowing the script to read/write data within the Kodi client userdata folder.

However since you're using Android for the Kodi client and Windows to run the script, I suspect both solutions will be unavailable to you as I don't think you can run the script on an unrooted Android box, and it's unlikely you can mount the Android userdata folder on Windows (the userdata folder would have to be shared over SMB by Android).
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Thank you so much for the detailed explanation. I admit I had assumed the path was being evaluated in the context of the JSON server/Kodi client, and not the python script, or that file access was being done via the web server. I can see now using the script fully remotely can be more of a challenge.

In any case, I'm more than happy to be able to use the 'p' and 'P' options remotely, so my thanks to you for that!

Also, you got me curious so I was looking at the JSON APi, and noticed the Files.GetDirectory method and friends, which seem to support listing files, modification times and sizes, but no deletion as you say. So perhaps the 'r' option might possible but not the 'R' one at this time? Hopefully the API will be extended in future...

Thanks again for such a helpful utility.
Reply
Unfortunately for security reasons, the Files.GetDirectory method will only return details of directories (and files) that are within your sources, so it's not currently possible to use this method to access the contents of the clients userdata folder.

You can see this behaviour with the call "texturecache.py rdirectory special://masterprofile" which tries to recursively list the contents of the master profile directory (ie. .kodi/userdata), but it will fail. Now add "special://masterprofile/" to sources.xml, restart Kodi and try the rdirectory call again, and all files will be successfully retrieved.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
Hi, I got yesterday a new FireTV where I installed Kodi in addition to my RasPi2 with OpenELEC.
For this I moved my local library via Export/Import from RasPi2 to my NAS where I setup a mySQL server.
Now I have many missing artworks/covers etc. on FireTV, on RasPi2 everything is fine.
If I rescrape a folder on RasPi2 then FireTV also gets the pictures, but I don't want to rescrape whole database.

Did I understand right from flying through this thread, that if I install python and this tool on my pc,
activate webserver on FireTV and running this line on PC:
Code:
./texturecache.py @xbmc.host=192.168.1.34 c movies
All missing pictures will be loaded for movies?
Thanks
Giz
Reply
If you've already set artwork for your media library than yupp, you can cache it like so.
Image Image
Reply
Yes I've already the artwork available from old scrap on RasPi2.
Reply
Hi,

first a big thanks to Milhouse for developing and maintaining this great utlity.

I cached my posters and fanart using the "c" option, but my discart, clearart are not being cached. I have tried looking for the answer in this thread, but I became none the wiser of it.

My movies are organised into individual folders. Here are the names of the files that do not want to be cached:
  • banner.jpg
  • clearart.png
  • disc.png
  • landscape.jpg
  • logo.png

What command or options should I be using?
Reply
Hi again,
found out my problem, I checked art table in mysql server.
All the not new scraped movies in library are pointing to a path where export file was written.
The new scraped movies and tv shows pointing to online links.
Is there a command to rescrape/update all movies without doing one by one or removing them from library?
Reply
(2015-11-03, 21:23)Gizmo81 Wrote: Hi again,
found out my problem, I checked art table in mysql server.
All the not new scraped movies in library are pointing to a path where export file was written.
The new scraped movies and tv shows pointing to online links.

How did you migrate from your local RPi2 SQLite library to your centralised RPi2/FireTV library on MySQL if you didn't re-scrape your exported library?

Assuming you exported your artwork too, then both devices should now be using this local artwork. Any artwork which was missing in the RPi2 at the time of the export may now be being sourced remotely from the internet.

By the way, if you want to view the artwork urls associated with a movie there's no need to poke around in SQL:
Code:
texturecache.py jd movies <moviename>

(2015-11-03, 21:23)Gizmo81 Wrote: Is there a command to rescrape/update all movies without doing one by one or removing them from library?

Code:
texturecache.py c movies

will cache any artwork yet to be cached, whether it's local or remote, assuming the artwork is accessible to the Kodi client.

Although to be honest I'm a bit confused about what you've done or what the problem is.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2015-11-03, 21:01)Orbi Wrote: I cached my posters and fanart using the "c" option, but my discart, clearart are not being cached. I have tried looking for the answer in this thread, but I became none the wiser of it.

The "c" option will cache all artwork by default, so if you've got discart, clearart, etc. in your media library then these will be cached too.

Dump one of your movies to see if you really have those discart, clearart items associated with the movie, eg. "texturecache.py jd movies avatar" to see what artwork items and urls are associated with Avatar.

If you definitely have those artwork items associated and you still don't think they're being cached, then run the following and upload the log to a pastebin site (change the movie name as appropriate):

Code:
texturecache.py c movies avatar @logfile=tc.log
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
(2015-11-03, 23:15)Milhouse Wrote: Although to be honest I'm a bit confused about what you've done or what the problem is.

OK, I will try to make it easier, was confusing myself too!

I setup a mySQL server on my NAS, after that I exportet Library from RasPi2 to one single file.
Then I added needed lines in advancedsettings and rebooted RasPi2.
After that I imported the Library and everything looks fine.
Now I prepared the FireTV, added advancedsettings and resources, rebooted
and everything was there except the artworks. If I manually updated a movie artwork was loaded.
I tried your tool but doesn't work. Later I found out reason was that artwork path was pointing
to local path from Raspi2 where I exported the library first and new scraped stuff has online path.
In meantime I rescraped whole library, but now I have to set watched state and some movie groups again.
I was searching for a way to update the movies to get back the online artwork path instead of local path.
Reply
If you export the old RPi2 library to separate files then it will write the artwork and nfo files alongside the movies, which should be on your network, and when you import everything back in to create the MySQL library the network paths of artwork and movies alike will be used and accessible to both clients.

There's no easy solution to fix your current MySQL database, you could use mktools.py, or SQL queries, but I think in your case it will be easier to export the old RPi2 library to separate files and then rescrape a new MySQL library.

For your watched statuses you can add an advancedsettings.xml option to import watched statuses from the exported nfo files:
Code:
    <importwatchedstate>false</importwatchedstate>  <!-- import previously exported playdate and playcount from .nfo files. Defaults to false -->
    <importresumepoint>false</importresumepoint>  <!-- import previously exported resume point from .nfo files. Defaults to false -->
or you can use the texturecache.py tool to backup the watched status and resume points from your old RPi2 library to a file, eg.
Code:
texturecache.py watched movies backup movies.dat
then later on restore the movies.dat file to your new MySQL library (doesn't matter which client you use when restoring):
Code:
texturecache.py watched movies restore movies.dat
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
  • 1
  • 112
  • 113
  • 114(current)
  • 115
  • 116
  • 197

Logout Mark Read Team Forum Stats Members Help
[RELEASE] Texture Cache Maintenance utility17