[BuckyKattNJ]

Hello.

I am trying to get texturecache.py working... but its not doing what I need.

I use Kodi mostly as a DLNA/UPNP server, and I need it to automagically cache thumbnails for mp3s I add without going into Kodi and browsing manually. All my music has thumbnails in the files and they are the only thumbs I want cached... no scraping or folder.jpg files (actually a few are present, but I'm weeding them out).

I have fiddled with the example .cfg and it is kinda working... but when I do a:

./texturecache.py c songs

Or a:

./texturecache.py C songs

It cranks along for a while and the summary shows it deleted 14, cached 14 (which seem to correspond to the folder.jpg/cover.jpgs that are present), it then finds like 10000 duplicates and 1000 ignored... which I imagine are my in-file thumbnails for about 1000 albums and 10000 songs.

What do I need to set to get it cache the in-file thumbs so I see album artwork via DLNA/UPNP?

IIRC, the only things I changed in the .cfg are:

allow.recacheall=yes
userdata = ~/.kodi/userdata
webserver.port = 80
qa.art.songs = thumbnail

I'm running OpenELEC 7.0 with Kodi 16.1 on a RPi2.

Thanks,

BKNJ

[Milhouse]

Embedded artwork is not supported by this script as there's no mechanism to force caching of such artwork. If you enable the logfile (@logfile=/tmp/tc.log) you'll see entries written to the log that provide details of the files being ignored, including why they were ignored - these will be your music files with embedded artwork. The only way to cache this artwork is to view them in the GUI as it's a GUI process that extracts the artwork from the file.

You don't need to set the userdata property - the value you're using is already the default (both ~/.kodi and ~/.xbmc are supported).

[BuckyKattNJ]

Argh!

I misunderstood and thought this was the original point to the script... reading different posts in the way, wayback.

I guess I may have to write my own script to kind-of automate this... I know that I can stimulate Kodi to cache the embedded artwork via Chorus... by basically repeatedly calling http://127.0.0.1/#artist/n, where n=1 to infinity... I just have to write some scrape code to detect when it runs out of artists.

Knowing that texturecache.py is also poking the same webserver port, I'm curious how requests are being routed to Kodi or to Chorus. I've never really poked and prodded the web server config, which I imagine is kernel-based.

BKNJ

[Milhouse]

That is the point of the script, but because embedded artwork has to be extracted from the media file before it can be displayed I haven't yet found a method for automating the caching of these embedded images - any attempt to view (download) the uncached embedded artwork will usually fail as the only url Kodi has at this time is either a video or audio file and not an image file. If you find a solution, please share!

[BuckyKattNJ]

Well, I see a solution, as I've written lots of scrapers... but trying to use Perl LWP with JSON is something I'm not used to working with.

Where does Kodi generally hide the .js files for the various web front ends? I imagine I'm going to have to start there to get an idea of the data sent back and forth. I wish I had some http sniffing tools ready to go... that would likely clue me in quite a bit faster.

Interestingly, it appears using the default web interface and clicking on Music caches all my music thumbnails without any further intervention... but its taking forever. I'll have to give an hour and then try out my DLNA/UPNP app.

BKNJ

[User 224999]

That is the point of the script, but because embedded artwork has to be extracted from the media file before it can be displayed I haven't yet found a method for automating the caching of these embedded images - any attempt to view (download) the uncached embedded artwork will usually fail as the only url Kodi has at this time is either a video or audio file and not an image file. If you find a solution, please share!

What I did in the past is just call the image through the webinterface (not json api!) and that seemed to work well:
A simple head request is enough to trigger the internals to go fetch the image and put it in the cache.


Code snippet:

PHP Code:

requests.head(
                url=("http://%s:%s/image/image://%s"
                     % (self.xbmc_host, self.xbmc_port, self.url_to_process)),
                auth=(self.xbmc_username, self.xbmc_password),
                timeout=(35.1, 35.1)) 


Where the image url is the full image path you get from the json API.
Or is this something you already tried ?

EDIT: I must also say that you can see in the above snippet it uses an extremely short timeout as we don't want the results. You must however notice that the webserver cannot handle a lot of requests at the same time. In the Emby addon (where I implemented this back than) we have a limiter of maximum 25 requests at the same time and that seems to go well even on low power machines.

[Milhouse]

What I remember (it was a long time ago) was that it would sometimes work when the artwork is embedded, but wasn't terribly reliable, which is why in the end I decided to cut my losses and disable support for embedded artwork. I did query the process with the Kodi developers who informed me that the extraction of a embedded artwork was connected to a GUI process and unlikely to be supported by JSON (and presumably web ui).

I'm going to be short of time to look into this for the next couple of weeks but assuming I find the time in the near future will take another look.

[BuckyKattNJ]

After pulling apart the JSON and Chorus using the Chrome developer tools, I now see, that the combination of the jsonrpc call and the GET on the image is what Chorus does... and this somehow makes Kodi cache the image... but seemingly only from Chrome.

Using cURL and wget to do the same thing fails... then I realized Chrome was double encoding the special characters.

Double encoded the URL and fed it into wget... and bam, it worked. I coded up something permanent in Perl and now all my embedded artwork is cached.

Its basically what marcelveldt was describing. Sorry, its in Perl... Hope this helps someone.

Code:

#!/usr/bin/perl

use warnings;
use strict;
use LWP::UserAgent;
use JSON;
use URI::Encode;

my $host = '127.0.0.1';

my $uri = 'http://' . $host . '/jsonrpc';
my $json = '{"jsonrpc":"2.0","method":"AudioLibrary.GetAlbums",'.
           '"id":"wtf","params":[["title","displayartist",'.
           '"thumbnail"],{"start":0,"end":50000},{"sort":{"method":'.
           '"dateadded","order":"descending"}}]}';
my $req = HTTP::Request->new( 'POST', $uri );
$req->header( 'Content-Type' => 'application/json' );
$req->content( $json );

my $lwp = LWP::UserAgent->new;
my $response=$lwp->request( $req );

my $obj = from_json($response->content);

my $loop;
my $encoded;
my $encoder=URI::Encode->new({double_encode=>1,encode_reserved=>1});

for $loop ( @{$obj->{result}->{albums}} ) {
  print $loop->{displayartist} . ' - ' . $loop->{title} . ' ';
  $encoded=$encoder->encode($loop->{thumbnail}) . "\n";
  $uri='http://' . $host . '/image/' . $encoded;
  $req = HTTP::Request->new( 'GET', $uri );
  $response=$lwp->request($req);
  print $response->code . ' ';
  print $response->message . "\n";

  }

BKNJ

[Uatschitchun]

Hi there, I get the following:

Code:

LibreELEC:~ # ./texturecache.py c
Need to cache: [  fanart  ] for addon: Xonfluence
Need to cache: [thumbnail ] for addon: Xonfluence

Caching artwork: 0 items remaining of 2 (qs: 0, qm: 0), 0 errors, 0 threads active (02.00 downloads per second, ETA: 00:00:00)
Traceback (most recent call last):e, Das Erste HD...
  File "./texturecache.py", line 8583, in <module>
    main(sys.argv[1:])
  File "./texturecache.py", line 8391, in main
    extraFields=_extraFields, query=_query, drop_items=_drop_items)
  File "./texturecache.py", line 4628, in jsonQuery
    cacheImages(mediatype, jcomms, database, data, title_name, id_name, force, nodownload, drop_items)
  File "./texturecache.py", line 4674, in cacheImages
    parseURLData(jcomms, mediatype, mediaitems, imagecache, data, title_name, id_name)
  File "./texturecache.py", line 5106, in parseURLData
    parseURLData(jcomms, "%s.channel" % mediatype, mediaitems, imagecache, item["channels"], "channel", "channelid", pvrGroup=title)
  File "./texturecache.py", line 5072, in parseURLData
    mediaitems.append(MyMediaItem(mediatype, a, name, season, episode, item[a], 0, None, item[id_name], False))
KeyError: 'channelid'

Any suggestions?

[Milhouse]

@Rusendusen can you enable logging by adding "@logfile=/tmp/tc.log" to your command line then send me a link to your tc.log file.

[Uatschitchun]

Here you are:
http://sprunge.us/YbWj

[Milhouse]

Here you are:
http://sprunge.us/YbWj

Thanks. For some reason some of your TV channels don't have a "channelid" property.

Can you try this version of texturecache.py and let me know if it works OK: http://sprunge.us/HZFE

If not please upload another logfile.

[Uatschitchun]

Working! Thank you!

Another short question:
I'd like to maintain my clients (all have 3 profiles including master). Therefore, I added sections to cfg. For the master profile no section is needed when using local on the clients.
Nevertheless, is there a performance issue when using texturecache over network?
I don't know if I use texturecache only locally or from one PC over network with xbmc.host not being localhost...?

[Milhouse]

Working! Thank you!

Thanks, I'll push a new version shortly.

Edit: New version v2.3.5 pushed.

Nevertheless, is there a performance issue when using texturecache over network?

It's probably a little slower, depending on how much data has to be transferred over the network, but I wouldn't expect it to be noticeably slower.

However running the script over the network from a central machine to each of your clients is usually a lot more convenient, and worth any slight increase in run time.

[Milhouse]

You don't need to set userdata at all - let the script work it out if possible. You only need to set it if it's compeletely non-standard.

If you're using profiles you need the script to switch the profile for you (which automatically recalculates userdata etc.), so that Kodi is reading the correct database(s) for the profile you are querying.

Add "profile.name=<profilename>" to your sections (the master profile has the name "Master user") and the script will instruct Kodi to switch profile whenever the required profile is not already loaded. Whether Kodi then works afterwards is impossible to say...it may do if you're lucky - profiles are very "fragile".

BTW, another way to write your config would be:

Code:

checkupdate = yes
download.threads = 10
extrajson.movies = streamdetails, file, mpaa, rating, plot
cache.castthumb = yes

section=Master

[Master]
profile.name=Master user
lastrunfile=/storage/lrf.dat

[fanny]
profile.name=fanny
lastrunfile=/storage/lrf_fanny.dat

and now your default section is "Master".

If Kodi has the Master profile loaded then "texturecache.py @section=fanny stats" would unload the "Master user" profile, load the "fanny" profile, and query the databases for the fanny profile.

Kodi will be left with the "fanny" profile loaded, so if you then run "texturecache.py stats" it will unload the "fanny" profile, load the "Master user" profile, and query the databases for the Master user profile.