Slow to generate photo previews
#1
Hi all,

I have an Odroid N2 connected via gigabit wired to my NAS. I have the photos NFS share mounted on the N2.

There are several thousand photos on the NAS spread over many folders and these folders could have 10’s or 100’s of photos.

When I go into Kodi 19.3 to view the photos I have to wait sometime for the photo previews to be generated before I can scroll thru the photos to find the one(s) I’m after. Is there a way to speed this process up in anyway?

Is there a way to generate all the thumbnails, maybe that would help?

TIA
Greg
Reply
#2
How large are the photos (dimensions, file size)?

All photos in a folder are set up for a thumbnail when the folder is accessed for the 1st time.
Thousands... yep, that can take up some time.

Speed up the process? Right now, get a faster computer.
Reply
#3
(2021-11-01, 10:46)Klojum Wrote: How large are the photos (dimensions, file size)?
Anything from 1MB to 17MB (5472 x 3648 pixels)

Is there anyway to get all the thumbnails generated without going into each folder and waiting for it to happen?

EDIT:

I have the following in advancedsettings.xml.  Would that effect the photos thumbnails.

Code:
    <!-- Change Covers and Fanart Resolution.  Defaults are 720 and 1080 -->
    <imageres>1080</imageres>
    <fanartres>1440p</fanartres>
Reply
#4
If you have a lot of those bigger ones, that's a butt load of network traffic for Kodi - it has to pull the full 17MB over the wire to then make the thumbnails etc.  Multiply that out by a whole stack of large images like that and of course that will become a slow process.

I export to a separate 'Fast album' of images for just this sort of thing - all images shrunk to fit 1920 by 1080 and a little more aggressively compressed (JPEG 75), to keep things small and fast (each file ends up ~400kb typically, some smaller, a few up to ~ 1 megabyte).  I routinely open large albums (500+ images) on my N2/N2+ machines.  There's some pause if there are literally hundreds of shots in the one folder still, but most of my folders are more like 0 to 100 images per folder, and that works very well indeed. 

This means also that there's no need for Kodi to scale the images when displaying them (e.g. I tested using 4K images, and the native 1920 by 1080 ones were better (less/no artefacts and sharper, Kodi's scaling is not awesome it seems).
Addons I wrote &/or maintain:
OzWeather (Australian BOM weather) | Check Previous Episode | Playback Resumer | Unpause Jumpback | Caber Toss | Switchback | XSqueezeDisplay | Kodi 2nd Display
Sorry, no help w/out a *full debug log*.
Reply
#5
(2021-11-02, 00:34)bossanova808 Wrote: If you have a lot of those bigger ones, that's a butt load of network traffic for Kodi - it has to pull the full 17MB over the wire to then make the thumbnails etc.  Multiply that out by a whole stack of large images like that and of course that will become a slow process.
Yes, I can now understand that and I did try reducing a folder of 150 17MB pictures to around 400-500kb each and it is definitely a lot quicker.
Quote:I export to a separate 'Fast album' of images for just this sort of thing - all images shrunk to fit 1920 by 1080 and a little more aggressively compressed (JPEG 75), to keep things small and fast (each file ends up ~400kb typically, some smaller, a few up to ~ 1 megabyte). 
While this solves the issue it means creating duplicates of all my 1000's of photos, the ones I have now and those I add later.  Not sure if it's a practical solution.

A better solution would be having a background task that looks for missing thumbnails and generates them after hours.
Reply
#6
Well sure, but then either you, or someone else, has to be motivated enough to make that happen in Kodi...and given pictures aren't part of the Kodi library system, it might not be considered a great idea for Kodi to go off indexing all the jpegs it can find on the media sources or whatever....I suspect it might take some convincing to get this to actually happen!

In my case, it's a simple and effective solution I don't need anybody's C++ skills (or Python plugin?) for.  My master library obviously lives in my RAW converter (Lightroom), so setting up an extra export from that to a specific folder for this fast library is trivial, and disk space is cheap, so as far as solves go, I'm pretty happy with this one (mind you I started doing this maybe 20 odd years ago....would be a bit harder if I started now, but even then because I have a good source library, it wouldn't be that hard - once again, success with media libraries comes from making an early decision to keep things neat & organised right from the beginning....but I digress....).

Another option I can think of - it's possible you might be able to use texturecache.py to cause the thumbs to pre-generate, I suppose - although I am not sure if it works for non library options - see: https://github.com/MilhouseVH/texturecac.../README.md

You could also look at the Screensaver Picture Slideshow addon - I am pretty sure it generates indexes of picture folders, so from there to getting Kodi to cache the thumbs is probably not a huge distance, if you do have some Python skills.
Addons I wrote &/or maintain:
OzWeather (Australian BOM weather) | Check Previous Episode | Playback Resumer | Unpause Jumpback | Caber Toss | Switchback | XSqueezeDisplay | Kodi 2nd Display
Sorry, no help w/out a *full debug log*.
Reply
#7
@bossanova808 - Thanks for your reply and suggestions.  I had already looked at texturecache.py but it does not work with pictures, seems more related to movies, videos and tv shows.

Unfortunately I have zero Python skills so the Screensaver Picture Slideshow addon is not an option.  Once again, many thanks.
Reply
#8
I solved the same issue (preview generation rendering old HP 635 laptop unresponsive) by creating a script to cache images to use with Yatse remote. I can run it after I add a new album or by cronjob. I can post it here if someone needs it.

Today I, however, used Video Database Cleaner add-on and it removed the cached images as orphaned ones Sad I will need to mark these somehow so they are not cleaned unless user wants them to be cleaned.
Maintainer of Kodi from Debian | Got an issue with Kodi from Debian? Report it here: Kodi from Debian - Support Thread
Reply
#9
(2021-11-13, 16:15)basilgello Wrote: I can post it here if someone needs it.
Yes please.
Reply
#10
(2021-11-13, 22:15)gregeeh Wrote:
(2021-11-13, 16:15)basilgello Wrote: I can post it here if someone needs it.
Yes please.

Here it is: https://paste.debian.net/1219418

You define KODI_ROOT_URL like:

shell:
export KODI_ROOT_URL="http://user:pass@host:port

and invoke the script:

shell:
sh yatse-recache-images.sh /path/to/picture/source/dir

where path to folder with pictures is your picture media source.
Maintainer of Kodi from Debian | Got an issue with Kodi from Debian? Report it here: Kodi from Debian - Support Thread
Reply
#11
Smile 
(2021-11-13, 22:57)basilgello Wrote:
(2021-11-13, 22:15)gregeeh Wrote:
(2021-11-13, 16:15)basilgello Wrote: I can post it here if someone needs it.
Yes please.

Here it is: https://paste.debian.net/1219418

You define KODI_ROOT_URL like:

shell:
export KODI_ROOT_URL="http://user:pass@host:port

and invoke the script:

shell:
sh yatse-recache-images.sh /path/to/picture/source/dir

where path to folder with pictures is your picture media source.

Thank you.  Smile
Reply
#12
(2021-11-13, 22:57)basilgello Wrote:
(2021-11-13, 22:15)gregeeh Wrote:
(2021-11-13, 16:15)basilgello Wrote: I can post it here if someone needs it.
Yes please.

Here it is: https://paste.debian.net/1219418

You define KODI_ROOT_URL like:

shell:
export KODI_ROOT_URL="http://user:pass@host:port

and invoke the script:

shell:
sh yatse-recache-images.sh /path/to/picture/source/dir

where path to folder with pictures is your picture media source.
Installed your script on my NAS, which is where the Photos in Kodi are located.  NAS share is NFS mounted in Kodi.  Getting ERROR 3:

Code:
[/share/homes/admin/bin] # ./recache-images.sh /share/CACHEDEV1_DATA/Photos/
/share/CACHEDEV1_DATA/Photos/7SB 2018/DSC03739.JPG: ERROR 3
/share/CACHEDEV1_DATA/Photos/7SB 2018/DSC03738.JPG: ERROR 3
/share/CACHEDEV1_DATA/Photos/7SB 2018/DSC03735.JPG: ERROR 3
/share/CACHEDEV1_DATA/Photos/7SB 2018/DSC03734.JPG: ERROR 3
/share/CACHEDEV1_DATA/Photos/7SB 2018/DSC03731.JPG: ERROR 3

set shows:

Code:
​​​​​​​KODI_ROOT_URL=http://kodi:[email protected]:8080
Reply
#13
@gregeeh ERROR 3 means a malformed URL. Can you please run

shell:
sh -x yatse-recache-images.sh /path/to/directory

and drop the example of URL fed to cURL? Real password and username redacted, of course.
Maintainer of Kodi from Debian | Got an issue with Kodi from Debian? Report it here: Kodi from Debian - Support Thread
Reply
#14
Also check if root directory is the media source. i.e if you run script on top-level directory, whose subdirs are some declared as image sources and others are not, Kodi will not process files it knows nothing about and spits an error in kodi.log.

EDIT: oh, I feel dumb Smile You told me your files are mounted on a NAS via NFS. How does your NFS mountpoint look like? We need to tweak the script so user can specify directory path substitution for remote mountpoints, I am using all-local picture setup, so I did not bother covering your case yet.
Maintainer of Kodi from Debian | Got an issue with Kodi from Debian? Report it here: Kodi from Debian - Support Thread
Reply
#15
(2021-11-14, 12:18)basilgello Wrote: Also check if root directory is the media source. i.e if you run script on top-level directory, whose subdirs are some declared as image sources and others are not, Kodi will not process files it knows nothing about and spits an error in kodi.log.

EDIT: oh, I feel dumb Smile You told me your files are mounted on a NAS via NFS. How does your NFS mountpoint look like? We need to tweak the script so user can specify directory path substitution for remote mountpoints, I am using all-local picture setup, so I did not bother covering your case yet.
Yes, all the pics are on the NAS (192.168.10.2) in a folder named Photos.

The mount is created on Kodi using the command:

Code:
systemctl enable storage-photos.mount

storage-photos.mount is here.

Hope this answers your questions.
Reply

Logout Mark Read Team Forum Stats Members Help
Slow to generate photo previews0