Kodi Community Forum

Full Version: No more shared thumbnails in XBMC 11?
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2
The following note for the XBMC 11 May cycle on 'Image Caching Improvements' worries me:
"...And MySQL users will no longer have to worry about pathsubstitutions, as each client will maintain its own thumbnail cache that gets registered in the bigger database."

Seems to me this change will mean that all my clients must spend time downloading their own copy of thumbnails. The whole point in using a centralized DB+thumbnail solution is so each client wont have to download and store separate graphics.
If you continue to use the path subs (wiki) instructions noted in the MySQL (wiki) guide, then nothing will change for you and you can still share a central thumbnail storage. However, as noted on the MySQL guide, this will likely cause some level of slow-down for your XBMC clients, as before.
Actually, it will change - each client will still download and overwrite the previous version as according to it's own database it doesn't have the textures. So you really are just wasting your time in pathsub'ing the thumbnails in current master.

If you don't want clients to download the art then either have local art (they'll just cache the local art then) or copy the thumbcache from one client to the next - it's completely portable now.
Well, from a storage standpoint it wouldn't change (like trying to save space on an ATV2 when one has a large library).
Correct.
(2012-07-02, 00:46)jmarshall Wrote: [ -> ]each client will still download and overwrite the previous version as according to it's own database it doesn't have the textures
Exactly what I figured Sad Running XBMC (OpenElec) from a USB-stick (no HDD) means I don't have room to save local fanart, and loading fanart from the network is faster anyway (than local USB data).

Would it be possible to cater to the users running with central thumbnails by making it an option to use shared location or not, ie "Use shared fanart"? This would populate a separate mySQL table (specifically for clients wanting to use shared thumbnails). Any client wanting to use shared thumbnails could then tick the option.
If memory storage is your concern, then I'll repeat myself, path subs will still save you storage space. Additional XBMC clients will attempt to re-download the same image again, but it will be saving to the same location.
(2012-07-02, 09:52)Ned Scott Wrote: [ -> ]If memory storage is your concern, then I'll repeat myself, path subs will still save you storage space. Additional XBMC clients will attempt to re-download the same image again, but it will be saving to the same location.
Storage size is a concern yes, but so is the fact that all clients will now have to spend time re-downloading the same fanart to the same location. This means each single client will have to go through the same update & cleanup run every time I store or remove stuff. This is in fact a step backwards from the current implementation. Yes it'll work, but it's hardly a very sensible or pretty solution.
Sorry, but we can't and don't attempt to cater to everyone.

If you don't want the hassle of many clients downloading the same thing, then copy the texture database from one machine to the next.

But seriously, get some storage on those clients - it will speed things up a heap.
(2012-07-02, 10:37)Innocence Wrote: [ -> ]
(2012-07-02, 09:52)Ned Scott Wrote: [ -> ]If memory storage is your concern, then I'll repeat myself, path subs will still save you storage space. Additional XBMC clients will attempt to re-download the same image again, but it will be saving to the same location.
Storage size is a concern yes, but so is the fact that all clients will now have to spend time re-downloading the same fanart to the same location. This means each single client will have to go through the same update & cleanup run every time I store or remove stuff. This is in fact a step backwards from the current implementation. Yes it'll work, but it's hardly a very sensible or pretty solution.

The update only happens the first time the media is access, so it is not so bad, and if you really do not want to waste that time, the easiest solution is to have local images.

But I have a doubt here... what happens if a cached thumb is deleted? for example, XBMC caches image1 to the thumbnail dir. Then you delete by mistake the cached image, will XBMC detect that it has not it and generate the thumb again or will it just show no image as it happens in v10?
(2012-07-02, 11:18)jmarshall Wrote: [ -> ]Sorry, but we can't and don't attempt to cater to everyone.
Understandable, just stating the fact that this will impact all users running centralized thumbnails. May not be the majority but still.
Quote:If you don't want the hassle of many clients downloading the same thing, then copy the texture database from one machine to the next.
Hmm, would require some SQL scripting knowledge and a way of running it after an update, but it's doable I guess. Wondering if it could be made as an add-on...
Quote:But seriously, get some storage on those clients - it will speed things up a heap.
The fanart from the NAS is displaying instantly already, I fail to see how thing can be faster than 'instantly'. Even worse, if I did go with local fanart storage, it would mean I would have to wait for each client to download, maintain and store their own copy of the fanart (as stated earlier).
But don't we still need path sub in advanced settings as per this post: http://forum.xbmc.org/showthread.php?tid...pid1121164

(2012-06-06, 16:11)john.doe Wrote: [ -> ]You will still need path substitution.

The difference is that there's no longer any relationship between the path to the video file and what thumbnail is assigned. This means that as long as the two XBMC libraries used the same web URL for their image, they'll both share the same thumbnail hash, regardless of whether the media path is /storage/media/movies/somemovie.avi or /mnt/NetworkDrive/movies/somemovie.avi. This means that there's no duplication of thumbnail data when moving files around or accessing them from different paths to the same file, since the hash is now based on the web URL and remains the same. That's one of the changes it brings. Less duplication of data. Another (even bigger) benefit is that when you're scrolling through lists of remote images, it's able to easily crc32() the URLs and see if a locally cached version already exists. Previously, when the thumbnails were named after the video filename hash, that wasn't possible; but now, with the URL-based hashes, it's easy to check if we already have a locally cached version of any URL we've previously downloaded, by simply hashing the desired URL and looking for that pre-cached filename in the Thumbnails folder.

However, to sync multiple libraries or share libraries, you'll still need <pathsubstitution> (http://wiki.xbmc.org/index.php?title=Path_substitution) and thumbnail path sharing to ensure that paths are normalized across platforms, in order to actually be able to ACCESS the media that the library references. I.e. the local library might refer to /storage/movies/coolmovie.avi, and you're accessing that library over a network which can find that file at /mnt/netstorage/movies/coolmovie.avi; so you'd use a path substitution for your network client to be able to translate those local paths in the library to the relevant network-paths.

An even simpler, non-synced case that also benefits from these changes is when you're having two nearly independent XBMC installations. A local one and a networked one.
* Local one: has a thumbnail cache and has all the media on its own local disk.
* The networked one: uses <pathsubstitution> to replace "special://masterprofile/Thumbnails/" with a writable network path to the thumbnail cache folder of the XBMC installation above, as well as scanning all media from a shared folder on the network.
In this setup, they'd share thumbnail caches, meaning that they don't have to re-download thumbnails that the other has already downloaded. However, they'd be independent in all other ways and would not know about each others library data, actual thumbnail assignments, etc. All they'd do is share media files and thumbnail cache to save some time.

So, no, you still need pathsubstitution just as much as before, but there's now much less data duplication no matter how you look at it.


I updated my MBP and my ATV2 to nightly today, and dumped my libraries so I could start fresh.
All of my media (Movies and TV) is stored in separate folders with .tbn , .nfo and fanart folders etc. on my QNAP NAS.
I use shared database with MySQL server running on the QNAP.
Everything shared using NFS.

Seeing as the MBP is a lot faster than the ATV2 I did the library scanning using XBMC on it. This was fast and painless due to my previously mentioned setup.

But, without the pathsubstitution in advancedsettings.xml no images / thumbnails would show up.

I had to copy to the MBP thumbnails folder to the QNAP, then enable the path subs I previously had with Eden on both the MBP and ATV2. Only then was all good with the world again.

The ATV2 took a while to load the images on first run, I had to gradually move through the Movies database to get the images to show up. I guess they were all caching locally on the ATV2. After first run it's definitely quicker than with Eden.

So, have I missed something entirely or have I set this up correctly?
That sounds like it's only needed when you are using a library that has local file paths instead of network file paths. If you are using network file paths then you won't need path subs at all, because all clients will see the media in the same network path.
But technically mine are using network file paths too. All media is on the QNAP, accessed from the ATV2 and MBP over the network. Correct?
Assuming you're using a central MySQL database, then you don't need pathsubstitution at all for the thumbnails, as the URLs to the source images are stored in the MySQL database.

Each client, then, will (as needed) download said images to it's cache (download here may be from your NAS if they're local). If it takes a while first time, then that is exactly what it's doing. The Debug Log will ofc show this.

Cheers,
Jonathan
Pages: 1 2