[N4TH4N]

QUICK SOLUTION: Delete your local *.jpg, *.tbn and *.nfo files if using metadata stored with the media and rely on XBMC scrapers. Set up in combination with a proxy/web cache server to solve the problem of multiple downloads and loading speed of thumbs.


Hello,

I have 6 media centres all running Windows 7 + XBMC

I have setup a dedicated MySQL database and SAMBA share (for "Thumbnails") on CentOS 6.4 Minimal.

There are 5 databases: "a", "b", "c", "d" and "e".
Databases: "b", "c", "d" and "e" are exact clones of "a" except they each have there own individual watch status.

- Master (Always on. Partial updates when notified. Full update and clean daily)
- "a"

- Lounge
- "a"
- "b"
- "c"
- "d"
- "e"

- Garage
- "a"
- "b"
- "c"
- "d"
- "e"

- Bedroom 1
- "a"

- Bedroom 2
- "b"

- Bedroom 3
- "c"

- Bedroom 4
- "d"

Note: "e" is just a guest account.

So now you see that i have 15 different "Thumbnail" and "Database" folders you will see why its necessary to share thumbs.

I have setup the Lounge and Garage computers to share "Database" folders between profiles by symbolic link, reducing the "Database" folders to 7. This seems to be working.

You cannot share the "Database" folders between computers as it can only be open by one XBMC at a time. Therefore each computer needs its own "Database" folder.

I have also setup all computers and profiles to symbolic link to "Thumbnails" on the SAMBA share in the hope that i wouldn't have to spin up disks just to get thumbs.

This does not seem to be working 100% and disks are being spun up to load thumbs on different computers, even though they have been previously loaded on another.

Is there anyone else in this situation with a better way of doing it, or a solution.

I plan to keep playing with it, ill post up once i find a solution.

Thank-you.

[Ned Scott]

When you say it spins up disks, do you mean the local ones on each XBMC instance? If so, that's fairly normal. XBMC still has a local textures13.db file that keeps track of the thumbnails, so it might just be looking at that file to see if there are any missing thumbnails, etc.

[N4TH4N]

When you say it spins up disks, do you mean the local ones on each XBMC instance? If so, that's fairly normal. XBMC still has a local textures13.db file that keeps track of the thumbnails, so it might just be looking at that file to see if there are any missing thumbnails, etc.

I have all of my content stored on a network drive which consists of 10 disks (unRAID). The disk sit dormant in sleep until XBMC requests a file to play.

It also uses something called "Folder Caching" which caches the folder/file structure when browsing files in explorer/finder. So disks remain sleeping.

The problem arrises when XBMC loads thumbs, its spinning up to 10 disks just to load thumbs, even though they have been previously loaded on a different computer and saved to a central "Thumbnails" directory. XBMC keeps insisting that it needs to load them again.

Im guessing from what someone else posted, that XBMC uses unique filenames for the "Thumbnails" directory, or that it just decides to get a fresh thumb rather then the one from the "Thumbnails" directory.

[Ned Scott]

Ah I see. So this would likely happen with it or without pathsubs. I'm not sure what the solution would be for this case. XBMC does check files for various things, including seeing if something is missing or has a new thumb, at least once a day. Is it happening more often than that? I'm not sure on the full technical details, so this could be something else that I'm not thinking of.

[N4TH4N]

Ah I see. So this would likely happen with it or without pathsubs. I'm not sure what the solution would be for this case. XBMC does check files for various things, including seeing if something is missing or has a new thumb, at least once a day. Is it happening more often than that? I'm not sure on the full technical details, so this could be something else that I'm not thinking of.

Thanks for the reply.

For example i was just looking at my recently added list in the garage no more then an hour ago. I'm now in my bedroom, when opening the recently added list the Thumbnails loaded again. I noticed the 3-5 second delay whilst the disk/s spun up.

Now that my bedroom client has loaded these thumbs they will continue to load instantly from this client.

If i went back to the garage, it would also be fine because its seen them before also.

It just seems to do checks when it loads them for the first time. Even if they already exist in Thumbnails.

[Milhouse]

You could try pre-loading the texture cache on each of your clients, and see if that helps. See sig for details.

If each client is randomly populating the texture cache with artwork as users browse the shared library, it's very likely that your disks will have to be spun up as the client attempts to display artwork that isn't yet in the cache - the original artwork will have to be retrieved before it can be stored in the cache and then subsequently displayed to the user.

Pre-loading the texture cache may stop the spin-up behaviour as each cache on each client will be always be fully populated. Although I'm not sure if the periodic hash-checks will continue to spin up the drives - it maybe an idea if these checks could be stopped with an advancedsetting, if the cache is being "manually" managed. Or hash-check only remote artwork, but not local/LAN based artwork.

[N4TH4N]

You could try pre-loading the texture cache on each of your clients, and see if that helps. See sig for details.

If each client is randomly populating the texture cache with artwork as users browse the shared library, it's very likely that your disks will have to be spun up as the client attempts to display artwork that isn't yet in the cache - the original artwork will have to be retrieved before it can be stored in the cache and then subsequently displayed to the user.

Pre-loading the texture cache may stop the spin-up behaviour as each cache on each client will be always be fully populated. Although I'm not sure if the periodic hash-checks will continue to spin up the drives - it maybe an idea if these checks could be stopped with an advancedsetting, if the cache is being "manually" managed. Or hash-check only remote artwork, but not local/LAN based artwork.

Thanks for the info, ill have a play around.




Also, further testing suggests that its loading straight from the share rather than the Thumbnails folder. Ive found tones of duplicates in the Thumbnails folder, which suggests that the filenames are different (possibly random) on each computer. Making a shared Thumbnails folder useless. Can anyone confirm this ?

[Milhouse]

It just seems to do checks when it loads them for the first time. Even if they already exist in Thumbnails.

Of course, because although the images may already exist in your shared thumbnail folder, each client has it's own local database (Textures13.db - the "texture cache") which is checked first by the client to see if there's a row for the artwork (ie, if the artwork has already been cached by this client) and if no row exists a row will be added, and the newly resized/resampled artwork written out to your shared thumbnail folder (only one image will be stored if your media paths are identical for all clients/databases).

The texture database on each client is guaranteed to be out of sync with the same cache on any other client, meaning that clients will cache artwork repeatedly until it has been "seen" by all of the clients.

Assuming you have an automated process to handle the addition of new content to your server and then update each of the MySQL databases, it would be trivial to automate clients so that they update their local texture cache with any newly added artwork - you can do this entirely remotely, from the server, by just iterating over the different client addresses and calling texturecache.py, and even wake the clients if they're asleep then suspend them again.

---

Also, further testing suggests that its loading straight from the share rather than the Thumbnails folder. Ive found tones of duplicates in the Thumbnails folder, which suggests that the filenames are different (possibly random) on each computer. Making a shared Thumbnails folder useless. Can anyone confirm this ?

If the media paths are exactly the same on all clients then the artwork should be stored only once in the thumbnail folder - it may be written several times (once by each client) but stored only once so no, there should be no duplicates. However if you have got duplicates then that would suggest one or more clients has a slightly different path to the media, resulting in different filenames being generated for the thumbnails (the thumbnail filename is a hash of the artwork path).

Personally I wouldn't bother with path substitution for the thumbnails folder, it causes as many problems as it solves... just keep the thumbnails entirely local if you can. Prune the cache fairly frequently if space is even an issue, although a 16-32GB USB3 flash drive (pretty cheap, no spin-up!) should be sufficient unless the library is particularly large.

[N4TH4N]

Assuming you have an automated process to handle the addition of new content to your server and then update each of the MySQL databases, it would be trivial to automate clients so that they update their local texture cache with any newly added artwork - you can do this entirely remotely, from the server, by just iterating over the different client addresses and calling texturecache.py, and even wake the clients if they're asleep then suspend them again.

Thanks for the idea, but i turn my devices off at the wall with smart power points.

I have my PC's set to resume on AC power failure and with the addition of a Pulse Eight USB CEC it turns my TV and amp on as well. So network waking is not really an option.






UPDATE: I have removed my local metadata saved with the media and am relying on XBMC scrapers. This means that XBMC wont spin-up my HDD's it will download it from the internet.

To solve the problem of multiple downloads, i'm sharing the "Database" and "Thumbnails" folders between profiles on both the lounge and garage computer. Since only one profile is logged in at any given time it seems to be working fine.

Also i setup a local proxy/web cache and assigned it 10GB for the XBMC clients so it caches thumbs, so that they are fetched less frequently.

SOLVED: The combination of web based thumbs and the proxy/web cache seem to work quite well.