Kodi Community Forum

Full Version: Optimisation tips for large libraries
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hello, I'm accessing my music and videos over NFS from a separate server and for the most part everything is fine. However there are times when loading my libraries which are in the thousands of movies, tv and music files, can be a little slow and take a few seconds to navigate and load images.

I'm just wondering if anyone has any generic tips for making a speedy XBMC with large libraries? I've tried a bunch of things, but I'm sure there's some more tweaks I haven't thought about.

Cheers.
I’d also like to know this.

There are different ways of organising /scraping storing our media that just work, but if there was a best way for speed then I would alter my media to suit.

I’m in the process of organising my media and would like to get it perfect first time.
You could rewire the universe so that electrons travel faster...
The delete button and a slight "Meh" attitude.

What works for some might not work for others. But this is how I roll...

NFS
MySQL

/Movies/A Knights Tale/A Knights Tale.avi
/Movies/A Knights Tale/Extra Fanart

So... Folder per movie, Everything in that folder. Including actor thumbs folders.

TV Shows/The Big Bang Theory/Season1/The big bang theory s01e01 - episode title.avi
TV Shows/The Big Bang Theory/info.nfo
So all the episode specific data (nfo and thumbnail) are kept with the media file in the season folder. All the show specific data, season thumbs, fan arts, etc are all in the root folder for the show.

I've heard it's "Faster" to skip the season folders and just dump everything into the root show folder, But some of my shows have 8+ seasons, and the show root folder gets cluttered without separate season folders. I do it this way cus I don't use any automation like sickbeard, and I need to be able to manually navigate my TV show folders.

You find yourself using things like "recently added" and the global search options rather than scroll thru A huge library.

The biggest bottleneck I found was with multipart sources. (IE my movies is 2x2tb drives, so 2 sources linked to the 1 source in xbmc)
Rather than mapping both movies drives to a single movies source, Mapping them as separate sources with the same content set, it only scans the drive that has changed upon updates, and it makes scraping ever so slightly quicker. (It's not scrapping nothing for ages before finding the new content). Does that make sense? I swear I know what I'm talking about/mean.

I've got 3k+ movies and 200+ Shows, and mines very quick, updates take less than a min usually, and a full movie list browse takes less than a second anywhere on the network.
•Turn off Settings -> Videos -> File lists -> Generate thumbnails (it also extracts other data, even if you have thumbnails, IIRC)
•Turn off fan-art in skin settings

Still need more?
•Lower thumbnail size in advancedsettings.xml (wiki)
1)
Split your sources into two folders:

- TV-shows finished
- TV-shows active.

Scrape both sources once, then disabling scraping for updates on the source "TVshows finished".
It only is a tiny difference but it happens every time you scrape your source.

2)
Export library to individual file regularly, so you can rescrape your library faster.
Use the <importwatchedstate> in advancedsettings to read watched state and playcount / last playdate from NFO files when importing, handy when moving stuff around.

3)
When using a MySQL server do not use pathsubstitution for your thumbnails folder. Different hardware might have different capabilities concerning thumbnails, which is why the intention is that every device has its own thumbnail cache. If you generate your thumnails in high quality on 'powerful machine a' and then use those generated thumbnails on 'low power device b' you might not have the performance on that machine you were looking for.
Cheers for the advice, just wanted to understand something, does exporting speed up loading of images and nfo, I assumed it kept them somewhere on my my SSD which I thought would be faster than retrieving them over the network.
No. Those files are only used during a library scan for the first time.
I suggested #2 because you might run into a situation where you need to reinstall XBMC or the XBMC machine gets corrupted or whatever. In my case rescraping my library takes hours when done over the internet, and minutes when done locally because i regularly export my database as seperate files which are stored on my (hadrware / fast) NAS.

Exporting only speeds up regenerating the entire database when it was lost (consider it a backup of your metadata).