Kodi Community Forum

Full Version: Faster Library scan ( Big library) - Ideas
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Been using xbmc over a year now and library is getting quite large 600+ movies and 6500+ tv shows.

It now takes forever to the auto library scan to complete when I launch xbmc to watch latest movie or tv show. Since I have not found a way to run xbmc from the task-bar I cant leave it always running as it gets in the way of normal computer browsing.

Is there any advanced techniques speed up the hourly scan of the harddrive folders?

Could I perhaps sort out old Tv shows that have no chance of having new episodes into a separate folder and choose do not scan for changes? Has anyone tried this? is there any cons before I go ahead and try this?

Perhaps choose not to scan all 600 old movies?

Also forgot to mention I'm running a mysql database on top of this as well =S
Hi,

I was in the same situation as you but had 2 instances of XBMC, Raspberry Pi in the bedroom and XBMCbuntu in the living room.

Anyways, all my videos are host on a micro-server running WHS2011, so I decided to give MySQL a go. It feels so much quicker!
use some form of external scraper perhaps?

http://wiki.xbmc.org/index.php?title=Sup...y_managers
You can split tv shows into 2 folders - 1 for shows that will never update with updates disabled - with no adverse effects.
You could also set movies in a main folder - doesn't update - and a 'new movies' folder which does. Not sure how much time that would save given it's a much smaller number, but it wouldn't do any harm. Every now and then you could move stuff into the main movies folder and clean / re-scan the library.
As long as you're only using library views the only difference with those changes would be faster scans.
It took me ages to get the library to scan and update how *I* wanted it to, without it taking an eternity, and needing to Update when I started XBMC, which was not what I want after a long day and just want to watch some TV.

I tried various Media managers (Media Companion, Ember etc) without finding a satisfactory solution, so in the end I decided to Export the entire TV and Film library (800+ Films, God only knows how many TV series)

So did a full and up to date scan of both, then exported as seperate files. Now, when I update the library, it only scans for NEW content. As an example, I just did an update, it found 3 new movies, a few new TV episodes, and a complete series (4 seasons, 2 specials) and it took just 8 minutes, then did an export, so the new stuffs relevant metadata is exported into the relevant folders, and that took another 8-9 minutes.

Just for ease, I then created a pair of scripts to do this automatically from a key press on my TV's remote control (Raspberry Pi running OpenELEC by the way) I also created a pair of new sub-headings (Update Library, Export Library) in the exit menu. For me this works perfectly, and just a LOT easier than any other way I've found.
Had the same problem as you are having with 1500+ Movies 4500+ TV Ep's and 6000+ Anime Ep's. Had no option than to separate them in to different profiles.

I now have 5 Profiles
1. HD Movies
2. SD Movies
3. TV Ep's
4. Doco's
5. Anime

Will need to separate Anime Movies and Anime Ep's into different Profiles before long.
if you have mutliple drives or folders set as source you can select the ones that are full or no longer need to be scanned and set them to be excluded from scanning i do this for drives that have been filled up and had a final scan through they do not need to be constantly scanned if there is nothing new on them so they get skipped and just the latest couple drives get scanned on startup
Thanks everyone for confirming my suspicions. Will work on implementing this one weekend, hopefully I don't take out my whole library Wink
(2013-10-21, 07:21)protocol77 Wrote: [ -> ]if you have mutliple drives or folders set as source you can select the ones that are full or no longer need to be scanned and set them to be excluded from scanning i do this for drives that have been filled up and had a final scan through they do not need to be constantly scanned if there is nothing new on them so they get skipped and just the latest couple drives get scanned on startup

THIS. Stranger, I want to kiss you <3

This solves my problem. Luckily I, for no particular reason, group the movies by the year and the month I download them, so the movies I am downloading right now are in ~/2015/03. I can easily exclude the folders where I'm not going to be putting new media again. Update times are really short now, like 40 seconds. I knew why it took so long, I just didn't know how to work around it. It's because the scrapper goes through every folder, checks to see if there is any new media to add and then goes on to the next folder. There usually are many folders, since in most torrents movies are enclosed their own folder. This adds up.
(2015-03-21, 05:12)rpgmaker Wrote: [ -> ]
(2013-10-21, 07:21)protocol77 Wrote: [ -> ]if you have mutliple drives or folders set as source you can select the ones that are full or no longer need to be scanned and set them to be excluded from scanning i do this for drives that have been filled up and had a final scan through they do not need to be constantly scanned if there is nothing new on them so they get skipped and just the latest couple drives get scanned on startup

THIS. Stranger, I want to kiss you <3

This solves my problem. Luckily I, for no particular reason, group the movies by the year and the month I download them, so the movies I am downloading right now are in ~/2015/03. I can easily exclude the folders where I'm not going to be putting new media again. Update times are really short now, like 40 seconds. I knew why it took so long, I just didn't know how to work around it. It's because the scrapper goes through every folder, checks to see if there is any new media to add and then goes on to the next folder. There usually are many folders, since in most torrents movies are enclosed their own folder. This adds up. 
 Problem with that Solution is, that if you habe a new Folder for new Stuff youll never be able to movie the files from there to a other Folder e.g. TV Shows, or Dokumentations...e.g.
Normally you first order by Type (e.g. TV Shows, Cartoons, Movies...) not by Drive Number.

The Only solution here seems to be give MySQL a go, but that means you have to run an new Server / PI / whatever 24/7 to just do that.