When the next stable is released I am porting from windows to Linux.
Having a huge library I am looking to choose the optimal filesystem that best matches having 100,000 thumbnails.
We all know NTFS sucks at this so finally that will be gone.
Based on some light google reading resiserfs seems to have a good reputation for this task.
I am looking for any tips/user experiences on this.
For the record changing hardware a.k.a get SSD or raptors rock are storys for another day and not what I am asking
Obviously this will be for Ubuntu 9.10
ta
I would like to know the same. However my stock is constantly rotating on a network attached fat32.
Have you considered booting from a live install?
I think you're right. Reiserfs with the noatime mount option set is about as fast as it gets when it comes to working with lots of small files.
Seems the:
notail increases performance of ReiserFS (at the expense of storage efficiency).
Can make it even faster. So reiserfs makes the top of the list so far.
Anyone else got any suggestions?
ext4 is plenty fast enough for reads
perhaps a better solution would be to put the cache into a database of some sort, or some other single or smaller number of containers (zip/rar/tar/etc).
though this doesn't really answer your question
dc2447 Wrote:ext4 is plenty fast enough for reads
Is that a statement that ext4 would be better than reiserfs in this usage scenario or just statement that "ext4 is good".
To give you an example under ntfs my library can take 30+ seconds to open the tvshows view being predominantly bottlenecked with thumbnail operations.
The question really is what fs action is causing this bottleneck and from that a measured conclusion of the best fs can be drawn.
xexe Wrote:Is that a statement that ext4 would be better than reiserfs in this usage scenario of just statement that "ext4 is good".
To give you an example under ntfs my library can take 30+ seconds to open the tvshows view being predominantly bottlenecked with thumbnail operations.
The question really is what fs action is causing this bottleneck and from that a measured conclusion of the best fs can be drawn.
If you don't know what is causing the delay how do you know it's the filesystem at all?
Reiser is pretty niche these days, ditto jfs and xfs which can do faster reads that ext journalling filesystems.
It was over a minute and an ntfs defrag halved the delay time.
Opening the main tv show view in library is > 1000 tv shows a.k.a alot of thumbnail actions. Remove the thumbnails and do the same view and response time increases massively.
I have also had many discussions in irc with developers to come to this conclusion.
So i do not expect miracles but since placing the userdata in a tuned fs is a trivial task it makes sense to make an informed decision on what fs would be best.
You can't polish a turd but other than using ramdisk I'd use ext4.
Opening a 1000 files in one go won't be fast
Ramdisk is not an option i don't want to use a 64 bit OS.
There is no reason to not research the best fs rather than blindly choosing the defaults or personal favourite. A 3 second response improvement is a full 10% improvement and that is not even vaguely out of realms of possibilty
Looking at:
http://linuxgazette.net/122/piszcz.html
we can clearly see certain fs blow others away for certain actions and that is what we are addressing here.
dude, you could have a ramdisk on 32bit OSs.
You have to way up a 10% improvement against knowing how to repair jfs without fsck or what happens when your XFS filesystem gets full...
dc2447 Wrote:dude, you could have a ramdisk on 32bit OSs.
Not if i want any ram left for the OS and XBMC to use i cant.
dc2447 Wrote:You have to way up a 10% improvement against knowing how to repair jfs without fsck or what happens when your XFS filesystem gets full...
Valid point however all userdata is backed up centrally so is completely expendable. This is part of the scheme to synchronize all my XBMC instances thoughout my home.
So as per OP i only care about maximum performance for userdata contents
xexe Wrote:For the record changing hardware a.k.a get SSD or raptors rock are storys for another day and not what I am asking
Quote:So as per OP i only care about maximum performance for userdata contents
while I certainly understand the desire for figuring out an optimal filesystem for the specific situation of thumbnail caches, an ssd will by far be the greatest possible improvement for 'maximum performance' you will get regardless of filesystem.