On Sun, Oct 18, 2015 at 10:31 AM, Carlos E. R. <robin.listas@telefonica.net> wrote:
On 18/10/2015 12:43, Per Jessen wrote:
The situation I have is as follows:
I have a 600Gb filesystem mounted on /var - key directories under /var are:
/var/spool/postfix-in/{incoming,active} with 2 levels of hashing. On a busy day or after delays, each subdir might easily reach 10.000 files. 99% small files of less than 100Kb. (emails).
/var/spool/elsewhere/dir{0,1,2,3...1000}/maildirs - each such maildir might have 100.000 files.
It would make sense to test several filesystem types with that workload. Years ago I would have said without hesitation "use reiserfs". Nowdays, I don't know which to suggest. Perhaps XFS.
But your system is in production, so you can not do such changes...
XFS implemented a very cool journal sort algorithm a few years ago. I guess we all know the kernel does an elevator sort on i/o when it can thus if it needs to write data to sectors 100. 9900.200.9800.300.9700, it will sort the actual writes to the drive to be 100,200,300,9700,9800,9900. That can also happen at the drive level. Doing so can significantly cut down on seek times, especially if some of the writes are to the exact same track. For whatever reason processing the XFS Journal was a very slow process and the lower level elevator sorts weren't helping much. in the 2005-2010 timeframe XFS was known to extremely slow on large deletes. Thus deleting the kernel source tree could take a long time relative to other filesystems. The XFS team (led by Dave Chinner?) implemented a process / algorithm in XFS that allowed the filesystem code to elevator sort the journal before pushing it to the hard drive for the final write.I believe they also combine updates to inodes that are adjacent to each other into a single bigger i/o. Disk drives often work better with larger i/o's. The data shows on more or less single threaded apps it now runs at ext4 types of speeds, aybe a little slower. Pretty good, but not necessarily anything to write home about. On high thread count apps such as a busy mail server like Per is describing, as of 3 years ago XFS is supposed to be much faster than ext4, etc. This takes an hour to listen to and is 3 years old, but is well worth it if you want to see XFS compared to EXT4 and BTRFS. I don't recall JFS being in the benchmarks. https://www.youtube.com/watch?v=FegjLbCnoBw The benchmarks start about 20 minutes in if that is all you want to see. Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org