On 2014-08-21 15:31, Anton Aylward wrote:
On 08/21/2014 08:33 AM, Carlos E. R. wrote:
Yes, it is is
an issue on btrfs, as I demonstrated it with a test that I
published here. The filesystem crashes completely if you create "too
many" small files, and speed diminishes hugely before that. There is an
unsolved bugzilla on it.
Date on that? Details?
Date: Mon, 21 Oct 2013 00:52:15 +0200 (CEST)
From: Carlos E. R. <>
To: OS-fctry <opensuse-factory(a)opensuse.org>
Subject: [opensuse-factory] [13.1 RC1] I managed to destroy a btrfs -
anyone wants to investigate it?
Reported 2013-10-21, last activity 2013-10-22
> XFS adjusts the equivalent of inodes dynamically,
it adapts, but
> nevertheless, it has a limit. The recommendation is to tune the
> filesytem, on creation time, for many small files.
> ext2/3/4 has a fixed number of inodes, but it can also be tuned for a
> large number of small files.
> Reiserfs is the only filesystem that has no issues with the situation,
> and is way faster than the rest (in this situation). But development is
> stuck, and it scales badly (only one thread is used for all the reiserfs
> partitions you have, regardless of the number of cores you have). It is
> possible that it disappears from the distribution not far in the future
> - in 13.1 you can not create a reiserfs in YaST, for instance, support
> has already been removed there.
Have a look at this post:
Date: Tue, 6 May 2014 16:11:12 +0200 (CEST)
From: Carlos E. R. <>
To: OS-EN <opensuse(a)opensuse.org>
Subject: [opensuse] Total system crash when test writing to reiserfs
Basically, it happens when writing many files with dd and
"conv=fdatasync" to a reiserfs partition.
That's a good summary.
Many feel that BtrFS is going to replace ReiseFS and give their
rationalization, but as you point out its not quite there yet.
The standardizing/default of BtrFS and the removal or ReiserFS are
premature, and I say that as someone who supports BtrFS.
Well, I wanted precisely to test if btrfs could replace reiserfs in that
exact task, millions of small files in a single directory (if it works
in a single directory, it will work also on many directories: a single
one is more stress).
And it failed. And no, I'm not happy with it.
My own experimentation shows that BtrFS works best on
huge file systems
that have no physical partitioning. That is no separate /home/ usr,
/var and so on. It optimizes and balances the whole damn lot.
Yes, that's a design point. But it goes against my ingrained practices
of decades: having several smaller partitions, so that a problem in one
can not affect others.
I suspect, or is that fear, that the optimization of BtrFS will
eventually do this:
Oh look! That sector of that file and this sector of this
other file are the same byte-for-byte so lets link then both
to single copy and tag it for COW.
Maybe. Interesting idea.
And so all of the /usr/bin and /usr/lib will be
optimized to hell.
What I want is compression, and that part is not stable yet.
And I also want CRC of data on the metadata, and verified. XFS is
working on this idea, I think.
No, really, with all of a 4T drive as a single BtrFS
what's the odds
that there are two files with a sector the same? I run FSLint and I
find I have quite a number of duplicate files at the size&md5 level and
sometimes even the name is the same or similar. LOTS of them. More
than the space wasted by maidir messages not filling all of 4K
partitions, which is a non issue on my ReiserFS where INBOX lives
because of the way, as you say, Carlos, that ReiserFS deals with inodes
and how it packs small chunks. Aren't Btrree's wonderful?
Sometimes the way BtrFS is developing scares me.
Yes... kind of, yes....
That reminds me. One project I have on very backstage-low-priority is
something to generate checksums of files on given paths, then locate
files with the the same checksums, and offer to rename them the same
way. I need that in order to sync several copies of the same files on
different disks, so that the name match and are easier to locate (the
files are very big, perhaps a terabyte or two in total, so copying them
from place to place, although works, is slow and a pain).
> Nor is the
'growth' of the size of the directory & compacting it nor, if
> you set up the options to start with, sorting huge directories.
The problem with ext-series is that you have to know about it at Mkfs
time. Yast and Installer aren't great about advising you on this issue
and the values in mke2fs.conf are someone else's idea of what you should
use; not as flexible as ReiserFS.
It is an issue
in the sense that it impacts performance a lot. You only
need to run tests on large mail folders using several formats, and
compare. Some workloads run faster on maildir, and some on mbox.
"Context is Everything".
Which is why I use both maildir and mbox .... in different contexts.
Cheers / Saludos,
Carlos E. R.
(from 13.1 x86_64 "Bottle" at Telcontar)