"Carlos E. R." wrote:

On 2013-10-23 12:59, David T-G wrote:

...

Any thoughts, shy of religious warfare, on the best FS to use for a data and archive server? [I promise this is it for me for today ;-]

XFS?

10 years ago I researched the same question and chose xfs. I've followed both xfs and ext3/ext4 development since then and would still pick xfs of the 2 (for the described workload). I think Per likes JFS for fileservers, but I don't know much about it. I don't think btrfs is the right choice for a archive server. Greg -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 2013-10-23 14:52, Ruediger Meier wrote:

Also a few times xfs made me a lot trouble after power failures. So I can't confirm the reputed robustness of XFS.

ext4 is not safe to be used as NFS server, at least the dir_index has to disabled. (Maybe this problem was fixed in recent kernels.)

I had filesystems totally destroyed because power failures and other causes, in xfs, reiserfs, and ext3. All are vulnerable. Ah, I managed to break btrfs too, beyond repair. -- Cheers / Saludos, Carlos E. R. (from 12.3 x86_64 "Dartmouth" at Telcontar)

On 10/23/2013 06:12 AM, Anton Aylward wrote:

I've broken BtrFS by power outage when ReiserFS has survived. Actually ReiserFS seems better at automatically recovering from systems crashes and power outages, than ext3/4 and BtrFS, but don't take that as an absolute, only a relative.

fwiw... I have yet to lose /anything/ on the reiserfs partitions. I was constantly losing files w/ ext[3|4], even without the power failures, crashes, etc. And after one kernel oops, lost the entire root partition. Gone. jd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Wed, Oct 23, 2013 at 8:52 AM, Ruediger Meier wrote:

On Wednesday 23 October 2013, Greg Freemyer wrote:

...

"Carlos E. R." wrote:

...

On 2013-10-23 12:59, David T-G wrote:

...

Any thoughts, shy of religious warfare, on the best FS to use for a

data

...

and archive server? [I promise this is it for me for today ;-]

XFS?

10 years ago I researched the same question and chose xfs. I've followed both xfs and ext3/ext4 development since then and would still pick xfs of the 2 (for the described workload).

XFS is _extremely_ slow when creating/deleting files, AFAIR something about factor 100! worse than the ext family. That's why I would not used it for (rsnapshot) backups and most other use cases.

I think you recall from 2003 not 2013! As 18 months ago, the delta was almost gone. At that time Dave Chinner (one of the core XFS developers) was argueing that he only saw a multi-year future in XFS and BtrFS. As such his argument was people should move to XFS if they wanted short term stability because it was going to be around and well supported for years. Then for desktops and small servers, migrate them to BtrFS when it was ready. I'll let you read some of it yourself:

From http://lwn.net/Articles/476565/ Dave Chinner says:

==

to summarize how I understood you: you recommend using XFS for "big storage" systems, while -- for the time being -- the desktop use-case is still better served by ext4?

No, that's exactly the opposite of what I am saying. My point is that even for desktop use cases, XFS is now so close ext4 performance on single threaded metadata intensive workloads that it ext4 has lost the one historical advantage it held over XFS. The example I use in the talk is untarring a kernel tarball - XFS used to take a minute, ext4 about 3s. That's one of the common "20x slower" workloads that people saw all the time. Now XFS will do that same untar in 4s. And the typical 50x slower workload was then doing a 'rm -rf' on that unpacked kernel tarball. XFS has gone from about a minute down to 3s, compared to 2s for ext4.... While these XFS numbers are still slightly slower from a "benchmark perspective", in practice most people will consider them both to be "instantaneous" because they now both complete faster than the time it takes to type your next command. IOWs, users will notice no practical difference between the performance of the filesystems for common desktop/workstation usage. Let's now fast forward a few months: your desktop and server system filesystems will be BTRFS, whilst XFS is fast enough and scales well enough for everything else that BTRFS can't be used for. I can't see where ext4 fits into this picture because, AFAIC, XFS is now the better choice for just about every workload you wouldn't use BTRFS for.... So I'm not sure ext4 has a future - ext4 was always intended as a stop-gap measure until BTRFS is ready to take over as the default linux filesystem. This milestone is rapidly approaching, so now is a good time to look at the future of the other filesystems that are typically used on systems. Everyone knows what I think now, so I'm very interested in what users and developers think about what I've said and the questions I've posed. The upcoming LSF workshop could be very interesting. :) Dave. ======= -- Greg Freemyer -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On Wed, Oct 23, 2013 at 11:16 AM, Ruediger Meier wrote:

...

...

XFS is _extremely_ slow when creating/deleting files, AFAIR something about factor 100! worse than the ext family. That's why I would not used it for (rsnapshot) backups and most other use cases.

I think you recall from 2003 not 2013!

No, from 2011, kernel 2.6.37. Even I re-validated my results using 3.0.80-52, which is the up-to-date kernel of SLE11.

I consider it my own huge mistake that I've started using xfs at all without benchmarking before. It was a lot work to migrate back and there are still some xfs partitions left which are as slow as measured in 2011.

You might want to read http://lwn.net/Articles/476263/ Per it, the last of major changes to make file creation / deletion went into the 3.3 kernel. Even openSUSE 12.2 uses the 3.4 kernel, so openSUSE has had the improvements in both of the last 2 major releases and of course will have them in 13.1 due out next month. Greg -- Greg Freemyer -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Duaine Hechler said the following on 10/23/2013 01:18 PM:

FWIW, I have never lost any files with reiserfs - however - I have lost files with ext2 and ext3 - and scared to try ext4

Maybe I've been lucky, but while I've had to run fsck a lot I've never (noticed any) lost files on ReiserFS. When I have set up ext3/4 FS I keep finding files in lost+found at the least. So at the very least, ReiserFS is better at preserving linkages. -- Never tell people how to do things. Tell them what to do and they will surprise you with their ingenuity. -- George S. Patton -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 10/23/2013 08:24 AM, David T-G wrote:

From what I can tell, XFS is a big contender, ReiserFS isn't bad, and Carlos can break pretty much everything:-) Barring more "XFS really is still slow" input, I think, at this point, that I'm going to lean toward XFS even though it's new to me, especially if I also give LVM a try and it does my mirroring so that I have the ability to break, reformat, transfer, and sync if I need a change (and, hey, manage volumes on the fly to tune their different purposes, too).

<- snip -> Coming in a bit late. I did benchmark XFS, EXT4 and btrfs recently for our application and found that XFS won. We're having to write binary data as quickly as possible to RAID filesystems exceeding 17-TB in size. The file sizes are all 4-GB. The data is coming from two optimized 1-Gig Ethernet (fiber) interfaces, so we don't have to read/write while collecting data. BTW, we're able to get continuous bandwidth of 1,900-Gbits/sec over the two interfaces. I don't have the numbers available now, but XFS was able to write to an 11-disk RAID-6 array that netted about 17-TB of space at a rate of about 1.6-GB/sec. EXT4 was a tiny bit faster, but it had more storage overhead. Btrfs completely failed at about 15-TB. So XFS won (again) for our job mix. Reiserfs wasn't considered here because it doesn't support partitions larger than 16-TB. FWIW, we've been using XFS on this project for seven years and I don't recall ever having a non-hardware related loss of data. We did have filesystem corruption caused by disk failures on JBOD drives without RAID protection, but no unclean shutdown loss of data. Tests were run on openSuSE 12.3 and 13.1-Beta1. (Yes, I did a bug report on the btrfs failure in 12.3 in July of this year, it's still there in 13.1-Beta1.) Regards, Lew -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Anton, et al -- ...and then Anton Aylward said... % % David T-G said the following on 10/23/2013 11:24 AM: % > % > From what I can tell, XFS is a big contender, ReiserFS isn't bad, and ... % >it does my mirroring so that I have the ability to break, reformat, % >transfer, and sync if I need a change (and, hey, manage volumes on the % >fly to tune their different purposes, too). % % I still read a subtext that your thinking one filesystem to rule them % all, and possibly just one partition. % % Tell me it ain't so, Joe! It is for now; see my other note for an explanation of why. But I admit that I also don't like the endless management of deciding how much space to allocate for this or that, even if it is easier to adjust with LVM and modern filesystems. Hey, who was it who keeps telling me to not micromanage? ;-) In the end, I think that I look forward to having some large slice(s) for mostly-static archiving and backups as well as some smaller, snapper-ed, COW FS slice(s) for more dynamic content and to managing this under LVM. I think I'm going to get away with having my 500G + 320G (oops; couldn't make the 400G drives work) pairs in there as scratch or other space, so I just might be able to have them available to hold content while I resize and reallocate, which will be helpful, too. % ... % Most of my server's partitions are ReiserFS 'cos its a joy to watch them % come back so quickly and faultlessly after a powerloss :-) Funny you should mention that... I decided last night, when formatting up both new drives, to make one XFS and the other ReiserFS. I then started multiple copies from half a dozen source drives along the lines of for V in XFS RFS do (cd /path/to/drive && tar cpf - .) | (cd /tmp/$V/drive && tar xpf - .) & done and let it chew overnight. After about six hours, I now see that the XFS disk has about 600G on it while the ReiserFS disk has about 1.5T on it. Curious... That's writing, which won't be as fast as reading, and it's interrupted by having multiple streams come in, but the difference is intriguing. % % Oh, and read up the docs, the man page and the various How-To on LVM. ... % The truth may or may not not be out there, for various values of % 'truth', but there are some good examples and illustration that go into % a lot of detail. [snip] Thanks, and I will. Of course, another part of my mental challenge is that if I go digging too deeply I'll get sucked in and then have to find The Right Way, and we all know how fruitless that search is :-| Worse yet, I don't have that kind of time; I want it all, and I want it NOW. Thanks again & HAND :-D -- David T-G See http://justpickone.org/davidtg/email/ See http://justpickone.org/davidtg/tofu.txt

ArnoB wrote:

Hi David,

I'm surprised no one mentioned ZFS yet...

Maybe because it's so brand-spanking new? From the wiki: "OpenZFS was announced in September 2013 as the truly open source successor to the ZFS project." Is there an openSUSE package ready to install? -- Per Jessen, Zürich (13.6°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 23 okt 2013, at 20:40, Per Jessen wrote:

ArnoB wrote:

...

Hi David,

I'm surprised no one mentioned ZFS yet...

Maybe because it's so brand-spanking new? From the wiki: "OpenZFS was announced in September 2013 as the truly open source successor to the ZFS project."

yup that's true, that website is very new ZFS is not though... its development started around 2001 at Sun and is currently a very mature, production proven filesystem prepared for the future. gr arno -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 24 okt 2013, at 11:01, Dave Howorth wrote:

ArnoB wrote:

...

I'm surprised no one mentioned ZFS yet...

...

Runs on OSX, Linux, Illumnos/Solaris, FreeBSD... and it loves old disks ;)

Not to mention LOTS and LOTS of memory! xfs_repair can eat memory, but it's got nothing on ZFS.

yes it needs a lot of memory with the idea that memory is affordable these days. but you can tweak zfs to use the drives as cache, which might be reasonable on backup servers. disadvantage of xfs: you can't tweak data integrity corruption, which is quite important on backup systems if you'd ask me :) gr arno-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org