[opensuse-kernel] BtrFS as default fs?
Hi all - Last month I posted queries to this list (and several other locations, including the forums) asking about people's experiences with btrfs. For the most part it seemed like the experience had improved over time. Most of the concerns were either with interactions with zypper or old perceptions of instability that were based more on old impressions than new testing. With the exception of an ENOSPC issue that had been recently fixed, users actively using the file system seemed pretty satisfied with it. I posted a followup question a week or two later asking what people thought about limiting the 'supported' feature set in the way we do in SLES so that it's clear to all users which parts of the file system are considered stable. A quick table of what that looks like: Supported Unsupported --------- ----------- Snapshots Inode cache Copy-on-Write Auto Defrag Subvolumes RAID Metadata Integrity Compression Data Integrity Send / Receive Online metadata scrubbing Hot add/remove Manual defrag Seeding devices Manual deduplication (soon) Multiple devices "Big" Metadata (supported read-only) Over time this table will change. Items from the Unsupported list will move to the Supported list as they mature. That proposal was pretty well received except, predictably, by those using the features listed. In practice, all that's required for those users to continue uninterrupted is to add the 'allow_unsupported=1' option to the btrfs module either on the kernel command line or /etc/modprobe.d. There is nothing inherently limiting to any openSUSE user with this practice. The features are all still in the code and available immediately just by setting a flag. It can even be done safely after module load or even after file systems that don't use the unsupported features have been mounted. I intend to introduce this functionality into openSUSE soon. One other aspect to consider: Even though they are independent projects, we've been focusing heavily on btrfs support in the SLES product. As a result, the openSUSE kernel will end up getting much of that work 'for free' since most of the same people maintain the kernel for both projects. So that's the "why it's safe" part of the proposal. I haven't gotten to the "why" yet, but then you probably already know the "whys". Subvolumes. Built-in snapshots that don't corrupt themselves when an exception table runs out of space. Built-in integrity verification via checksums. Built-in proactive metadata semantic checking via scrubbing. Online defrag. Soon we'll see online deduplication of arbitrary combinations of files. The code is written, it just needs to be pulled in. You've seen the rest of the feature set. Once we test more of it under load and ensure that it's mature enough to roll out, you'll get those features for free. So, I'd like to propose that we use btrfs as the default file system for the 13.1 release before we release the first beta. Thanks for your time. -Jeff -- Jeff Mahoney SUSE Labs
On Tue, Sep 03, Jeff Mahoney wrote:
Supported Unsupported --------- ----------- Snapshots Inode cache
In the past the snapshots were enabled automatically. I once installed a VM with just a 20G disk and soon this small disk was filled up after some zypper patch/dup calls. It was not immediately obvious why the root filesystem ran out of space. Google pointed to snapper and somehow I managed to remove the (unwanted) snapshots. Unless the snapshot handling has improved to not suddenly fillup the disk during ordinary usage I suggest to disable them per default. Maybe add a big red button in the installer proposal to easily enable/disable them. Up to now it was easy to install a system/VM by just clicking Next/Next/+ and get system that continues to work without further interaction. With enabled snapshots this experience would break. Olaf -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On 9/3/13 11:04 AM, Olaf Hering wrote:
On Tue, Sep 03, Jeff Mahoney wrote:
Supported Unsupported --------- ----------- Snapshots Inode cache
In the past the snapshots were enabled automatically. I once installed a VM with just a 20G disk and soon this small disk was filled up after some zypper patch/dup calls. It was not immediately obvious why the root filesystem ran out of space. Google pointed to snapper and somehow I managed to remove the (unwanted) snapshots.
Unless the snapshot handling has improved to not suddenly fillup the disk during ordinary usage I suggest to disable them per default. Maybe add a big red button in the installer proposal to easily enable/disable them.
Yep, this is definitely on the list of things that need doing. I'm not sure what the current status is here, though. -Jeff -- Jeff Mahoney SUSE Labs
On Tuesday 03 of September 2013 17:04EN, Olaf Hering wrote:
In the past the snapshots were enabled automatically. I once installed a VM with just a 20G disk and soon this small disk was filled up after some zypper patch/dup calls. It was not immediately obvious why the root filesystem ran out of space. Google pointed to snapper and somehow I managed to remove the (unwanted) snapshots.
Unless the snapshot handling has improved to not suddenly fillup the disk during ordinary usage I suggest to disable them per default. Maybe add a big red button in the installer proposal to easily enable/disable them.
I think we should distinguish between two different things: 1. Enabling btrfs snapshots by default 2. Letting the system (by default) take snapshots periodically and/or upon certain actions (e.g. zypper d?up) My opinion would be yes for 1, no for 2. Michal Kubeček -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
El 03/09/13 10:32, Jeff Mahoney escribió:
In practice, all that's required for those users to continue uninterrupted is to add the 'allow_unsupported=1' option to the btrfs module either on the kernel command line or /etc/modprobe.d. There is nothing inherently limiting to any openSUSE user with this practice. The features are all still in the code and available immediately just by setting a flag. It can even be done safely after module load or even after file systems that don't use the unsupported features have been mounted. I intend to introduce this functionality into openSUSE soon.
That's not gonna fly, I will fight that to death, please do not add enterprise crippling module parameters to openSUSE, it *really* does not belong there at all. that's absolutely insane. -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On 9/3/13 11:13 AM, Cristian Rodríguez wrote:
El 03/09/13 10:32, Jeff Mahoney escribió:
In practice, all that's required for those users to continue uninterrupted is to add the 'allow_unsupported=1' option to the btrfs module either on the kernel command line or /etc/modprobe.d. There is nothing inherently limiting to any openSUSE user with this practice. The features are all still in the code and available immediately just by setting a flag. It can even be done safely after module load or even after file systems that don't use the unsupported features have been mounted. I intend to introduce this functionality into openSUSE soon.
That's not gonna fly, I will fight that to death, please do not add enterprise crippling module parameters to openSUSE, it *really* does not belong there at all. that's absolutely insane.
That's not really the reaction I got when I posted exactly this question a few weeks ago. The reaction was, in fact, pretty split with a bunch of people appreciating that we share our opinion on the maturity of particular features so that they don't have to do the research themselves. As I said then, the point isn't to take away functionality from anyone. If we have the ability to enable it in YaST (with a warning) or do it automatically (with a warning) if the features are already enabled on an older file system, nobody's lost anything. The point is that it will prevent users from losing their file systems using a feature set that hasn't been well tested. It's great that you want to try features out that we don't consider safe yet. That's how we get bug reports that we can use to improve and further gauge the quality of a particular feature set. I just don't think that /all/ openSUSE users want to have that particular experience without knowing what's up beforehand. -Jeff -- Jeff Mahoney SUSE Labs
El 03/09/13 11:53, Jeff Mahoney escribió:
As I said then, the point isn't to take away functionality from anyone. If we have the ability to enable it in YaST (with a warning) or do it automatically (with a warning) if the features are already enabled on an older file system, nobody's lost anything. The point is that it will prevent users from losing their file systems using a feature set that hasn't been well tested.
It's great that you want to try features out that we don't consider safe yet. That's how we get bug reports that we can use to improve and further gauge the quality of a particular feature set. I just don't think that /all/ openSUSE users want to have that particular experience without knowing what's up beforehand.
Jeff, I do not object to the idea of warning people that certain features are unstable or dangerous a log message: "btrfs warning: FOOBAR is unstable".. will suffice in achiving this result. what I do strongly object is this pervasive tendency to add SUSE specific flags that are not documented anywhere else other than probably a wiki page or the openSUSE documentation, has distribution specific semantics,is not in upstream plus will cause confusion and increase the load of volunteers that answer questions in mailing list and the forums. This makes sense for SLE where you only want to support things known to be production quality and therefore reducing the number of support incidents. in the openSUSE land will only cause confusion (pretty much like the "unsupported module flag") -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On 9/3/13 12:24 PM, Cristian Rodríguez wrote:
El 03/09/13 11:53, Jeff Mahoney escribió:
As I said then, the point isn't to take away functionality from anyone. If we have the ability to enable it in YaST (with a warning) or do it automatically (with a warning) if the features are already enabled on an older file system, nobody's lost anything. The point is that it will prevent users from losing their file systems using a feature set that hasn't been well tested.
It's great that you want to try features out that we don't consider safe yet. That's how we get bug reports that we can use to improve and further gauge the quality of a particular feature set. I just don't think that /all/ openSUSE users want to have that particular experience without knowing what's up beforehand.
Jeff, I do not object to the idea of warning people that certain features are unstable or dangerous a log message: "btrfs warning: FOOBAR is unstable".. will suffice in achiving this result. what I do strongly object is this pervasive tendency to add SUSE specific flags that are not documented anywhere else other than probably a wiki page or the openSUSE documentation, has distribution specific semantics,is not in upstream plus will cause confusion and increase the load of volunteers that answer questions in mailing list and the forums.
Your approach is favoring the experienced user over the inexperienced user, which can be reasonable especially if that's where we want to focus as a project. But it's dangerous when you also suggest a completely unintuitive method of informing the user. Nobody checks the log after successfully mounting a file system to see if there were any warnings. Documenting that in the release notes as a "best practice" isn't going to fly either. People do check the log when something unexpected happens and that's why we issue a message like the following when we reject a mount with immature features: "btrfs: compression is not supported, load module with allow_unsupported=1" (We can change the wording of that for openSUSE so that it reads "compression is considered an immature feature, load module with allow_unsupported=1".) I'm not sure many users will be comforted by being asked if they see a warning in their log after they've used an immature feature and discovered the hard way why it's considered immature. That's why I think an automated approach during upgrade and a yast checkbox to allow unsupported features is the way to go. Experienced users still get the full feature set to play with. Users with existing file systems with immature features enabled get to continue using them[1]. Inexperienced users get a reasonable picture of what is considered mature and what isn't and enjoy better data safety as a result. I'd prefer a better name than "allow_unsupported" but since this feature comes from SLES, it's better to use the existing name than to introduce another incompatibility.
This makes sense for SLE where you only want to support things known to be production quality and therefore reducing the number of support incidents. in the openSUSE land will only cause confusion (pretty much like the "unsupported module flag")
The unsupported module flag was removed from openSUSE for exactly that reason. This case is different, though. This isn't about managing support cases. Well, I suppose it is, but that's a nice side-effect of ensuring that users expecting that their file system uses only mature features have their expectations met. The naming of the module option is independent of the meaning of the option. That's largely a messaging thing and in the UI, it should be presented as "unstable" or "immature" feature enablement rather than trying to make a distinction over supportability. -Jeff [1] With the exception of file systems not in /etc/fstab. -- Jeff Mahoney SUSE Labs
El 03/09/13 13:26, Jeff Mahoney escribió:
Your approach is favoring the experienced user over the inexperienced user, which can be reasonable especially if that's where we want to focus as a project. But it's dangerous when you also suggest a completely unintuitive method of informing the user. Nobody checks the log after successfully mounting a file system to see if there were any warnings. Documenting that in the release notes as a "best practice" isn't going to fly either.
Ok, Jeff, since I am a reasonable person I will concede that your argument is reasonable and workable. I still do not like the idea of SUSE specific parameter but there are much bigger fish to catch around to continue this argument. Just some questions: - How and who will determine what it is stable/production-ready? - How will you deal with the parameter going away ? I ask because the kernel moves at the pace of a cheetah and YAST like a old turtle. suggesting that a configuration option has to be added to YAST turns a kernel implementation detail into an userspace interface. does parameter stays forever as as no-op when everything becomes "stable" ? -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On 9/3/13 9:47 PM, Cristian Rodríguez wrote:
El 03/09/13 13:26, Jeff Mahoney escribió:
Your approach is favoring the experienced user over the inexperienced user, which can be reasonable especially if that's where we want to focus as a project. But it's dangerous when you also suggest a completely unintuitive method of informing the user. Nobody checks the log after successfully mounting a file system to see if there were any warnings. Documenting that in the release notes as a "best practice" isn't going to fly either.
Ok, Jeff, since I am a reasonable person I will concede that your argument is reasonable and workable.
I still do not like the idea of SUSE specific parameter but there are much bigger fish to catch around to continue this argument.
As far as SUSE-specific patches go, it's minor. Realistically, we could push it to a last-line file in the Makefile if we had to. For now, it's integrated. The maintenance load is pretty minimal.
Just some questions:
- How and who will determine what it is stable/production-ready?
As I mentioned in another thread, the answer is essentially the "SUSE Labs Storage and File System Team" with a ton of input from our users. The trust has to start somewhere, and it seems that an obvious place to start would be the people maintaining the code for the release.
- How will you deal with the parameter going away ? I ask because the kernel moves at the pace of a cheetah and YAST like a old turtle. suggesting that a configuration option has to be added to YAST turns a kernel implementation detail into an userspace interface. does parameter stays forever as as no-op when everything becomes "stable" ?
Short answer: yes. We basically do need to maintain its existence forever. We pretty much need to do that anyway for compatibility with SLES. Ideally it will eventually become a no-op. Eventually it may be re-used to describe the maturity of other features. -Jeff -- Jeff Mahoney SUSE Labs
Hi Jeff,
Last month I posted queries to this list (and several other locations, including the forums) asking about people's experiences with btrfs. For the most part it seemed like the experience had improved over time. Most of the concerns were either with interactions with zypper or old perceptions of instability that were based more on old impressions than new testing. With the exception of an ENOSPC issue that had been recently fixed, users actively using the file system seemed pretty satisfied with it.
I wonder a bit how that is different to the current default, ext4. As a convinced user of ext4, what would be the personal reason for _me_ to switch or for the intended target group of openSUSE in general? What did the current default of openSUSE do wrong in order to reconsider the default choice? What are the alternatives to btrfs? How does it meet the majority of requirements better than the current default choice? It seems quite obvious to me that the default choice should be depending on what is suitable for the majority of use cases and provides the needed stability. My personal experience with btrfs and with responsitivity to bug reports was mediocre at best, but my last contact with it was ~ April this year, a lot of things might have improved since then.
So, I'd like to propose that we use btrfs as the default file system for the 13.1 release before we release the first beta.
It has been some time already available in Factory (and older openSUSE) releases. Do we know how big (the percentagewise) the happy btrfs userbase is? How do we avoid risking switching the default to something that only X% of the user base is happy with (with X being in the 5-20% area) ? Thanks, Dirk -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
Hello Dirk and all, On 2013-09-06 T 09:51 +0200 Dirk Müller wrote:
[...] what would be the personal reason for _me_ to switch or for the intended target group of openSUSE in general? [...] What are the alternatives to btrfs? How does it meet the majority of requirements better than the current default choice?
Besides Scalability there are other attributes where btrfs exceeds other filesystems. See various comparison tables out there, including the one in my blog from three years ago (yes, it's a bit dated): https://www.suse.com/communities/conversations/data-is-customers-gold/ Yet, there is this _one_ point, which made me switch to btrfs for "/" since Februar 2011: Peace of mind on adminstrative tasks (package updates and installations, configuration changes, ...) based on the snapshots / rollback capability. That's where I personally like btrfs for and where I see unique capabilities. And I changed my /home to btrfs last year for the same reason -- on my company and private systems. I even did some development in this area during last hack week: https://www.suse.com/communities/conversations/menu-du-jour-vivaneau-vert-su... But that might be off topic ...
It seems quite obvious to me that the default choice should be depending on what is suitable for the majority of use cases and provides the needed stability.
I can't complain about btrfs' stability.
It has been some time already available in Factory (and older openSUSE) releases. Do we know how big (the percentagewise) the happy btrfs userbase is? How do we avoid risking switching the default to something that only X% of the user base is happy with (with X being in the 5-20% area) ?
Well. Isn't this chicken-egg question the challenge for every new technology? Do we want to stop innovation based on that challenge? Happy - MgE -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
Matthias G. Eckermann wrote:
Hello Dirk and all,
On 2013-09-06 T 09:51 +0200 Dirk Müller wrote:
[...] what would be the personal reason for _me_ to switch or for the intended target group of openSUSE in general? [...] What are the alternatives to btrfs? How does it meet the majority of requirements better than the current default choice?
Besides Scalability there are other attributes where btrfs exceeds other filesystems.
I wouldn't think scalability is of much relevance to most openSUSE users. I don't remember seeing any threads about people coming up against filesystem limits. Except 2 TB in certain filesystems!
Yet, there is this _one_ point, which made me switch to btrfs for "/" since Februar 2011:
Peace of mind on adminstrative tasks (package updates and installations, configuration changes, ...) based on the snapshots / rollback capability.
That's where I personally like btrfs for and where I see unique capabilities.
But that feature is one that I think it has been agreed should be OFF by default, so again not a majority concern.
Well. Isn't this chicken-egg question the challenge for every new technology?
Do we want to stop innovation based on that challenge?
I'd suggest it might be a reasonable default for /home or /data and then after a year or two of no problems, make it the default for root. Cheers, Dave -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On 9/6/13 10:47 AM, Dave Howorth wrote:
Matthias G. Eckermann wrote:
Yet, there is this _one_ point, which made me switch to btrfs for "/" since Februar 2011:
Peace of mind on adminstrative tasks (package updates and installations, configuration changes, ...) based on the snapshots / rollback capability.
That's where I personally like btrfs for and where I see unique capabilities.
But that feature is one that I think it has been agreed should be OFF by default, so again not a majority concern.
I wouldn't say I've seen a consensus on that at all. The complaints haven't been about the snapshots. They've been about snapper being too aggressive about taking them and not aggressive enough about keeping out of the way as space becomes tighter. Those complaints have been heard by the snapper developers and they've invested effort in improving that situation. -Jeff -- Jeff Mahoney SUSE Labs
Jeff Mahoney wrote:
On 9/6/13 10:47 AM, Dave Howorth wrote:
Matthias G. Eckermann wrote:
Yet, there is this _one_ point, which made me switch to btrfs for "/" since Februar 2011:
Peace of mind on adminstrative tasks (package updates and installations, configuration changes, ...) based on the snapshots / rollback capability.
That's where I personally like btrfs for and where I see unique capabilities. But that feature is one that I think it has been agreed should be OFF by default, so again not a majority concern.
I wouldn't say I've seen a consensus on that at all. The complaints haven't been about the snapshots. They've been about snapper being too aggressive about taking them and not aggressive enough about keeping out of the way as space becomes tighter.
True. I misspoke, sorry. -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On 09/06/2013 09:51 AM, Jeff Mahoney wrote:
I wouldn't say I've seen a consensus on that at all. The complaints haven't been about the snapshots. They've been about snapper being too aggressive about taking them and not aggressive enough about keeping out of the way as space becomes tighter. Those complaints have been heard by the snapper developers and they've invested effort in improving that situation.
As I have followed this thread, it seems clear that BtrFS has matured a lot in the recent past, and that many of the problems are now addressed; however, I'm still not sure about making it the default. In the Forums, there have been many postings from inexperienced users that have chosen BtrFS because it was an option on the install page. I guess that Windows has trained them to select all options. As many of them are trying Linux and openSUSE for the first time, they often do not allocate much disk space for the activity, and will allocate 20 GB or less for everything, and they quickly run out of space, and snapshots make it even worse. When the advisers in the Forum cannot give them detailed instructions on how to recover, they quickly flee back to Windows. Does openSUSE have the support structure in place to handle BtrFS problems the way we do with ext4? Larry -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
Le Friday 06 September 2013 à 15:47 +0100, Dave Howorth a écrit :
I'd suggest it might be a reasonable default for /home or /data and then after a year or two of no problems, make it the default for root.
If anything, that should be exactly the other way around. Known-good ext4 for user data, and might-break btrfs for root. Or do you really value your system files more than your personal data? Plus I believe the snapshot/rollback feature is more useful for the system partition than for user data. -- Jean Delvare Suse L3 -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
Jean Delvare wrote:
Le Friday 06 September 2013 à 15:47 +0100, Dave Howorth a écrit :
I'd suggest it might be a reasonable default for /home or /data and then after a year or two of no problems, make it the default for root.
If anything, that should be exactly the other way around. Known-good ext4 for user data, and might-break btrfs for root. Or do you really value your system files more than your personal data?
No, I think that the average openSUSE user probably does have a backup for their data (since that's easy to do) but may well not have an easily usable system backup (since that's harder to do) and may have lots of barely remembered config and installs in the system, and may also choose to install an upgraded version if there is a hardware problem. So I think losing the root system is likely to be more painful than losing a data filesystem. JMHO. -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On Tuesday 10 of September 2013 11:42EN, Dave Howorth wrote:
No, I think that the average openSUSE user probably does have a backup for their data
I would be extremely surprised if this was true. Michal Kubeček -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On 9/10/13 6:42 AM, Dave Howorth wrote:
Jean Delvare wrote:
Le Friday 06 September 2013 à 15:47 +0100, Dave Howorth a écrit :
I'd suggest it might be a reasonable default for /home or /data and then after a year or two of no problems, make it the default for root.
If anything, that should be exactly the other way around. Known-good ext4 for user data, and might-break btrfs for root. Or do you really value your system files more than your personal data?
No, I think that the average openSUSE user probably does have a backup for their data (since that's easy to do) but may well not have an easily usable system backup (since that's harder to do) and may have lots of barely remembered config and installs in the system, and may also choose to install an upgraded version if there is a hardware problem.
So I think losing the root system is likely to be more painful than losing a data filesystem. JMHO.
FWIW, and maybe the typical openSUSE user experience is different in this case, but we've found the opposite to be true in the SLES space. -Jeff -- Jeff Mahoney SUSE Labs
On Friday 2013-09-06 16:17, Matthias G. Eckermann wrote:
On 2013-09-06 T 09:51 +0200 Dirk Müller wrote:
[...] what would be the personal reason for _me_ to switch or for the intended target group of openSUSE in general? [...] What are the alternatives to btrfs? How does it meet the majority of requirements better than the current default choice?
Besides Scalability there are other attributes where btrfs exceeds other filesystems. See various comparison tables out there, including the one in my blog from three years ago (yes, it's a bit dated): https://www.suse.com/communities/conversations/data-is-customers-gold/
Since somewhat-dated data seems to be popular, here is another: http://www.youtube.com/watch?v=FegjLbCnoBw in this talk, David Chinner showed at LCA 2012 that btrfs was rather non-scaling (for the features it shares with existing filesystems, IOW, just vanilla storing). -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
Hello Jan and all, On 2013-09-06 T 18:35 +0200 Jan Engelhardt wrote:
On Friday 2013-09-06 Matthias G. Eckermann wrote:
On 2013-09-06 T 09:51 +0200 Dirk Müller wrote:
[...] what would be the personal reason for _me_ to switch or for the intended target group of openSUSE in general? [...] What are the alternatives to btrfs? How does it meet the majority of requirements better than the current default choice?
Besides Scalability there are other attributes where btrfs exceeds other filesystems. See various comparison tables out there, including the one in my blog from three years ago (yes, it's a bit dated): https://www.suse.com/communities/conversations/data-is-customers-gold/
Since somewhat-dated data seems to be popular, here is another:
http://www.youtube.com/watch?v=FegjLbCnoBw
in this talk, David Chinner showed at LCA 2012 that btrfs was rather non-scaling (for the features it shares with existing filesystems, IOW, just vanilla storing).
I am afraid, we have a wording issue here: When Dave says "Scalability" in that presentations, he means "Performance" (see his slides). When I say "Scalability" above, and use that word comparing btrfs to the current openSUSE default, I am not talking Performance, but talking about "Scalability" in the sense of filesystem size, dealing with huge amounts of (small) files, ... Hope this explains the different view. so long - MgE -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On Fri, Sep 06, 2013 at 07:04:35PM +0200, Matthias G. Eckermann wrote:
I am afraid, we have a wording issue here:
When Dave says "Scalability" in that presentations, he means "Performance" (see his slides).
When I say "Scalability" above, and use that word comparing btrfs to the current openSUSE default, I am not talking Performance, but talking about "Scalability" in the sense of filesystem size, dealing with huge amounts of (small) files, ...
Hope this explains the different view.
Not really. "Scalability" in the sense of huge amounts of small files sense means exactly "Performance" for me, as that's where XFS before was dog slow, i.e. it didn't *scale". M. -- Michael Schroeder mls@suse.de SUSE LINUX Products GmbH, GF Jeff Hawn, HRB 16746 AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
Hi Matthias,
Besides Scalability there are other attributes where btrfs exceeds other filesystems.
Regarding the scalability part, lets not compare something from 3 years ago, lets compare the 13.1 kernel, kernel 3.11.0. Ext4 has had pretty nice improvements in 3.11 regarding scalability, see http://lkml.indiana.edu/hypermail/linux/kernel/1307.0/00286.html for details. http://www.phoronix.com/scan.php?page=article&item=linux_311_filesystems You might or not like this benchmark, but the headline is pretty clear: "EXT4 wins". Also, did btrfs fix the backlink issue? that seems to be a major scalability burden actually. And just to compare the _scalability_ we're talking about. the corner cases are: BTRFS supports filesytems up to 16384 Petabytes. Ext4 has a slight disadvantage here, only spporting filesystems up to 1024 Petabytes. While that sounds like a serios scalability issue for SLE, it is less of a concern for the typical openSUSE case. Other scalability marks are not that interesting. But if we care about scalability and SSD support, F2FS might be interesting to look at as well.. Greetings, Dirk -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On 9/11/13 6:16 AM, Dirk Müller wrote:
Hi Matthias,
Besides Scalability there are other attributes where btrfs exceeds other filesystems.
Regarding the scalability part, lets not compare something from 3 years ago, lets compare the 13.1 kernel, kernel 3.11.0. Ext4 has had pretty nice improvements in 3.11 regarding scalability, see http://lkml.indiana.edu/hypermail/linux/kernel/1307.0/00286.html for details.
http://www.phoronix.com/scan.php?page=article&item=linux_311_filesystems
You might or not like this benchmark, but the headline is pretty clear: "EXT4 wins".
Also, did btrfs fix the backlink issue? that seems to be a major scalability burden actually.
Again, yes:
commit f186373fef005cee948a4a39e6a14c2e5f517298
Author: Mark Fasheh
And just to compare the _scalability_ we're talking about. the corner cases are:
BTRFS supports filesytems up to 16384 Petabytes. Ext4 has a slight disadvantage here, only spporting filesystems up to 1024 Petabytes. While that sounds like a serios scalability issue for SLE, it is less of a concern for the typical openSUSE case.
Other scalability marks are not that interesting. But if we care about scalability and SSD support, F2FS might be interesting to look at as well..
Greetings, Dirk
-- Jeff Mahoney SUSE Labs
On Wed, Sep 11, 2013 at 12:16:57PM +0200, Dirk Müller wrote:
Also, did btrfs fix the backlink issue? that seems to be a major scalability burden actually.
The feature is turned on by default for 13.1 -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On Wed, Sep 11, 2013 at 7:16 AM, Dirk Müller
Besides Scalability there are other attributes where btrfs exceeds other filesystems.
Regarding the scalability part, lets not compare something from 3 years ago, lets compare the 13.1 kernel, kernel 3.11.0. Ext4 has had pretty nice improvements in 3.11 regarding scalability, see http://lkml.indiana.edu/hypermail/linux/kernel/1307.0/00286.html for details.
http://www.phoronix.com/scan.php?page=article&item=linux_311_filesystems
You might or not like this benchmark, but the headline is pretty clear: "EXT4 wins".
And, they didn't check, but at least for database workloads ext3 beats ext4. So, if btrfs is slower than ext4 by that much, then for databases it's a no-no (ext4 is already a no-no, so perhaps I should say no-no-no-no-no) -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On 9/11/13 1:52 PM, Claudio Freire wrote:
On Wed, Sep 11, 2013 at 7:16 AM, Dirk Müller
wrote: Besides Scalability there are other attributes where btrfs exceeds other filesystems.
Regarding the scalability part, lets not compare something from 3 years ago, lets compare the 13.1 kernel, kernel 3.11.0. Ext4 has had pretty nice improvements in 3.11 regarding scalability, see http://lkml.indiana.edu/hypermail/linux/kernel/1307.0/00286.html for details.
http://www.phoronix.com/scan.php?page=article&item=linux_311_filesystems
You might or not like this benchmark, but the headline is pretty clear: "EXT4 wins".
And, they didn't check, but at least for database workloads ext3 beats ext4.
So, if btrfs is slower than ext4 by that much, then for databases it's a no-no (ext4 is already a no-no, so perhaps I should say no-no-no-no-no)
No, it's not currently intended for database workloads. But even then there's some tuning to be done. Big databases run their own checksums and don't care about the data checksumming inside the file system. They also probably want nodatacow enabled on the database files because they really just want space on disk and for the OS to get out of the way otherwise. -Jeff -- Jeff Mahoney SUSE Labs
В Wed, 11 Sep 2013 14:27:09 -0400
Jeff Mahoney
On 9/11/13 1:52 PM, Claudio Freire wrote:
On Wed, Sep 11, 2013 at 7:16 AM, Dirk Müller
wrote: Besides Scalability there are other attributes where btrfs exceeds other filesystems.
Regarding the scalability part, lets not compare something from 3 years ago, lets compare the 13.1 kernel, kernel 3.11.0. Ext4 has had pretty nice improvements in 3.11 regarding scalability, see http://lkml.indiana.edu/hypermail/linux/kernel/1307.0/00286.html for details.
http://www.phoronix.com/scan.php?page=article&item=linux_311_filesystems
You might or not like this benchmark, but the headline is pretty clear: "EXT4 wins".
And, they didn't check, but at least for database workloads ext3 beats ext4.
So, if btrfs is slower than ext4 by that much, then for databases it's a no-no (ext4 is already a no-no, so perhaps I should say no-no-no-no-no)
No, it's not currently intended for database workloads. But even then there's some tuning to be done. Big databases run their own checksums and don't care about the data checksumming inside the file system. They also probably want nodatacow enabled on the database files because they really just want space on disk and for the OS to get out of the way otherwise.
Oh, tell this NetApp users, they will really be surprised :) The ability to create space efficient snapshots in no time opens up the whole new world of possibilities for database maintenance.
On Wed, Sep 11, 2013 at 12:52 PM, Claudio Freire
On Wed, Sep 11, 2013 at 7:16 AM, Dirk Müller
wrote: Besides Scalability there are other attributes where btrfs exceeds other filesystems.
Regarding the scalability part, lets not compare something from 3 years ago, lets compare the 13.1 kernel, kernel 3.11.0. Ext4 has had pretty nice improvements in 3.11 regarding scalability, see http://lkml.indiana.edu/hypermail/linux/kernel/1307.0/00286.html for details.
http://www.phoronix.com/scan.php?page=article&item=linux_311_filesystems
You might or not like this benchmark, but the headline is pretty clear: "EXT4 wins".
And, they didn't check, but at least for database workloads ext3 beats ext4.
Why do you say that? I'm of the impression that ext4 is - highly - recommended over ext3. -- Jon -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On Wed, Sep 11, 2013 at 7:16 AM, Dirk Müller
wrote: Besides Scalability there are other attributes where btrfs exceeds other filesystems.
Regarding the scalability part, lets not compare something from 3 years ago, lets compare the 13.1 kernel, kernel 3.11.0. Ext4 has had pretty nice improvements in 3.11 regarding scalability, see http://lkml.indiana.edu/hypermail/linux/kernel/1307.0/00286.html for details.
http://www.phoronix.com/scan.php?page=article&item=linux_311_filesystems
You might or not like this benchmark, but the headline is pretty clear: "EXT4 wins".
And, they didn't check, but at least for database workloads ext3 beats ext4. Do you have any data to share? Preferably also with
[trimmed CC list a bit]
On Wed 11-09-13 14:52:45, Claudio Freire wrote:
linux-ext4@vger.kernel.org. I'd be interested in tracking that down because
I (and I believe other ext4 developers as well) am not aware of this.
Honza
--
Jan Kara
participants (16)
-
Andrey Borzenkov
-
Claudio Freire
-
Cristian Rodríguez
-
Dave Howorth
-
David Sterba
-
Dirk Müller
-
Jan Engelhardt
-
Jan Kara
-
Jean Delvare
-
Jeff Mahoney
-
Jon Nelson
-
Larry Finger
-
Matthias G. Eckermann
-
Michael Schroeder
-
Michal Kubeček
-
Olaf Hering