[Bug 886362] New: btrfs "crap" filesystem full signaling
https://bugzilla.novell.com/show_bug.cgi?id=886362 https://bugzilla.novell.com/show_bug.cgi?id=886362#c0 Summary: btrfs "crap" filesystem full signaling Classification: openSUSE Product: openSUSE 13.1 Version: Final Platform: Other OS/Version: Other Status: NEW Severity: Enhancement Priority: P5 - None Component: Basesystem AssignedTo: bnc-team-screening@forge.provo.novell.com ReportedBy: diego.ercolani@gmail.com QAContact: qa-bugs@suse.de Found By: --- Blocker: --- User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0 I know that is this by design but is very annoying that btrfs (expecially) when used as compressed filesystem doesn't manage the df command well because of metadata, deduplication, compression, balancing and so on. But we need that btrfs manage something useful with df, df is easy to process and with a single simple command to have the snapshot of what is happending. It's very annoying to manage the problem of having the filesystem full just when it happens. Reproducible: Always Steps to Reproduce: 1. 2. 3. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=886362 https://bugzilla.novell.com/show_bug.cgi?id=886362#c1 Bernhard Wiedemann <bwiedemann@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bwiedemann@suse.com, | |mge@suse.com --- Comment #1 from Bernhard Wiedemann <bwiedemann@suse.com> 2014-07-24 15:41:04 CEST --- btw there is btrfs filesystem df / which is a bit more accurate, but in the end, I think it will also report free space even when you get ENOSPC (No space left on device) on write There are many things programs want to do to files and with btrfs the only safe way (I know of) is to create a whole new copy with the new content - and that does not work well with many RL workloads, among them DBs and VMs. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=886362 https://bugzilla.novell.com/show_bug.cgi?id=886362#c2 Matthias Eckermann <mge@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jeffm@suse.com --- Comment #2 from Matthias Eckermann <mge@suse.com> 2014-07-24 14:10:38 UTC --- The btrfs community is addressing the free space handling and free space tracking in multiple ways currently. One of these ways is to improve the implementation of quota groups (qgroups), thus the "real size" of snapshots can be queried from the filesystem. The problem: copy on write works on a block basis, thus one block can be shared by multiple copies. So the question is: what is the "real" amount of bytes a specific file or file version consumes? This question is even harder to answer, if compression is used (compression is done block wise, if I am not mistaken). That's the primary reason, why I do not use and would not use btrfs' compression yet. With openSUSE 13.2 we might be in a better shape in the future... I suggest to close this as an enhancement request..... -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
From this approach that is what is useful (I think) to get we should start and
https://bugzilla.novell.com/show_bug.cgi?id=886362 https://bugzilla.novell.com/show_bug.cgi?id=886362#c3 --- Comment #3 from Diego Ercolani <diego.ercolani@gmail.com> 2014-07-24 14:31:42 UTC --- I think the approach should be different. Compression and deduplication are functionality that enhance artificially the available space but we face with a real world where you have limited spaces. If I have a partition with 1 GB free I could generate more "virtual space" with deduplication and all... but I have only 1 GB (worst case without metadata and indirection). If I use the df to ask about available space I need to be answered with 1GB. If BTRFS need to periodically "optimize" his data structure, It shoud do but this will (hopefully) free real space that will be (hopefully) more virtual space. put in real world target. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=886362 https://bugzilla.novell.com/show_bug.cgi?id=886362#c4 --- Comment #4 from Bernhard Wiedemann <bwiedemann@suse.com> 2014-07-24 16:42:56 CEST --- even without deduplication and compression, df should give a pessimistic value on free space, so that programs that check this value do not run into disk-full errors. And that will mean, that you need a lot more free space before you do zypper patch because via snapper the old versions of replaced files will remain in a snapshot. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=886362 https://bugzilla.novell.com/show_bug.cgi?id=886362#c5 Jeff Mahoney <jeffm@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- AssignedTo|bnc-team-screening@forge.pr |mfasheh@suse.com |ovo.novell.com | --- Comment #5 from Jeff Mahoney <jeffm@suse.com> 2014-07-25 09:02:06 EDT --- We can stop treating deduplication as a some sort of special case. Btrfs (and ocfs2) have the ability for multiple files to share references to the same extents on disk. This allows snapshots to exist efficiently, but it also requires that the file system overcommit storage to handle those. Deduplication uses the same mechanisms that snapshots use, but does it retroactively. For the most part, it works fine since snapshots usually don't change so the only divergent branch is the active "main" part of the file system. But, to address the specific suggestion in comment #3, to report space in that manner would make space reporting essentially useless. The entire point of using efficient snapshots is that you can overcommit your file system. Having 10 snapshots of your root file system *shouldn't* take 10x the space of your root file system. UNIX systems have taken the approach that overcommitting is "ok" for a while. Think of the memory management subsystem and shared libraries. It's possible to configure your system /not/ to overcommit, but you'll find yourself either needing massive amounts of swap or running out of memory on a regular basis. The swap may never get used, but you'll still need it there. For a concrete example, take a look at the output of "ps" and add up the VSZ and RSS columns. You'll find that the VSZ column dwarfs the RSS column. This is because we 1) only page in executables as we need them and, more importantly, 2) we have shared libraries so that even though it's technically possible for each process to have its own copy of a library, we get massive memory savings by sharing a single copy in memory. Btrfs space reporting reports the equivalent of RSS -- the space *actually* used on disk. You're asking for the VSZ column, which would give you a worst-case number but not actually give you anything you could base space allocation decisions on. All of that said, there is definitely room for improvement in btrfs space reporting. I just don't agree with your solution. Mark Fasheh has been working on improving the qgroups implementation to make it much more reliable. With that, you'll be able to tell at a glance how much space a subvolume is using exclusively vs shared. The next step is to fix hierarchical qgroups, which will allow the system admin to see how much space is exclusive for a set of subvolumes. That will make it more clear (to an extent) what will happen if you change the active file system in a manner like 'zypper dup.' Ultimately, it's a matter of what you're replacing, not just the raw numbers. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply@novell.com