[opensuse-kernel] btrfs performance problem with almost full partition
Hi Jeff, The day after the openSUSE conference, Jos showed me his laptop and a problem with it under the most recent Tumbleweed kernel (and older kernels as well.) He is copying a very large (3Gb) file to his btrfs partition from a USB external drive (running ext3) such that if it succeeds, the partition will almost be full (only a few Mb left). This should be fine, but it almost kills the system, slowing responsiveness down to nothing being possible to do at all on the box until the copy completes 15+ minutes later. It is possible to eventually interrupt the copy, and when that happens, the system returns back to life. Copying the other way (from the btrfs partition to the USB disk), works absolutly fine, so it doesn't seem like the system is hurting from memory pressure on anything else. Is this a currently-known issue on btrfs when the filesystem is almost out of space and it just takes forever to find empty blocks in which to put things, or is it something that we should log in a bugzilla entry and try to get resolved? thanks, greg k-h -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
On Mon, Sep 19, 2011 at 1:56 PM, Greg KH <gregkh@suse.de> wrote:
Hi Jeff,
The day after the openSUSE conference, Jos showed me his laptop and a problem with it under the most recent Tumbleweed kernel (and older kernels as well.)
Greg, I suggest it go to the kernel bugzilla, not the openSUSE one. Btrfs is just not yet stable enough for openSUSE to consider it anything but experimental. === fyi Partially in hopes of convincing myself btrfs would be ready for 12.1, I put together a wiki page explaining how to install and run the filesystem test suite the xfs team maintains. I pointed the testing team at it, so they have run at least some automated btrfs tests. http://en.opensuse.org/SDB:XFStests Over time a lot of those tests have been expanded from xfs only to multiple filesystems or generic to linux. I found btrfs failed 3 of the existing functional tests. Failures: 075 112 254 (with the tumbleweed 3.0.0-rc7-4-desktop kernel. Similar failures with older kernels.) I asked on the list that maintains the xfstests script to see if was a test bug, or a btrfs bugs. They said the tests should be fine, so they represent real btrfs bugs. fyi: I don't know what those specific tests are, so maybe they should not be considered btrfs "ship stoppers". I just assumed that until all the pre-existing tests pass, btrfs will be considered experimental. Greg -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
On Mon, Sep 19, 2011 at 4:23 PM, Greg Freemyer <greg.freemyer@gmail.com> wrote:
On Mon, Sep 19, 2011 at 1:56 PM, Greg KH <gregkh@suse.de> wrote: <snip>
Failures: 075 112 254 (with the tumbleweed 3.0.0-rc7-4-desktop kernel. Similar failures with older kernels.)
I just re-ran the full xfstests auto group (check -g auto) and it's actually 4 failures. Failures: 075 112 127 254 I haven't put any of those failures in the novell bugzilla because I have just assumed they were vanilla kernel issues and known to the btrfs team since xfstests is a well-known test suite. Greg -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
Hi, On Mon, Sep 19, 2011 at 04:23:31PM -0400, Greg Freemyer wrote:
I suggest it go to the kernel bugzilla, not the openSUSE one.
Rather post it to the mailinglist or ask at IRC. kenrel bugzilla is down and nearly nobody looks there (although there are worth reports).
Btrfs is just not yet stable enough for openSUSE to consider it anything but experimental.
Correct. Have backups.
=== fyi Partially in hopes of convincing myself btrfs would be ready for 12.1, I put together a wiki page explaining how to install and run the filesystem test suite the xfs team maintains. I pointed the testing team at it, so they have run at least some automated btrfs tests.
I have packaged xfstests, https://build.opensuse.org/package/show?package=xfstests&project=home:dsterba:tools as I use them heavily and got tired of doing manual installation, namely for automatic testing. (The package needs xfsprogs from the same repo.)
Over time a lot of those tests have been expanded from xfs only to multiple filesystems or generic to linux.
I found btrfs failed 3 of the existing functional tests.
Failures: 075 112 254 (with the tumbleweed 3.0.0-rc7-4-desktop kernel. Similar failures with older kernels.)
075 and 112 do not fail for me at 3.1.0-rc6+, although 254 reports failure, but it's a minor one (just basic snapshot/subovl tests, this area deserves much harder testing).
I asked on the list that maintains the xfstests script to see if was a test bug, or a btrfs bugs. They said the tests should be fine, so they represent real btrfs bugs.
fyi: I don't know what those specific tests are, so maybe they should not be considered btrfs "ship stoppers". I just assumed that until all the pre-existing tests pass, btrfs will be considered experimental.
The failing tests are not ship-stoppers. There are 100+ commits between 3.0 and 3.1-rc, many of them stability and correctness fixes. Anybody using btrfs should try to stick to the latest kernel. And keep backups. The stable series received only small number of fixes. david -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
On Tue, Sep 20, 2011 at 8:13 AM, David Sterba <dsterba@suse.cz> wrote:
Hi,
On Mon, Sep 19, 2011 at 04:23:31PM -0400, Greg Freemyer wrote:
I suggest it go to the kernel bugzilla, not the openSUSE one.
Rather post it to the mailinglist or ask at IRC. kenrel bugzilla is down and nearly nobody looks there (although there are worth reports).
Btrfs is just not yet stable enough for openSUSE to consider it anything but experimental.
Correct. Have backups.
=== fyi Partially in hopes of convincing myself btrfs would be ready for 12.1, I put together a wiki page explaining how to install and run the filesystem test suite the xfs team maintains. I pointed the testing team at it, so they have run at least some automated btrfs tests.
I have packaged xfstests, https://build.opensuse.org/package/show?package=xfstests&project=home:dsterba:tools
Fairly new package I see. I'd like to reference that on the wiki page I wrote. Are you going to submit it to a devel repo, or keep it in your home? If in your home, is it going to hang around long enough to be worth pointing the testing team at? Or are they already using it? A couple months ago, they simply were running a script that downloaded and compiled what they needed. Greg -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
On Tue, Sep 20, 2011 at 10:52:16AM -0400, Greg Freemyer wrote:
Fairly new package I see. I'd like to reference that on the wiki page I wrote. Are you going to submit it to a devel repo, or keep it in your home?
I stopped enhancing the package as soon it worked for me, the specfile is not perfect and I did not attempt to push it to devel project (lack of time). I will keep the project in my home and update from git (or mailinglist patches), but no problem if somebody pushes it to devel.
If in your home, is it going to hang around long enough to be worth pointing the testing team at?
They can use if course, but I give no guarantees on upates or correctness. I will most probably not delete it anytime soon.
Or are they already using it?
I don't know, I've only announced it to filesystem developers in SUSE. david -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
As a general "rule of thumb", full partitions are slow partitions irrespective of filesystem structures. There are probably filesystems that work well when full for some specific types of I/O, but *random* I/O is going to suck on a rotating seeking set of platters if there are very few places to write new stuff and previously written stuff is scattered over many tracks and sectors. blktrace is your friend if you want to help debug btrfs on this use case. ;-) On Mon, Sep 19, 2011 at 10:56 AM, Greg KH <gregkh@suse.de> wrote:
Hi Jeff,
The day after the openSUSE conference, Jos showed me his laptop and a problem with it under the most recent Tumbleweed kernel (and older kernels as well.)
He is copying a very large (3Gb) file to his btrfs partition from a USB external drive (running ext3) such that if it succeeds, the partition will almost be full (only a few Mb left). This should be fine, but it almost kills the system, slowing responsiveness down to nothing being possible to do at all on the box until the copy completes 15+ minutes later. It is possible to eventually interrupt the copy, and when that happens, the system returns back to life.
Copying the other way (from the btrfs partition to the USB disk), works absolutly fine, so it doesn't seem like the system is hurting from memory pressure on anything else.
Is this a currently-known issue on btrfs when the filesystem is almost out of space and it just takes forever to find empty blocks in which to put things, or is it something that we should log in a bugzilla entry and try to get resolved?
thanks,
greg k-h -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
-- http://twitter.com/znmeb http://borasky-research.net "A mathematician is a device for turning coffee into theorems." -- Paul Erdős -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-kernel+help@opensuse.org
participants (4)
-
David Sterba -
Greg Freemyer -
Greg KH -
M. Edward (Ed) Borasky