All,
I didn't see any openSUSE specific instructions for running xfstests, so I put together a simple wiki page:
http://en.opensuse.org/SDB:XFStests
Is anyone running xfstests on factory for btrfs?
=== I just tried it for 11.4 with Kernel:Head (3.0-rc6 I think).
I got 3 failed tests. But it almost looks like that's normal. ie. It looks like issues with the test infrastructure, not xfstests itself.
My GSoC student is working on xfstests to add some snapshot specific tests, so it could be a local issue on our end.
Greg
Am Mittwoch 13 Juli 2011, 02:23:11 schrieben Sie:
All,
I didn't see any openSUSE specific instructions for running xfstests, so I put together a simple wiki page:
http://en.opensuse.org/SDB:XFStests
Is anyone running xfstests on factory for btrfs?
=== I just tried it for 11.4 with Kernel:Head (3.0-rc6 I think).
I got 3 failed tests. But it almost looks like that's normal. ie. It looks like issues with the test infrastructure, not xfstests itself.
My GSoC student is working on xfstests to add some snapshot specific tests, so it could be a local issue on our end.
Greg
Hi Greg, thanks for the info, I will try to include this in one of my next test runs. Any results I will post here. Any additonal
Regards Jürgen
2011/7/13 Jürgen Radzuweit juergen@radzuweit.eu:
Am Mittwoch 13 Juli 2011, 02:23:11 schrieben Sie:
All,
I didn't see any openSUSE specific instructions for running xfstests, so I put together a simple wiki page:
http://en.opensuse.org/SDB:XFStests
Is anyone running xfstests on factory for btrfs?
=== I just tried it for 11.4 with Kernel:Head (3.0-rc6 I think).
I got 3 failed tests. But it almost looks like that's normal. ie. It looks like issues with the test infrastructure, not xfstests itself.
My GSoC student is working on xfstests to add some snapshot specific tests, so it could be a local issue on our end.
Greg
Hi Greg, thanks for the info, I will try to include this in one of my next test runs. Any results I will post here. Any additonal
Regards Jürgen
Jürgen,
Just so you know, I fully expect btrfs (as well as the other filesystems) will pass the 254 official xfstest checks assuming the test themselves are not broken. (As 3 of them seem to be to me.)
They've been around a long time and I'm sure the btrfs team test with them.
But xfstest does not have any snapshot related testing per my knowledge.
I'm working with a GSoC student to change that. We have one test basically working now, and the 2.6.39 kernel is passing it with no issues if we run it standalone.
But if we run the official 254 checks first, then our new test last it fails!
We've yet to determine if that is a bug in our new test, or in btrfs.
Greg
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Am 13.07.2011 02:23, schrieb Greg Freemyer:
All,
I didn't see any openSUSE specific instructions for running xfstests, so I put together a simple wiki page:
http://en.opensuse.org/SDB:XFStests
Is anyone running xfstests on factory for btrfs?
=== I just tried it for 11.4 with Kernel:Head (3.0-rc6 I think).
I got 3 failed tests. But it almost looks like that's normal. ie. It looks like issues with the test infrastructure, not xfstests itself.
My GSoC student is working on xfstests to add some snapshot specific tests, so it could be a local issue on our end.
Greg
I still get failures with btrfs as / on Factory [1] - but once that changes, I would like to add the xfstests in openQA testruns. An alternative would be to create a separate filesystem with btrfs specifically for the test.
Ciao Bernhard M.
[1] http://openqa.opensuse.org/results/openSUSE-DVD-i586-Build0120-btrfs
On Thu, Jul 14, 2011 at 2:34 AM, Bernhard M. Wiedemann bernhardout@lsmod.de wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Am 13.07.2011 02:23, schrieb Greg Freemyer:
All,
I didn't see any openSUSE specific instructions for running xfstests, so I put together a simple wiki page:
http://en.opensuse.org/SDB:XFStests
Is anyone running xfstests on factory for btrfs?
=== I just tried it for 11.4 with Kernel:Head (3.0-rc6 I think).
I got 3 failed tests. But it almost looks like that's normal. ie. It looks like issues with the test infrastructure, not xfstests itself.
My GSoC student is working on xfstests to add some snapshot specific tests, so it could be a local issue on our end.
Greg
I still get failures with btrfs as / on Factory [1] - but once that changes, I would like to add the xfstests in openQA testruns. An alternative would be to create a separate filesystem with btrfs specifically for the test.
Ciao Bernhard M.
Bernhard,
For clarity xfstests is not in OBS / factory. It has to pulled from git and compiled as documented in the above wiki page.
Does that work for your automated testing? If not, it seems easy enough to package and put into a devel repo. I don't know if it makes sense in factory?
Let me know if I can do anything.
Greg
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Am 14.07.2011 17:07, schrieb Greg Freemyer:
Bernhard,
For clarity xfstests is not in OBS / factory. It has to pulled from git and compiled as documented in the above wiki page.
Does that work for your automated testing? If not, it seems easy enough to package and put into a devel repo. I don't know if it makes sense in factory?
Let me know if I can do anything.
Greg
I made this non-interactive script from the wiki page (fixing the BuildRequires in the process): http://openqa.opensuse.org/opensuse/qatests/qa_btrfs.sh
and got (after running quite a while) as result Failed 4 of 50 tests which might match your results...
We can still package xfstests on OBS and push it to devel:openSUSE:Factory or such where os-autoinst also resides. Could make things somewhat simpler.
Ciao Bernhard M.
On Thu, Jul 14, 2011 at 4:09 PM, Bernhard M. Wiedemann bernhardout@lsmod.de wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Am 14.07.2011 17:07, schrieb Greg Freemyer:
Bernhard,
For clarity xfstests is not in OBS / factory. It has to pulled from git and compiled as documented in the above wiki page.
Does that work for your automated testing? If not, it seems easy enough to package and put into a devel repo. I don't know if it makes sense in factory?
Let me know if I can do anything.
Greg
I made this non-interactive script from the wiki page (fixing the BuildRequires in the process): http://openqa.opensuse.org/opensuse/qatests/qa_btrfs.sh
great
and got (after running quite a while) as result Failed 4 of 50 tests which might match your results...
I reached out to the xfs list to see if test 075 as an example had a bug, or if it was btrfs itself that was faulty. I was told for that one in particular the test was fine, so btrfs is the culprit. They implied the other tests should be stable as well.
As to 4 failures, I get different results depending on the history of tests I've run since the last mkfs. So, I think it is safe to say at least one of the tests causes filesystem corruption that causes other tests to fail.
I think your automated test is fine as is, but anyone delving into a particular failure should mkfs a nice clean filesystem, then run the test standalone.
As to speed, the "-g auto" arg to ./check is controlling which tests to run. You can run -f quick and it should finish in under 5 minutes. For me the failing tests are also part of the quick group.
See the file "group" in the source to see which tests are in which group.
Also, if the quick set of tests is fast enough, you could use xfstests to test xfs and ext4 as well. I don't know what other filesystems xfstests supports.
We can still package xfstests on OBS and push it to devel:openSUSE:Factory or such where os-autoinst also resides. Could make things somewhat simpler.
I will try to package it up and submit it. Looks pretty simple.
Ciao Bernhard M.
Greg
On Fri, Jul 15, 2011 at 10:08 AM, Greg Freemyer greg.freemyer@gmail.com wrote:
On Thu, Jul 14, 2011 at 4:09 PM, Bernhard M. Wiedemann bernhardout@lsmod.de wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Am 14.07.2011 17:07, schrieb Greg Freemyer:
Bernhard,
For clarity xfstests is not in OBS / factory. It has to pulled from git and compiled as documented in the above wiki page.
Does that work for your automated testing? If not, it seems easy enough to package and put into a devel repo. I don't know if it makes sense in factory?
Let me know if I can do anything.
Greg
I made this non-interactive script from the wiki page (fixing the BuildRequires in the process): http://openqa.opensuse.org/opensuse/qatests/qa_btrfs.sh
great
and got (after running quite a while) as result Failed 4 of 50 tests which might match your results...
I reached out to the xfs list to see if test 075 as an example had a bug, or if it was btrfs itself that was faulty. I was told for that one in particular the test was fine, so btrfs is the culprit. They implied the other tests should be stable as well.
As to 4 failures, I get different results depending on the history of tests I've run since the last mkfs. So, I think it is safe to say at least one of the tests causes filesystem corruption that causes other tests to fail.
I think your automated test is fine as is, but anyone delving into a particular failure should mkfs a nice clean filesystem, then run the test standalone.
As to speed, the "-g auto" arg to ./check is controlling which tests to run. You can run -f quick and it should finish in under 5 minutes. For me the failing tests are also part of the quick group.
See the file "group" in the source to see which tests are in which group.
Also, if the quick set of tests is fast enough, you could use xfstests to test xfs and ext4 as well. I don't know what other filesystems xfstests supports.
We can still package xfstests on OBS and push it to devel:openSUSE:Factory or such where os-autoinst also resides. Could make things somewhat simpler.
I will try to package it up and submit it. Looks pretty simple.
Ciao Bernhard M.
Greg
All,
Several of the discrete tests from xfstests have pending patches for btrfs. They were just posted in the last week.
I believe the patches will increase the number of tests that attempt to run against btrfs and thus cause more failures, as opposed to fixing tests that are showing false failures on btrfs.
I definitely think we need to keep xfstests in the QA process of btrfs for now (and the long term?). And it seems in the short term we can expect the number of failing tests to grow, not shrink.
Greg