On Fri, Jul 15, 2011 at 10:08 AM, Greg Freemyer email@example.com wrote:
On Thu, Jul 14, 2011 at 4:09 PM, Bernhard M. Wiedemann firstname.lastname@example.org wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Am 14.07.2011 17:07, schrieb Greg Freemyer:
For clarity xfstests is not in OBS / factory. It has to pulled from git and compiled as documented in the above wiki page.
Does that work for your automated testing? If not, it seems easy enough to package and put into a devel repo. I don't know if it makes sense in factory?
Let me know if I can do anything.
I made this non-interactive script from the wiki page (fixing the BuildRequires in the process): http://openqa.opensuse.org/opensuse/qatests/qa_btrfs.sh
and got (after running quite a while) as result Failed 4 of 50 tests which might match your results...
I reached out to the xfs list to see if test 075 as an example had a bug, or if it was btrfs itself that was faulty. I was told for that one in particular the test was fine, so btrfs is the culprit. They implied the other tests should be stable as well.
As to 4 failures, I get different results depending on the history of tests I've run since the last mkfs. So, I think it is safe to say at least one of the tests causes filesystem corruption that causes other tests to fail.
I think your automated test is fine as is, but anyone delving into a particular failure should mkfs a nice clean filesystem, then run the test standalone.
As to speed, the "-g auto" arg to ./check is controlling which tests to run. You can run -f quick and it should finish in under 5 minutes. For me the failing tests are also part of the quick group.
See the file "group" in the source to see which tests are in which group.
Also, if the quick set of tests is fast enough, you could use xfstests to test xfs and ext4 as well. I don't know what other filesystems xfstests supports.
We can still package xfstests on OBS and push it to devel:openSUSE:Factory or such where os-autoinst also resides. Could make things somewhat simpler.
I will try to package it up and submit it. Looks pretty simple.
Ciao Bernhard M.
Several of the discrete tests from xfstests have pending patches for btrfs. They were just posted in the last week.
I believe the patches will increase the number of tests that attempt to run against btrfs and thus cause more failures, as opposed to fixing tests that are showing false failures on btrfs.
I definitely think we need to keep xfstests in the QA process of btrfs for now (and the long term?). And it seems in the short term we can expect the number of failing tests to grow, not shrink.