On Sunday 07 February 2016 10:03:28 Per Jessen wrote:
Richard Brown wrote:
On 7 February 2016 at 00:40, Greg Freemyer <greg.freemyer@gmail.com>
wrote:
<lots of good stuff ending on this point>
If anyone thinks pre-2015 Factory ever had pre-release testing anywhere close to that they're simply wrong.
Well said
Not only that, but since 2014, SUSE have now adopted openQA for testing their enterprise distributions and addons, so it should go without saying that it's effective, broad, trustworthy testing. oooh, I guess we can say it's 'Enterprise Grade' even ;)
(and if anyone is wondering - yes, SUSE actively contribute their SLE test cases back into openSUSE - we have just one shared test suite for SLE/Leap/Tumbleweed which provides the code for those hundreds of scenarios and thousands of different test module runs)
Do you know where one can go to see what's being tested?
TL;DR: Yes, Greg already described it, read again :-) Greg just posted a very nice introduction of how you can get it from openQA but I will try again to explain that maybe from a slightly different perspective even though this would be a more appropriate topic for "opensuse- factory". Starting from http://openqa.opensuse.org/ you can see what products are currently tested, e.g. Tumbleweed - as a DVD image from Factory snapshot, as well as updates for the releases of e.g. openSUSE 13.2 and openSUSE Leap 42.1. Selecting any Tumbleweed build, say https://openqa.opensuse.org/tests/overview?distri=opensuse&version=Tumbleweed&build=20160205&groupid=1 you can see a table with a column "Test" describing individual scenarios next to green, not-so-green and red bubbles under the column "x86_64", probably the architecture you are interested in. The test scenarios as stated describe which "scenarios" are tested, e.g. starting from top a "RAID0" installation and such. Other scenarios go further than installing trying out if applications work, e.g. "gnome" also testing browsers, etc.. Click on the bubbles to find out what steps exactly are executed in which scenario. If you are interested if a certain application is working fine, say "firefox", this is tested for each Tumbleweed snapshot, as well as e.g. gnucash. So you can be sure that basic functionality is tested every time. If one would encounter a serious issue in one of these applications and it is missed by the openQA tests, feel free to help providing these tests. If you think an application which is not covered should be tested I would recommend to propose it to the openQA test developers first and discuss if and where it should be included. Of course, if we would include all possible applications and scenarios for every product the tests would run too long and we would never get a new Tumbleweed snapshot so a decision has to be made of which importance to a release a certain application is. Certainly, major priority falls to packages included in the DVD image. If you want to learn more about openQA, go to http://openqa.opensuse.org/, click "Learn more", ask on opensuse-factory mailling list or IRC channel. Regards, Oliver -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org