Holger Sickenberg wrote:
Hi Alberto,
Hi guys, my name is Christian Hueller, i also work in the QA department at Novell/SUSE and also would like to offer help and information to a community quality group.
Do you have that already or do you want me to tell something about the current way of testing at SUSE?
We did not talk about the current way of testing with Henne, so it would be great if you could tell me something :-)
Going not too much into details: current testing of openSUSE is divided in three parts: 1)Installation/Update Tests Test installing and updating from the former openSUSE version (for 11.1 it was 11.0) of every milestone (beta) version both on x86 and x86_64. (If time permits also installation tests on PlayStation 3) 2)Automated Kernel Test Running a subset of available Kernel Test Suites on every milestone version (x86/x86_64) 3)Feature Tests Based on the list of new features test if features are implemented and working as expected (see http://en.opensuse.org/Testing:Features_11.1)
All test cases and results are stored in the tool "Testopia" (an enhancement to Bugzilla) which is unfortunately just accessable for people inside Novell.
We are working on opening it asap, however this might still be some time into the future. In case somebody is interested in the tool itself, one can find it's project pages here: http://www.mozilla.org/projects/testopia/.
With the help of the openSUSE community (developers/testing team) I do see room for improvement for milestone testing in following areas:
- test of installation/updates on a wider hardware base
- better test coverage of feature tests
- creating and executing of test cases for base system and applications (also useful for maintenance updates of packages during product lifetime)
I hope this information is a good base for further discussions.
Let me add some more bits, especially areas where we are still expanding our efforts. o Benchmarking is becoming a real big item in QA. Kernel performance benchmarks showed some performance degradation in recent kernel releases which pointed to on the one hand still insufficient benchmarks and the need for some process to check for regression on the other hand. o Automated testing and coverage (outside the kernel) - currently mostly CLI based applications - next big item: GUI testing, since QT4 is used, LDTP (http://ldtp.freedesktop.org/wiki/) is becoming an option * Related to both above: Open Source Test Suites. There are plenty available, however reviewing and adopting them eats up time, so this also is an ongoing process. Here I see a lot of gain from more people simply looking and talking about them in a public discussion. o Documentation. It is there, though it could be better - as always ;) o much more I'm just not thinking of right now. There are three pages on opensuse.org right now (maybe more I'm not aware of), which I think can cause confusion or lead to missing information. I see these currently: http://en.opensuse.org/QA -> Redirects to http://en.opensuse.org/QA_Team, an Overview of QA personnel at Novell/SUSE http://opensuse.org/QA -> Empty page http://en.opensuse.org/Testing -> Testing page created around 11.1, mostly of 11.1 related test material My idea would be to use http://opensuse.org/QA as kind of entry point and then link to all other QA/Testing related pages from there, but thats just an suggestion, I do not want to tell you what to do but mere try to support you in the decisions that you make and give input where I have it. I'll send out some presentations and documentation if it is up to date and valid, just to give an better impression what QA is doing in here. If you have questions, please ask, we'll try to help the best we can. Have fun and thanks, Chris
Best wishes, Holgi
-- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org