On Thu, May 16, 2013 at 12:37 AM, Alberto Planas Dominguez <aplanas@suse.de> wrote:
The number of combinations that we need to cover is 240, and we need to do this for every stable build that comes from Factory, with more attention before milestones, betas or RCs.
If we add a new medium, this number will add 48 more different scenarios where we need to test, and this is only in the installation part, where the desktop has less impact.
Hi, Alberto, I understand your concern and really appreciate your work to make our distro unbreakable. But this growing trend will certainly drive us mad someday. I don't know if our OpenQA can do all the testing, but I think anyway visual machines can't replace human-beings in the near future. So you'll be always lacking of capable hands. As testers can't grow as fast as maintainers, why not make maintainers themselves do the testing by policy instead of ethic? eg: some DEs are "officially tested by tester team", some DEs of small user database are "maintainer guaranteed". We just test base-system and GNOME/KDE. And actually that's just the fact right now...lightweight DEs always have few maintainers, if they don't even guarante its working state, I certainly won't use it. Actually those DEs under development in OBS are all "maintainer guaranteed" fow now, they've already started getting users but have to wait for OpenQA to test and make them official. So why not, in the mid term, give them an entrance to second DVD by making a policy like "if you want it in, show me 100 people's test results", give them an "official unstable" entrance? It's kinda like creating a DVD on SuSEstudio yourself, but get backup from and promoted by openSUSE because you've proved you're usable although still need further professional testing. I think that's what KlyDE do for now, I see Jos, Will and AJ, so I know it's of course usable even it's now in SuSEstudio. All we need is to add an entry on s.o.o and tell users it hasn't been officially tested yet. It's kinda like "fix them then give out" vs "give them something to play first then fix them". I think users may prefer the last way, we just need some way to pick ourselves out of the responsibility. We can just tell them "okay I'm about to test it, but I got my hands full so it's on schedule. you're encouraged to fix and find workarounds". At the same time, we can still improve our OpenQA's capacity in the long term. They do not conflict with each other. Greetings Marguerite -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org