Re: Fw: [opensuse-boosters] Proposal for Building, Managing Code Testing Ecosystem

Hello Bernard, Thx for those links, am reviewing to get a picture what is currently setup. 1. Your openqa deployment looks very interesting. Good stuff. I'll take a closer look at what you're doing there. 2. From your links and mentioning Testopia I can see that not long ago (Oct 2010) Holer Sickenberg did a presentation on how openSUSE development is using Testopia. Was able to locate his presentation slides and am going to digest that info over the next few days. Early impression is that like so many other new technologies there doesn't seem to be ready easy documentation for anyone inexperienced with the Dev/Testing cycle to install and use and although I can see the Testopia manual tries very hard to explain as simply as possible it's either too slow or not organized well enough for my taste. Testopia is interesting as one approach to create a formal structure bridging Dev and Testing, how widely is it used and what is the feedback? For starters, a quick search of the Technical Forums (Beta Testing and Applications) returns zero results for Testopia so it seems that no one has attempted to implement with "average" users. 3. I won't disagree with your observations about the FOSS development cycle you describe, but to the End User current results often appear like Waterfall... In other words, Users typically evaluate maturity and progression of code by the Release, not by the steps along the way. a. Detailed specs. I don't know if that should be specific to any Dev model, IMO a stated roadmap itemizing both major and minor objectives on a timeline is important even if not closely followed. b. That's good if design can be modified during building, is a departure from Waterfall c. Not sure what you mean by verification. Skimming the Testopia manual, I agree with it that before code is assigned for testing, Q/A should at least consider <what> is being tested and possibly inform the Tester what to look for, and what kind of information is relevant and needs to be collected if errors are encountered. d.That's interesting(... allowing for feedback...). I just posted on the Technical Forums a few weeks ago my frustration that current OBS and other openSUSE package repositories <do not> provide any way of sending feedback to the Package Maintainer, which is different than Sourceforge and Freshmeat. AFAIK the only way to report a problem is Bugzilla which isn't the friendliest software unless you work with it constantly. For now, perhaps the more on point question I might ask is: For anyone who might feel that the pool of people willing to test software needs to be enlarged, have those needs been explicitly itemized? What really needs to be tested(and maybe how)? What is the profile of the target Prospective Tester? Maybe only after those questions are answered might followup questions be asked like how to grow that community. Tony On Tue, Mar 29, 2011 at 6:48 AM, Bernhard M. Wiedemann wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

...

Date: Mon, 28 Mar 2011 10:47:59 -0700 From: Tony Su To: opensuse-boosters@opensuse.org Subject: [opensuse-boosters] Proposal for Building, Managing Code Testing Ecosystem

Hello Group,

I was informed that this was the best place to submit a rather large and potentially influential openSUSE project.

Through Bryen and the openSUSE marketing group, I came to understand that a call for Users willing to test code has been made.

Along with one other in that group and considering my ongoing participation in the openSUSE Technical Help forums, the two of us are proposing building a state of art ecosystem of Users willing to test and assist in developing Code.

Please take a few minutes to read the proposal I've posted at the following URL, the website is set to be accessible only by those who know the URL (it's not supposed to be publicly searchable) and although it's visible anonymously, editing requires logging in with a Google account.

https://sites.google.com/site/projectcodedev/

Do be forthright with any opinion about the proposed Project, myself and anything else, I will take any comment constructively and after many years have seen and experienced just about anything that can and has been said in unmoderated as well as moderated forums. If the Proposal is deemed unsupportable, I might be disappointed but can accept that as well.

:)

Thank you, Tony Su

Hi,

I have been driving quite some part of the testing efforts for 11.4 (sometimes jokingly calling myself "HobbyQA")

In particular, I created http://openqa.opensuse.org/ on a dedicated machine in Nuremberg (because it needs hardware virt and plenty resources). This helps a lot to auto-maintain the factory-tested repo as something that will not be arbitrarily broken and thus is better suited for human testers than openSUSE-Factory.

for testing coordination/communication there is atm the ML http://lists.opensuse.org/archive/opensuse-testing/

and wiki http://en.opensuse.org/openSUSE:Testing

and IRC meetings http://en.opensuse.org/openSUSE:Testing_meeting

and the new social thingy: https://connect.opensuse.org//pg/groups/12174/opensuse-testers/

in principle, there is also testopia on bugzilla.novell.com, but until now it was not really put to good use - possibly because of certain shortcomings in the setup which we can not influence.

I read most of the proposal pages and was thinking, that Linux Distributions do not do "Waterfall" model much, because there are a) rarely detailed specs upfront, b) design often happens as needed during implementation, c) verification is rarely comprehensive or specific to new code d) there are several pre-releases allowing for feedback and subsequent improvements (even Factory(-tested) providing daily updates)

of course, distributions also mainly consist of upstream project code with their own devel/release cycles and methods.

However, some way to better coodinate testers and testing would be nice. Giving credits and/or rewards to testers is also a good thing to do. And there is more room for improvement.

Ciao Bernhard M. - -- you have to know your goal for it makes it much easier to reach -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org/

iEYEARECAAYFAk2R40oACgkQSTYLOx37oWRlkwCbBZc/fgUxJDvAa9e+5j8AjXBt 6rAAoMSwq+fBXvT3Fh4EWtVqPUhIVzg2 =ELSM -----END PGP SIGNATURE-----

-- To unsubscribe, e-mail: opensuse-boosters+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-boosters+help@opensuse.org