Re: Fw: [opensuse-boosters] Proposal for Building, Managing Code Testing Ecosystem
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Date: Mon, 28 Mar 2011 10:47:59 -0700 From: Tony Su
To: opensuse-boosters@opensuse.org Subject: [opensuse-boosters] Proposal for Building, Managing Code Testing Ecosystem Hello Group,
I was informed that this was the best place to submit a rather large and potentially influential openSUSE project.
Through Bryen and the openSUSE marketing group, I came to understand that a call for Users willing to test code has been made.
Along with one other in that group and considering my ongoing participation in the openSUSE Technical Help forums, the two of us are proposing building a state of art ecosystem of Users willing to test and assist in developing Code.
Please take a few minutes to read the proposal I've posted at the following URL, the website is set to be accessible only by those who know the URL (it's not supposed to be publicly searchable) and although it's visible anonymously, editing requires logging in with a Google account.
https://sites.google.com/site/projectcodedev/
Do be forthright with any opinion about the proposed Project, myself and anything else, I will take any comment constructively and after many years have seen and experienced just about anything that can and has been said in unmoderated as well as moderated forums. If the Proposal is deemed unsupportable, I might be disappointed but can accept that as well.
:)
Thank you, Tony Su
Hi, I have been driving quite some part of the testing efforts for 11.4 (sometimes jokingly calling myself "HobbyQA") In particular, I created http://openqa.opensuse.org/ on a dedicated machine in Nuremberg (because it needs hardware virt and plenty resources). This helps a lot to auto-maintain the factory-tested repo as something that will not be arbitrarily broken and thus is better suited for human testers than openSUSE-Factory. for testing coordination/communication there is atm the ML http://lists.opensuse.org/archive/opensuse-testing/ and wiki http://en.opensuse.org/openSUSE:Testing and IRC meetings http://en.opensuse.org/openSUSE:Testing_meeting and the new social thingy: https://connect.opensuse.org//pg/groups/12174/opensuse-testers/ in principle, there is also testopia on bugzilla.novell.com, but until now it was not really put to good use - possibly because of certain shortcomings in the setup which we can not influence. I read most of the proposal pages and was thinking, that Linux Distributions do not do "Waterfall" model much, because there are a) rarely detailed specs upfront, b) design often happens as needed during implementation, c) verification is rarely comprehensive or specific to new code d) there are several pre-releases allowing for feedback and subsequent improvements (even Factory(-tested) providing daily updates) of course, distributions also mainly consist of upstream project code with their own devel/release cycles and methods. However, some way to better coodinate testers and testing would be nice. Giving credits and/or rewards to testers is also a good thing to do. And there is more room for improvement. Ciao Bernhard M. - -- you have to know your goal for it makes it much easier to reach -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org/ iEYEARECAAYFAk2R40oACgkQSTYLOx37oWRlkwCbBZc/fgUxJDvAa9e+5j8AjXBt 6rAAoMSwq+fBXvT3Fh4EWtVqPUhIVzg2 =ELSM -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-boosters+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-boosters+help@opensuse.org
Hello Bernard,
Thx for those links, am reviewing to get a picture what is currently setup.
1. Your openqa deployment looks very interesting. Good stuff. I'll
take a closer look at what you're doing there.
2. From your links and mentioning Testopia I can see that not long ago
(Oct 2010) Holer Sickenberg did a presentation on how openSUSE
development is using Testopia.
Was able to locate his presentation slides and am going to digest that
info over the next few days. Early impression is that like so many
other new technologies there doesn't seem to be ready easy
documentation for anyone inexperienced with the Dev/Testing cycle to
install and use and although I can see the Testopia manual tries very
hard to explain as simply as possible it's either too slow or not
organized well enough for my taste.
Testopia is interesting as one approach to create a formal structure
bridging Dev and Testing, how widely is it used and what is the
feedback? For starters, a quick search of the Technical Forums (Beta
Testing and Applications) returns zero results for Testopia so it
seems that no one has attempted to implement with "average" users.
3. I won't disagree with your observations about the FOSS development
cycle you describe, but to the End User current results often appear
like Waterfall... In other words, Users typically evaluate maturity
and progression of code by the Release, not by the steps along the
way.
a. Detailed specs. I don't know if that should be specific to any Dev
model, IMO a stated roadmap itemizing both major and minor objectives
on a timeline is important even if not closely followed.
b. That's good if design can be modified during building, is a
departure from Waterfall
c. Not sure what you mean by verification. Skimming the Testopia
manual, I agree with it that before code is assigned for testing, Q/A
should at least consider <what> is being tested and possibly inform
the Tester what to look for, and what kind of information is relevant
and needs to be collected if errors are encountered.
d.That's interesting(... allowing for feedback...). I just posted on
the Technical Forums a few weeks ago my frustration that current OBS
and other openSUSE package repositories <do not> provide any way of
sending feedback to the Package Maintainer, which is different than
Sourceforge and Freshmeat. AFAIK the only way to report a problem is
Bugzilla which isn't the friendliest software unless you work with it
constantly.
For now, perhaps the more on point question I might ask is:
For anyone who might feel that the pool of people willing to test
software needs to be enlarged, have those needs been explicitly
itemized? What really needs to be tested(and maybe how)? What is the
profile of the target Prospective Tester?
Maybe only after those questions are answered might followup questions
be asked like how to grow that community.
Tony
On Tue, Mar 29, 2011 at 6:48 AM, Bernhard M. Wiedemann
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Date: Mon, 28 Mar 2011 10:47:59 -0700 From: Tony Su
To: opensuse-boosters@opensuse.org Subject: [opensuse-boosters] Proposal for Building, Managing Code Testing Ecosystem Hello Group,
I was informed that this was the best place to submit a rather large and potentially influential openSUSE project.
Through Bryen and the openSUSE marketing group, I came to understand that a call for Users willing to test code has been made.
Along with one other in that group and considering my ongoing participation in the openSUSE Technical Help forums, the two of us are proposing building a state of art ecosystem of Users willing to test and assist in developing Code.
Please take a few minutes to read the proposal I've posted at the following URL, the website is set to be accessible only by those who know the URL (it's not supposed to be publicly searchable) and although it's visible anonymously, editing requires logging in with a Google account.
https://sites.google.com/site/projectcodedev/
Do be forthright with any opinion about the proposed Project, myself and anything else, I will take any comment constructively and after many years have seen and experienced just about anything that can and has been said in unmoderated as well as moderated forums. If the Proposal is deemed unsupportable, I might be disappointed but can accept that as well.
:)
Thank you, Tony Su
Hi,
I have been driving quite some part of the testing efforts for 11.4 (sometimes jokingly calling myself "HobbyQA")
In particular, I created http://openqa.opensuse.org/ on a dedicated machine in Nuremberg (because it needs hardware virt and plenty resources). This helps a lot to auto-maintain the factory-tested repo as something that will not be arbitrarily broken and thus is better suited for human testers than openSUSE-Factory.
for testing coordination/communication there is atm the ML http://lists.opensuse.org/archive/opensuse-testing/
and wiki http://en.opensuse.org/openSUSE:Testing
and IRC meetings http://en.opensuse.org/openSUSE:Testing_meeting
and the new social thingy: https://connect.opensuse.org//pg/groups/12174/opensuse-testers/
in principle, there is also testopia on bugzilla.novell.com, but until now it was not really put to good use - possibly because of certain shortcomings in the setup which we can not influence.
I read most of the proposal pages and was thinking, that Linux Distributions do not do "Waterfall" model much, because there are a) rarely detailed specs upfront, b) design often happens as needed during implementation, c) verification is rarely comprehensive or specific to new code d) there are several pre-releases allowing for feedback and subsequent improvements (even Factory(-tested) providing daily updates)
of course, distributions also mainly consist of upstream project code with their own devel/release cycles and methods.
However, some way to better coodinate testers and testing would be nice. Giving credits and/or rewards to testers is also a good thing to do. And there is more room for improvement.
Ciao Bernhard M. - -- you have to know your goal for it makes it much easier to reach -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org/
iEYEARECAAYFAk2R40oACgkQSTYLOx37oWRlkwCbBZc/fgUxJDvAa9e+5j8AjXBt 6rAAoMSwq+fBXvT3Fh4EWtVqPUhIVzg2 =ELSM -----END PGP SIGNATURE-----
-- To unsubscribe, e-mail: opensuse-boosters+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-boosters+help@opensuse.org
participants (2)
-
Bernhard M. Wiedemann
-
Tony Su