Re: [opensuse-testing] CommunityWeek - Testing/QA Slot
Stephen G. Shaw Software Engineer stshaw@novell.com 801 861 7526 Novell Making IT Work As One
Alberto Passalacqua <alberto.passalacqua@tin.it> 04/30/09 8:06 AM >>> Hi Chris,
Il giorno mer, 29/04/2009 alle 23.50 -0500, Chris Cox ha scritto:
I suppose I'm willing. What exactly do "we" need? I don't mind "starting". I sort of "test" openSUSE all the time. I guess I don't know exactly "what" we test (everything?).
That's something that can be discussed during the community week I guess. In the meanwhile I would say "just use it in your regular routine usage".
In conclusion, at the moment it seems the idea of creating a stable, community-based, testing team is not feasible. We'll see however if someone shows up at the community week.
How do I "show up"? I didn't see specifics about webcasts or IRCs? Did I miss something?
The request for volunteers was in the slides I sent around and in almost all my emails about the idea of testing team. ;-) Anyway, thanks for showing up :)
On my side, I'm oriented to dedicate my efforts to other projects, more related to my activity. I hope someone else will try again, and succeed in making this idea reality. Good luck.
Willing to get involved.... you've been warned :)
Do you want to lead the idea? If so, that would be great. Best, Alberto You might not get all the volunteers that you'd hope to have a very successful group, but if you start it and pursue it I'd imagine that over time it will grow because you have created the infrastructure. Even if there are few, I'd still create a test plan. You might not be able to complete it, but at least there is a clear path to follow and anyone with a spare minute can jump in and complete a task or 2. I'll see if we can get access to testopia on bugzilla.novell.com. I think that could help with trying to build a community around testing. What are everyone's thought? Stephen -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Hi, Il giorno lun, 04/05/2009 alle 08.55 -0600, Stephen Shaw ha scritto:
You might not get all the volunteers that you'd hope to have a very successful group, but if you start it and pursue it I'd imagine that over time it will grow because you have created the infrastructure.
Well I agree at a certain extent, but I don't think systematic testing can be done with the current number of people involved, simply because they're not working on it full-time. That's why I said "test it you your regular routine": something more specific and organized would required to divide tasks about participants, which implies a sufficient number of involved people.
Even if there are few, I'd still create a test plan. You might not be able to complete it, but at least there is a clear path to follow and anyone with a spare minute can jump in and complete a task or 2. I'll see if we can get access to testopia on bugzilla.novell.com. I think that could help with trying to build a community around testing.
I don't know. The idea didn't really attract the interest of users as I thought, and given the feedback it seems to me that setting up the infrastructure is going to be an effort which will be used only by a very limited number of interested people. At this point, instead of thinking to a stable test team, I would try to improve what already exists for those who are willing to report the issues they find. Also, I am not really willing to push the testing team idea much further, because I keep reading bad replies in bugzilla and on ML, where people reporting a problem is required to provide fix if they want a problem solved, and where the tone is frequently perceived as challenging for the reporter. This kind of behaviour is one of the reasons why people potentially interested in the testing idea did not show up, and I actually share the same feelings everytime I have to deal with bugzilla reports. If problems are not solved timely and the attitude towards who does the reporting activity change, the testing team idea can't become reality, but this cannot really come from the community. Best, A. -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Hi all,
[...]
Also, I am not really willing to push the testing team idea much further, because I keep reading bad replies in bugzilla and on ML, where people reporting a problem is required to provide fix if they want a problem solved, and where the tone is frequently perceived as challenging for the reporter. This kind of behaviour is one of the reasons why people potentially interested in the testing idea did not show up, and I actually share the same feelings everytime I have to deal with bugzilla reports. If problems are not solved timely and the attitude towards who does the reporting activity change, the testing team idea can't become reality, but this cannot really come from the community.
It looks like Alberto is a bit disappointed with his original idea. But there seem to be quite a number people that are interested in such a team. What about setting up an IRC meeting to talk about possiblenext steps in creating a testing team? Maybe Alberto will get some motivation when he sees that there are still people that are supporting his idea and willing to contribute. To find a date for such a meeting it would be good to know how everybody's available. I'm usually available whole day from 6:00 to 19:00 UTC (=8:00 to 21:00 CEST). Best wishes, Holgi -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Il giorno mar, 05/05/2009 alle 15.39 +0200, Holger Sickenberg ha scritto:
It looks like Alberto is a bit disappointed with his original idea.
But there seem to be quite a number people that are interested in such a team. What about setting up an IRC meeting to talk about possiblenext steps in creating a testing team? Maybe Alberto will get some motivation when he sees that there are still people that are supporting his idea and willing to contribute.
To find a date for such a meeting it would be good to know how everybody's available. I'm usually available whole day from 6:00 to 19:00 UTC (=8:00 to 21:00 CEST).
Me too, excluding on Wednesday from 17.00 to 19.00 UTC because I have a regular weekly meeting at office. Of course if there is people interested, I'm available too and we can start to think to what can be done. Probably the initial idea was a bit too wide... Best, A. -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
On Tuesday 05 May 2009 10:38:19 am Alberto Passalacqua wrote:
Probably the initial idea was a bit too wide...
:-) Yes it was, and hidden in nice presentation. I'll extract points from presentation to normal wiki text. The part I already did helped me to see that BugBusters would be just a part of total testing. Also, creating testing infrastructure is the way to attract contributors that have no time nor skills, to develop their own. Having plan (and infrastructure) that counts on occasional testers, like Google image labeling counts on occasional contributors. I have, still just gut feeling, that what you can see and try on http://images.google.com/imagelabeler/ is possible with almost anything. Make lists of tests, very small ones, and offer it randomly, within topic/application. It is not necessary to create duel game style tests, it will help a lot to describe one particular task and give expected result, and ask what happened. Examples: - like drag and drop something and ask for description, then evaluate result. (it will give idea how good is drag'n'drop) - open application and find version, and evaluate time to find it (this will tell how obvious is that placed) - tell user to do set something up and ask him did he succeeded. You get the idea. Instead of asking expertize and hours of user time, you get detailed tests results with user having to spend 5-10 minutes per test. -- Regards, Rajko http://news.opensuse.org/category/people-of-opensuse/ -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Il giorno mar, 05/05/2009 alle 18.52 -0500, Rajko M. ha scritto:
On Tuesday 05 May 2009 10:38:19 am Alberto Passalacqua wrote:
Probably the initial idea was a bit too wide...
:-)
Yes it was, and hidden in nice presentation.
Probably. I still think it is what we need, not a bit less, to obtain something.
I'll extract points from presentation to normal wiki text. The part I already did helped me to see that BugBusters would be just a part of total testing.
Also, creating testing infrastructure is the way to attract contributors that have no time nor skills, to develop their own.
No sorry. The stable testing team is for dedicated people with some experience, as clarified in the slides. They have at least to know the basics of Linux, how to report bugs, write tutorials and such. Of course others on the long run can learn, but from who if nobody has that knowledge?
Having plan (and infrastructure) that counts on occasional testers, like Google image labeling counts on occasional contributors. I have, still just gut feeling, that what you can see and try on http://images.google.com/imagelabeler/ is possible with almost anything.
I don't think there is an analogy. There is not a basic infrastructure like Google has. We _need_ people that _stably_ in the community perform testing.
Make lists of tests, very small ones, and offer it randomly, within topic/application. It is not necessary to create duel game style tests, it will help a lot to describe one particular task and give expected result, and ask what happened.
I don't want "random testing", that's what we have, and still it leaves too much room to too many problems. Feel free to bring the idea on though.
Instead of asking expertize and hours of user time, you get detailed tests results with user having to spend 5-10 minutes per test.
Well, hours is what we need. In ten minutes you barely report the bug you find on your way, and it is what we do already. Hours don't need to be contiguous, but we need people that is willing to spend some hour and not ten minutes on testing. I think it is not useful to think it can be done "in ten minutes", because it is unrealistic. Bye, A. -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
On Tuesday 05 May 2009 08:26:03 pm Alberto Passalacqua wrote:
Il giorno mar, 05/05/2009 alle 18.52 -0500, Rajko M. ha scritto:
On Tuesday 05 May 2009 10:38:19 am Alberto Passalacqua wrote:
Probably the initial idea was a bit too wide...
:-)
Yes it was, and hidden in nice presentation.
Probably. I still think it is what we need, not a bit less, to obtain something.
I agree on that one. Goal is comprehensive testing, but we have to start somewhere. First is to define smaller goals. What would be that? - testing features from openFATE that are implemented Take few, list it, reserve it from - to and enjoy. Ask for assistance on the list. The scratch book could be openSUSE wiki. - looking for bugs that need confirmation/assistance Similar as above. - defining area of interest, expertise, available hardware Freedom to choose whatever we want is nice, but giving some rough directions what one can test is better. Instead of asking on the list does someone can confirm bug, it will be "hey Joe, do you have this?"
I'll extract points from presentation to normal wiki text. The part I already did helped me to see that BugBusters would be just a part of total testing.
Also, creating testing infrastructure is the way to attract contributors that have no time nor skills, to develop their own.
No sorry. The stable testing team is for dedicated people with some experience, as clarified in the slides.
I would add word "core" testing team. We need more then few dedicated.
They have at least to know the basics of Linux, how to report bugs, write tutorials and such. Of course others on the long run can learn, but from who if nobody has that knowledge?
Having plan (and infrastructure) that counts on occasional testers, like Google image labeling counts on occasional contributors. I have, still just gut feeling, that what you can see and try on http://images.google.com/imagelabeler/ is possible with almost anything.
I don't think there is an analogy. There is not a basic infrastructure like Google has. We _need_ people that _stably_ in the community perform testing.
Like above. The goal and starting point can differ a lot. The Google didn't have infrastructure for this just short time ago.
Make lists of tests, very small ones, and offer it randomly, within topic/application. It is not necessary to create duel game style tests, it will help a lot to describe one particular task and give expected result, and ask what happened.
I don't want "random testing", that's what we have, and still it leaves too much room to too many problems. Feel free to bring the idea on though.
The testing will be random from casual tester point of view, but it will cover much more than it is covered now, when many people "test" just small fraction of features that one can discover. Besides, it will serve additional purpose of educating users about features. That can be also motivation to play a game that will benefit everyone. Users can find the ways to do things they did not suspect exist, openSUSE can have statistics what users look for, how easy is to find features, what functions work as expected, and what do strange things.
Instead of asking expertize and hours of user time, you get detailed tests results with user having to spend 5-10 minutes per test.
Well, hours is what we need. In ten minutes you barely report the bug you find on your way, and it is what we do already. Hours don't need to be contiguous, but we need people that is willing to spend some hour and not ten minutes on testing.
That people willing to spend hours can use their time better to create tests for the rest, analyze results, and create new targeted tests for problematic areas.
I think it is not useful to think it can be done "in ten minutes", because it is unrealistic.
You know my comment about 10 minutes for bug report. Almost each is asking for few hours. From finding how to reproduce it, check upstream, downstream, left and right, find someone with similar configuration, file bug report and followup with openSUSE developers, file a comment, or two with upstream, check changes, compile driver with debug enabled, and so on. If we go with current system, we need people able to give away many hours + interested + knowledgeable. That will unnecessary shrink number of candidates to few that currently show interest. When you add other divisions, KDE/GNOME and per subsystem, there will be no worthwhile testing before number of testers grow to few hundreds. Another idea, maybe creating named list of testers with contribution and moral and material incentives can help to attract more those that keep their results for them selves.
Bye, A.
-- Regards, Rajko http://news.opensuse.org/category/people-of-opensuse/ -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Il giorno mar, 05/05/2009 alle 23.47 -0500, Rajko M. ha scritto:
When you add other divisions, KDE/GNOME and per subsystem, there will be no worthwhile testing before number of testers grow to few hundreds.
Hehe, I said it was a "bit" too wide, not extremely wide. I hardly intended to create a testing team of a comparable size to the number of recognised community members ;-) Anyway, let's discuss this all together following Holger's idea of a meeting. Maybe people will come when they start to see some movement. Best, A. -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
participants (4)
-
Alberto Passalacqua -
Holger Sickenberg -
Rajko M. -
Stephen Shaw