Hi all,

[...]

Also, I am not really willing to push the testing team idea much further, because I keep reading bad replies in bugzilla and on ML, where people reporting a problem is required to provide fix if they want a problem solved, and where the tone is frequently perceived as challenging for the reporter. This kind of behaviour is one of the reasons why people potentially interested in the testing idea did not show up, and I actually share the same feelings everytime I have to deal with bugzilla reports. If problems are not solved timely and the attitude towards who does the reporting activity change, the testing team idea can't become reality, but this cannot really come from the community.

It looks like Alberto is a bit disappointed with his original idea. But there seem to be quite a number people that are interested in such a team. What about setting up an IRC meeting to talk about possiblenext steps in creating a testing team? Maybe Alberto will get some motivation when he sees that there are still people that are supporting his idea and willing to contribute. To find a date for such a meeting it would be good to know how everybody's available. I'm usually available whole day from 6:00 to 19:00 UTC (=8:00 to 21:00 CEST). Best wishes, Holgi -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org

On Tuesday 05 May 2009 10:38:19 am Alberto Passalacqua wrote:

Probably the initial idea was a bit too wide...

:-) Yes it was, and hidden in nice presentation. I'll extract points from presentation to normal wiki text. The part I already did helped me to see that BugBusters would be just a part of total testing. Also, creating testing infrastructure is the way to attract contributors that have no time nor skills, to develop their own. Having plan (and infrastructure) that counts on occasional testers, like Google image labeling counts on occasional contributors. I have, still just gut feeling, that what you can see and try on http://images.google.com/imagelabeler/ is possible with almost anything. Make lists of tests, very small ones, and offer it randomly, within topic/application. It is not necessary to create duel game style tests, it will help a lot to describe one particular task and give expected result, and ask what happened. Examples: - like drag and drop something and ask for description, then evaluate result. (it will give idea how good is drag'n'drop) - open application and find version, and evaluate time to find it (this will tell how obvious is that placed) - tell user to do set something up and ask him did he succeeded. You get the idea. Instead of asking expertize and hours of user time, you get detailed tests results with user having to spend 5-10 minutes per test. -- Regards, Rajko http://news.opensuse.org/category/people-of-opensuse/ -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org

On Tuesday 05 May 2009 08:26:03 pm Alberto Passalacqua wrote:

Il giorno mar, 05/05/2009 alle 18.52 -0500, Rajko M. ha scritto:

...

On Tuesday 05 May 2009 10:38:19 am Alberto Passalacqua wrote:

...

Probably the initial idea was a bit too wide...

:-)

Yes it was, and hidden in nice presentation.

Probably. I still think it is what we need, not a bit less, to obtain something.

I agree on that one. Goal is comprehensive testing, but we have to start somewhere. First is to define smaller goals. What would be that? - testing features from openFATE that are implemented Take few, list it, reserve it from - to and enjoy. Ask for assistance on the list. The scratch book could be openSUSE wiki. - looking for bugs that need confirmation/assistance Similar as above. - defining area of interest, expertise, available hardware Freedom to choose whatever we want is nice, but giving some rough directions what one can test is better. Instead of asking on the list does someone can confirm bug, it will be "hey Joe, do you have this?"

...

I'll extract points from presentation to normal wiki text. The part I already did helped me to see that BugBusters would be just a part of total testing.

Also, creating testing infrastructure is the way to attract contributors that have no time nor skills, to develop their own.

No sorry. The stable testing team is for dedicated people with some experience, as clarified in the slides.

I would add word "core" testing team. We need more then few dedicated.

They have at least to know the basics of Linux, how to report bugs, write tutorials and such. Of course others on the long run can learn, but from who if nobody has that knowledge?

...

Having plan (and infrastructure) that counts on occasional testers, like Google image labeling counts on occasional contributors. I have, still just gut feeling, that what you can see and try on http://images.google.com/imagelabeler/ is possible with almost anything.

I don't think there is an analogy. There is not a basic infrastructure like Google has. We _need_ people that _stably_ in the community perform testing.

Like above. The goal and starting point can differ a lot. The Google didn't have infrastructure for this just short time ago.

...

Make lists of tests, very small ones, and offer it randomly, within topic/application. It is not necessary to create duel game style tests, it will help a lot to describe one particular task and give expected result, and ask what happened.

I don't want "random testing", that's what we have, and still it leaves too much room to too many problems. Feel free to bring the idea on though.

The testing will be random from casual tester point of view, but it will cover much more than it is covered now, when many people "test" just small fraction of features that one can discover. Besides, it will serve additional purpose of educating users about features. That can be also motivation to play a game that will benefit everyone. Users can find the ways to do things they did not suspect exist, openSUSE can have statistics what users look for, how easy is to find features, what functions work as expected, and what do strange things.

...

Instead of asking expertize and hours of user time, you get detailed tests results with user having to spend 5-10 minutes per test.

Well, hours is what we need. In ten minutes you barely report the bug you find on your way, and it is what we do already. Hours don't need to be contiguous, but we need people that is willing to spend some hour and not ten minutes on testing.

That people willing to spend hours can use their time better to create tests for the rest, analyze results, and create new targeted tests for problematic areas.

I think it is not useful to think it can be done "in ten minutes", because it is unrealistic.

You know my comment about 10 minutes for bug report. Almost each is asking for few hours. From finding how to reproduce it, check upstream, downstream, left and right, find someone with similar configuration, file bug report and followup with openSUSE developers, file a comment, or two with upstream, check changes, compile driver with debug enabled, and so on. If we go with current system, we need people able to give away many hours + interested + knowledgeable. That will unnecessary shrink number of candidates to few that currently show interest. When you add other divisions, KDE/GNOME and per subsystem, there will be no worthwhile testing before number of testers grow to few hundreds. Another idea, maybe creating named list of testers with contribution and moral and material incentives can help to attract more those that keep their results for them selves.

Bye, A.

-- Regards, Rajko http://news.opensuse.org/category/people-of-opensuse/ -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org