Building the OpenSUSE testing team
Hello, some time ago I proposed the creation of a community based testing team for openSUSE pre-releases. I did not know of the existance of some infrastructure already in place, line the #opensuse-testing IRC channel and the opensuse-testing@opensuse.org mailing list, as I did not know of the work done by Holgie, daemon and their friends! Anyway, after some discussion with Zonker, I have been encouraged to move the topic here officially. I'm sorry for the repetitions that happened in different mailing lists. I'll try to do better next time! The main idea was to create a group of stable testers to deeply test openSUSE functionalities on their machines, in order to catch annoying bugs earlier in the development stages, allowing the fixes to be done and tested again in a more timely and less rushed manner. It won't be easy, we need motivated volunteers and we have a lot to learn to do it the right way, but I think it is worth the effort, if we want to keep faith to the guiding principles, which states openSUSE aims to be "the most usable Linux distribution" (and not the most cutting edge one!). I summed up the motivations, a short problem analysis about the current status quo, and some proposal of solution in some slides you can download from here: http://en.opensuse.org/Testing_Team Any suggestion on how to start is more than welcome! With kind regards, Alberto -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Alberto Passalacqua wrote:
Hello,
some time ago I proposed the creation of a community based testing team for openSUSE pre-releases. I did not know of the existance of some infrastructure already in place, line the #opensuse-testing IRC channel and the opensuse-testing@opensuse.org mailing list, as I did not know of the work done by Holgie, daemon and their friends!
Anyway, after some discussion with Zonker, I have been encouraged to move the topic here officially. I'm sorry for the repetitions that happened in different mailing lists. I'll try to do better next time!
The main idea was to create a group of stable testers to deeply test openSUSE functionalities on their machines, in order to catch annoying bugs earlier in the development stages, allowing the fixes to be done and tested again in a more timely and less rushed manner.
It won't be easy, we need motivated volunteers and we have a lot to learn to do it the right way, but I think it is worth the effort, if we want to keep faith to the guiding principles, which states openSUSE aims to be "the most usable Linux distribution" (and not the most cutting edge one!).
I summed up the motivations, a short problem analysis about the current status quo, and some proposal of solution in some slides you can download from here:
http://en.opensuse.org/Testing_Team
Any suggestion on how to start is more than welcome!
With kind regards, Alberto
Hi, you can put my name down, I'm already testing 11.2 mostly package management as I don't have unlimited bandwidth. Besides alpha 11.2 is working remarkably well at the moment. I also don't have enough hd space for multiple systems. Regards Dave P -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Tirsdag 17 marts 2009 18:23:53 skrev Alberto Passalacqua:
Any suggestion on how to start is more than welcome!
I think the first thing we need is for developers/packagers to drop a line here, when they think something is ready for serious widespread testing. One big problem with the current situation is that noone knows what to test and when. One of the main reasons I usually don't test early alphas is that I expect everything to change dramatically later on anyway, hence thinking testing is probably pointless at that point anyway - but I guess some components _are_ ready for widespread testing and that point, we just need to know which. Additionally maybe some automation could be set up where openfate would spam this list when a new feature reaches a certain status, so as to let us know it might be something that needed testing. -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Martin Schlander wrote:
One big problem with the current situation is that noone knows what to test and when. One of the main reasons I usually don't test early alphas is that I expect everything to change dramatically later on anyway, hence thinking testing is probably pointless at that point anyway - but I guess some components _are_ ready for widespread testing and that point, we just need to know which.
I partially agree with this. When 11.0 GM was released, there was a critical bug in the installation process which affected only Japanese users. https://bugzilla.novell.com/show_bug.cgi?id=385172#c34 The most annoying was, that the bug seemed to be fixed in RC1, which we non-Novell persons can test, but reproduced in RC4, which only Novell persons can test, and the bug was not fixed in GM. However, during the 11.1 release process, there was a call for feature testing. http://lists.opensuse.org/archive/opensuse-testing/2008-11/msg00005.html But you can see most of the Test Results for listed features remain "Idle". http://en.opensuse.org/Testing:Features_11.1 We could know *what to test and when* in this case, but as far as I can see, not so many people participated the testing. I think we have to analyze why many people don't have keen interest to testing. Do we need to spread the word much more ? Do we need to clarify what and how things to be tested and where the result should be reported ? Or, do we need to prepare benefits for dedicated testers ? IMHO, although testing itself is very important, it would be more important to figure out a way to participate testing easily, so that as many people as possible can feel "openSUSE is *our* distribution".
Additionally maybe some automation could be set up where openfate would spam this list when a new feature reaches a certain status, so as to let us know it might be something that needed testing.
openFATE screening team has just begun its activities and we are now brainstorming what can and should we do. So your opinion will be inputted. ;-) Best, -- _/_/ Satoru Matsumoto - openSUSE Member - Japan _/_/ _/_/ Marketing/Weekly News/openFATE Screening Team _/_/ _/_/ mail: helios_reds_at_gmx.net / irc: HeliosReds _/_/ _/_/ http://blog.geeko.jp/author/heliosreds _/_/ -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Satoru Matsumoto schrieb:
However, during the 11.1 release process, there was a call for feature testing. http://lists.opensuse.org/archive/opensuse-testing/2008-11/msg00005.html But you can see most of the Test Results for listed features remain "Idle". http://en.opensuse.org/Testing:Features_11.1 We could know *what to test and when* in this case, but as far as I can see, not so many people participated the testing.
I think we have to analyze why many people don't have keen interest to testing. Do we need to spread the word much more ? Do we need to clarify what and how things to be tested and where the result should be reported ? Or, do we need to prepare benefits for dedicated testers ?
This particular call for testing had two reasons why I didn't consider participating: a) It called for testing new features, but I am much more interested, and consider it much more important, to make sure the old features still work correctly, ie. there are no regressions. b) "Basic reqirements: * install openSUSE 11.1 beta 4 * meet in #opensuse-testing on irc.freenode.org" I do not use IRC. Apart from that, I must admit that my testing enthusiasm has been somewhat dampened by a recent series of WONTFIX resolutions for problems I reported, a few of them with closing remarks that seemed to imply what I reported was "my problem". So I guess positive feedback from developers that test reports are appreciated, taken seriously, and the reported issues addressed, would go a long way in encouraging more testers. HTH T. -- Tilman Schmidt E-Mail: tilman@imap.cc Bonn, Germany Diese Nachricht besteht zu 100% aus wiederverwerteten Bits. Ungeöffnet mindestens haltbar bis: (siehe Rückseite)
On Thursday 19 March 2009 10:32:55 Tilman Schmidt wrote:
Satoru Matsumoto schrieb:
However, during the 11.1 release process, there was a call for feature testing. http://lists.opensuse.org/archive/opensuse-testing/2008-11/msg00005.html But you can see most of the Test Results for listed features remain "Idle". http://en.opensuse.org/Testing:Features_11.1 We could know *what to test and when* in this case, but as far as I can see, not so many people participated the testing.
I think we have to analyze why many people don't have keen interest to testing. Do we need to spread the word much more ? Do we need to clarify what and how things to be tested and where the result should be reported ? Or, do we need to prepare benefits for dedicated testers ?
This particular call for testing had two reasons why I didn't consider participating:
a) It called for testing new features, but I am much more interested, and consider it much more important, to make sure the old features still work correctly, ie. there are no regressions.
We need both IMO. We need on the one hand a defined set of use-cases that the distribution is tested against - so that in the end we can say: All use-cases were tested and that's their status so we know exactly which fail and can base a decision of releasing on that. On the other hand, we need to test new features as well.
b) "Basic reqirements: * install openSUSE 11.1 beta 4 * meet in #opensuse-testing on irc.freenode.org" I do not use IRC.
So, let's look at other ways of coordinating as well. IRC can help a lot but should not be a prerequisite for coordinated testing.
Apart from that, I must admit that my testing enthusiasm has been somewhat dampened by a recent series of WONTFIX resolutions for problems I reported, a few of them with closing remarks that seemed to imply what I reported was "my problem". So I guess positive feedback from developers that test reports are appreciated, taken seriously, and the reported issues addressed, would go a long way in encouraging more testers.
If you have any concrete examples, let's discuss these. Andreas -- Andreas Jaeger, Director Platform / openSUSE, aj@suse.de SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) Maxfeldstr. 5, 90409 Nürnberg, Germany GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
Andreas Jaeger schrieb:
On Thursday 19 March 2009 10:32:55 Tilman Schmidt wrote:
Apart from that, I must admit that my testing enthusiasm has been somewhat dampened by a recent series of WONTFIX resolutions for problems I reported, a few of them with closing remarks that seemed to imply what I reported was "my problem". So I guess positive feedback from developers that test reports are appreciated, taken seriously, and the reported issues addressed, would go a long way in encouraging more testers.
If you have any concrete examples, let's discuss these.
Ok: https://bugzilla.novell.com/show_bug.cgi?id=234529 - complete with patch and support offer, but after two years of waiting it won't be merged because of "no resources for ISDN anymore" https://bugzilla.novell.com/show_bug.cgi?id=302004 - WONTFIXed after a year and a half of inactivity, with a comment I frankly don't understand at all (except for the "send patch" bit) https://bugzilla.novell.com/show_bug.cgi?id=309382 https://bugzilla.novell.com/show_bug.cgi?id=301997 - seriously strange - I reported the problem as fixed, with no reaction from anyone, then a year later it is "set to WONTFIX because LATER and REMIND resolutions have been removed" https://bugzilla.novell.com/show_bug.cgi?id=332696 https://bugzilla.novell.com/show_bug.cgi?id=239695 - not recent, but perfect illustrations of the point I'm trying to make HTH T. -- Tilman Schmidt E-Mail: tilman@imap.cc Bonn, Germany Diese Nachricht besteht zu 100% aus wiederverwerteten Bits. Ungeöffnet mindestens haltbar bis: (siehe Rückseite)
On Thursday 19 March 2009 19:10:07 Tilman Schmidt wrote:
Andreas Jaeger schrieb:
On Thursday 19 March 2009 10:32:55 Tilman Schmidt wrote:
Apart from that, I must admit that my testing enthusiasm has been somewhat dampened by a recent series of WONTFIX resolutions for problems I reported, a few of them with closing remarks that seemed to imply what I reported was "my problem". So I guess positive feedback from developers that test reports are appreciated, taken seriously, and the reported issues addressed, would go a long way in encouraging more testers.
If you have any concrete examples, let's discuss these.
Ok:
https://bugzilla.novell.com/show_bug.cgi?id=234529 - complete with patch and support offer, but after two years of waiting it won't be merged because of "no resources for ISDN anymore"
https://bugzilla.novell.com/show_bug.cgi?id=302004 - WONTFIXed after a year and a half of inactivity, with a comment I frankly don't understand at all (except for the "send patch" bit)
https://bugzilla.novell.com/show_bug.cgi?id=309382 https://bugzilla.novell.com/show_bug.cgi?id=301997 - seriously strange - I reported the problem as fixed, with no reaction from anyone, then a year later it is "set to WONTFIX because LATER and REMIND resolutions have been removed"
https://bugzilla.novell.com/show_bug.cgi?id=332696 https://bugzilla.novell.com/show_bug.cgi?id=239695 - not recent, but perfect illustrations of the point I'm trying to make
Besides the first one, all are old examples. We have to become better and I hope we do - but people are also overloaded and they will not fix everything, some stuff also needs to go upstream. If in the future, you have concrete problems, please speak up - but I'm not following up with developers on old bugs. thanks for the list, Andreas -- Andreas Jaeger, Director Platform / openSUSE, aj@suse.de SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) Maxfeldstr. 5, 90409 Nürnberg, Germany GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
On Thu, 26 Mar 2009 16:11 +0100, "Andreas Jaeger" <aj@suse.de> wrote:
On Thursday 19 March 2009 19:10:07 Tilman Schmidt wrote:
Andreas Jaeger schrieb: [...]
If you have any concrete examples, let's discuss these.
Ok:
https://bugzilla.novell.com/show_bug.cgi?id=234529 - complete with patch and support offer, but after two years of waiting it won't be merged because of "no resources for ISDN anymore" [...] Besides the first one, all are old examples. [...]
I notice that the ensuing discussion has concentrated exclusively on the other, out-dated examples and the question whether it is opportune to discuss them again today or if the people who where responsible for them already understand sufficiently what went wrong at the time. I would however like to get back to this first one which is actually still relevant, and becoming somewhat of a touchstone for openSUSE's willingness to accept outside help: - opened on 2007-01-13 for 10.2 - was told it had to wait "for 10.3 or later" - submitted a patch for it on 2007-01-31 - was told on 2007-02-13 by Karsten Keil (then ISDN maintainer) that he would merge it for 10.3 - reported as still missing in 10.3ß1 on 2007-08-20 - was told (in a form so cryptic I didn't understand at first) it was postponed to 11.0 - reported as still missing in 11.0ß1 on 2008-03-24 - WONTFIXed because of "no resources" on 2009-02-26 - My immmediate question which resources would be needed didn't even get a reply. - Obviously my citing that case here didn't have any effect either, because the entry is still RESOLVED/WONTFIX as of today. What else can I do? Thanks, Tilman -- Tilman Schmidt tilman@imap.cc -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Tilman Schmidt a écrit :
On Thu, 26 Mar 2009 16:11 +0100, "Andreas Jaeger" <aj@suse.de> wrote:
- Obviously my citing that case here didn't have any effect either, because the entry is still RESOLVED/WONTFIX as of today.
is the maintainer still maintaining it? jdd -- http://www.dodin.net http://valerie.dodin.org http://www.youtube.com/watch?v=t-eic8MSSfM http://www.facebook.com/profile.php?id=1412160445 -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
On Sat, 11 Apr 2009 20:12 +0200, "jdd" <jdd@dodin.org> wrote:
Tilman Schmidt a écrit :
On Thu, 26 Mar 2009 16:11 +0100, "Andreas Jaeger" <aj@suse.de> wrote:
- Obviously my citing that case here didn't have any effect either, because the entry is still RESOLVED/WONTFIX as of today.
is the maintainer still maintaining it?
I don't know, nor do I know how to find out. I have a hunch that he might not be working for SUSE/Novell anymore, but I may be completely wrong. Does it matter? Should it matter? -- Tilman Schmidt tilman@imap.cc -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Tilman Schmidt a écrit :
I don't know, nor do I know how to find out. I have a hunch that he might not be working for SUSE/Novell anymore, but I may be completely wrong.
Does it matter? Should it matter?
of course, it matters! jdd -- http://www.dodin.net http://valerie.dodin.org http://www.youtube.com/watch?v=t-eic8MSSfM http://www.facebook.com/profile.php?id=1412160445 -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
On Sun, 12 Apr 2009 00:23 +0200, "jdd" <jdd@dodin.org> wrote: [quoting amended]
Tilman Schmidt a écrit :
is the maintainer still maintaining it?
I don't know, nor do I know how to find out. I have a hunch that he might not be working for SUSE/Novell anymore, but I may be completely wrong.
Does it matter? Should it matter?
of course, it matters!
Ok, that splits my question in three: a) How do I get my patch included if he is still maintaining it? b) How do I get my patch included if he isn't maintaining it anymore? c) How do I find out which of the two is the case? Which are, of course, instances of general questions about the way openSUSE want's to handle external contributions: a) What happens to contributions if a maintainer is so overburdened he or she doesn't even have the capacity to approve contributions that are otherwise ready for inclusion? b) What happens to contributions if a maintainer leaves? c) How can contributors find their responsible maintainer (or absence of same)? Thanks, Tilman -- Tilman Schmidt tilman@imap.cc -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
* Tilman Schmidt <tilman@imap.cc> [04-12-09 06:50]:
Ok, that splits my question in three:
a) How do I get my patch included if he is still maintaining it?
Send it to the packager.
b) How do I get my patch included if he isn't maintaining it anymore?
Build the package yourself, or beg that someone else will pick it up.
c) How do I find out which of the two is the case?
I have found that *usually* the last entry in the "changelog" will provide the name/email addr of the packager.
Which are, of course, instances of general questions about the way openSUSE want's to handle external contributions:
a) What happens to contributions if a maintainer is so overburdened he or she doesn't even have the capacity to approve contributions that are otherwise ready for inclusion?
Request/assume those duties, fork, or do without.
b) What happens to contributions if a maintainer leaves?
Someone else picks it up or it dies.
c) How can contributors find their responsible maintainer (or absence of same)?
rpm -qi <package-name> or read the latest entry in the changelog -- Patrick Shanahan Plainfield, Indiana, USA HOG # US1244711 http://wahoo.no-ip.org Photo Album: http://wahoo.no-ip.org/gallery2 Registered Linux User #207535 @ http://counter.li.org -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Am 12.04.2009 15:05 schrieb Patrick Shanahan:
* Tilman Schmidt <tilman@imap.cc> [04-12-09 06:50]:
Ok, that splits my question in three:
a) How do I get my patch included if he is still maintaining it?
Send it to the packager.
When I started trying to help improving (Open)SUSE, the advice I got was that the best way to submit a patch was to open a Bugzilla entry describing the problem the patch was going to solve, and attach the proposed patch to it. Is that no longer true, and is it now acceptable to send personal mail to the packager of the package the file to be patched happens to be distributed in? Also, won't the packager ask for approval from the maintainer(s) of the affected subsystem(s)?
b) How do I get my patch included if he isn't maintaining it anymore?
Build the package yourself, or beg that someone else will pick it up.
ts@xenon:~> rpm -qf /usr/share/hwinfo/ISDN.CDB.* hwinfo-13.57-2 hwinfo-13.57-2 How can I build the standard openSUSE package hwinfo myself and get my version included instead of the current one?
c) How do I find out which of the two is the case?
I have found that *usually* the last entry in the "changelog" will provide the name/email addr of the packager.
ts@xenon:~> rpm -q --changelog hwinfo | head * Di Sep 25 2007 - snwint@suse.de - braille: fix el2d detection; wait longer in ht code - fix mv643xx detection (#300613) * Mo Sep 24 2007 - snwint@suse.de - updated X11 data (#326503) * Mo Sep 24 2007 - snwint@suse.de - updated X11 data (#327568) Doesn't look very promising. Name there ain't, and the E-mail address seems to bounce.
Which are, of course, instances of general questions about the way openSUSE want's to handle external contributions:
a) What happens to contributions if a maintainer is so overburdened he or she doesn't even have the capacity to approve contributions that are otherwise ready for inclusion?
Request/assume those duties,
How?
fork,
How?
or do without.
Ok, I guess that's the only option. Meaning, in this case: do not try to help openSUSE getting that problem fixed anymore.
b) What happens to contributions if a maintainer leaves?
Someone else picks it up or it dies.
Ok. So ISDN in openSUSE dies. SuSE was once the distribution of choice for ISDN users. I guess that time is over.
c) How can contributors find their responsible maintainer (or absence of same)?
rpm -qi <package-name> or read the latest entry in the changelog
See above. Also note that packager != maintainer. -- Tilman Schmidt E-Mail: tilman@imap.cc Bonn, Germany Diese Nachricht besteht zu 100% aus wiederverwerteten Bits. Ungeöffnet mindestens haltbar bis: (siehe Rückseite)
Hi, Il giorno lun, 13/04/2009 alle 23.25 +0200, Tilman Schmidt ha scritto:
When I started trying to help improving (Open)SUSE, the advice I got was that the best way to submit a patch was to open a Bugzilla entry describing the problem the patch was going to solve, and attach the proposed patch to it. Is that no longer true, and is it now acceptable to send personal mail to the packager of the package the file to be patched happens to be distributed in?
I think opening a bugreport and proposing a patch is still the way to go. But of course someone at openSUSE/Novell should confirm. Maybe we (community) can write a sort of how-to on the wiki to explain how to submit a patch, if not there already once a procedure is defined.
How can I build the standard openSUSE package hwinfo myself and get my version included instead of the current one?
To have your patch in the official package, it must be approved by some "insider", of course.
Ok. So ISDN in openSUSE dies. SuSE was once the distribution of choice for ISDN users. I guess that time is over.
I would say there is a lower number of users, so probably priorities changed. However, I agree with you it should not be so hard to get a working patch included. P.S. We are working on the creation of a community testing team (see thread on this ML). Why don't you take part to it? There will be a discussion to set it up during the openSUSE community week. Best, A. -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Mandag den 13. april 2009 23:25:47 skrev Tilman Schmidt:
When I started trying to help improving (Open)SUSE, the advice I got was that the best way to submit a patch was to open a Bugzilla entry describing the problem the patch was going to solve, and attach the proposed patch to it. Is that no longer true, and is it now acceptable to send personal mail to the packager of the package the file to be patched happens to be distributed in?
It's true if the patch is for openSUSE maintained code or packaging. But if the patch is for upstream software, clearly the patch should be sent upstream and not to bnc. -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Martin Schlander schrieb:
Mandag den 13. april 2009 23:25:47 skrev Tilman Schmidt:
When I started trying to help improving (Open)SUSE, the advice I got was that the best way to submit a patch was to open a Bugzilla entry describing the problem the patch was going to solve, and attach the proposed patch to it. Is that no longer true, and is it now acceptable to send personal mail to the packager of the package the file to be patched happens to be distributed in?
It's true if the patch is for openSUSE maintained code or packaging.
But if the patch is for upstream software, clearly the patch should be sent upstream and not to bnc.
AFAICT neither YaST (the software exposing the deficiency) nor hwinfo (the package containing the file that needs to be patched) are upstream software in that sense. But hwinfo doesn't seem to be openSUSE maintained either. So apparently there's a third category. How should that be handled? Thanks, Tilman PS: What is "bnc"? -- Tilman Schmidt E-Mail: tilman@imap.cc Bonn, Germany Diese Nachricht besteht zu 100% aus wiederverwerteten Bits. Ungeöffnet mindestens haltbar bis: (siehe Rückseite)
Tirsdag den 14. april 2009 13:34:20 skrev Tilman Schmidt:
Martin Schlander schrieb:
Mandag den 13. april 2009 23:25:47 skrev Tilman Schmidt:
When I started trying to help improving (Open)SUSE, the advice I got was that the best way to submit a patch was to open a Bugzilla entry describing the problem the patch was going to solve, and attach the proposed patch to it. Is that no longer true, and is it now acceptable to send personal mail to the packager of the package the file to be patched happens to be distributed in?
It's true if the patch is for openSUSE maintained code or packaging.
But if the patch is for upstream software, clearly the patch should be sent upstream and not to bnc.
AFAICT neither YaST (the software exposing the deficiency) nor hwinfo (the package containing the file that needs to be patched) are upstream software in that sense. But hwinfo doesn't seem to be openSUSE maintained either. So apparently there's a third category. How should that be handled?
If something is neither maintained by suse nor someone upstream, then I guess the only option left is that it's unmaintained. In that case I guess you have nowhere to send a patch until a new maintainer is found. Note that I do not know if this is true for hwinfo. Of course there are no rules without exceptions and it could happen that something was considered important enough to be patched by opensuse, but in general upstream is the place to go.
PS: What is "bnc"?
Bugzilla.novell.com -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
I think we have to analyze why many people don't have keen interest to testing
real tesing have to be done on a day to day use (apart from install stuff). So, IMHO, the tesing should divided in steps: * first step: install. use the basic dvd and network, etc... only with the basic config. If it installs: pass, if it don't->bugzilla. We are just now on this side with 10.2 * make a live cd with just this basic software and possibly all the hardware modules available. Testing all network and wlan is extremely important * make a general call on opensuse list: try to install the basis, try the live cd and report pass/don't pass with the hardware info (very important) - this is fast and easy, so many people is able to do * try to install and use most usually used aps. For me it's Mozilla and Thunderbird, then digikam. With this, I begin to be able to work with the new install. then go on with more apps, of course. Once a module or an app is validated by a sufficient number of users, slowdown any new feature on it. you can without doubt (you, any reader here) add some important apps jdd -- http://www.dodin.net http://valerie.dodin.org http://www.youtube.com/watch?v=t-eic8MSSfM http://www.facebook.com/profile.php?id=1412160445 -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Il giorno gio, 19/03/2009 alle 11.32 +0200, Tilman Schmidt ha scritto:
This particular call for testing had two reasons why I didn't consider participating:
a) It called for testing new features, but I am much more interested, and consider it much more important, to make sure the old features still work correctly, ie. there are no regressions.
Well, nobody said to test ONLY new features in that call for testing. I think also reports on regressions are welcome!
b) "Basic reqirements: * install openSUSE 11.1 beta 4 * meet in #opensuse-testing on irc.freenode.org" I do not use IRC.
You can use this ML, who is in the IRC channel is subscribed here too usually. I don't personally like ML, but I hate more to have the information spread on a lot of media, so a bit of flexibility doesn't hurt either. ;-)
Apart from that, I must admit that my testing enthusiasm has been somewhat dampened by a recent series of WONTFIX resolutions for problems I reported, a few of them with closing remarks that seemed to imply what I reported was "my problem". So I guess positive feedback from developers that test reports are appreciated, taken seriously, and the reported issues addressed, would go a long way in encouraging more testers.
I agree on this and you are not alone with these feelings. I would call that one of the "risks of the job" though. Regards, A. -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Given that SLE is out now, and the environment should be a bit more relaxed, I hope, it seems time for an update on this. @Zonker: did you talk with Oliver Ries? What did he say? Is there any actual and practical interest in doing this? Thanks, A. -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Hi Alberto,
some time ago I proposed the creation of a community based testing team for openSUSE pre-releases. I did not know of the existance of some infrastructure already in place, line the #opensuse-testing IRC channel and the opensuse-testing@opensuse.org mailing list, as I did not know of the work done by Holgie, daemon and their friends!
Anyway, after some discussion with Zonker, I have been encouraged to move the topic here officially. I'm sorry for the repetitions that happened in different mailing lists. I'll try to do better next time!
The main idea was to create a group of stable testers to deeply test openSUSE functionalities on their machines, in order to catch annoying bugs earlier in the development stages, allowing the fixes to be done and tested again in a more timely and less rushed manner.
It won't be easy, we need motivated volunteers and we have a lot to learn to do it the right way, but I think it is worth the effort, if we want to keep faith to the guiding principles, which states openSUSE aims to be "the most usable Linux distribution" (and not the most cutting edge one!).
I summed up the motivations, a short problem analysis about the current status quo, and some proposal of solution in some slides you can download from here:
http://en.opensuse.org/Testing_Team
Any suggestion on how to start is more than welcome!
I was following this thread quite interested and my summary of it is what I did find out before also: It seem to be quite hard to find people that are really interested in testing openSUSE (by various reasons). So I really like your idea of creating such a testing team and would be glad if you/we do succeed with it. I think it would be worth to spend efforts into such a task and I'm willing to help. If there is anything I can do in enabling this team just let me know. You will get my full support. Best wishes, Holgi -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Il giorno mar, 07/04/2009 alle 17.50 +0200, Holger Sickenberg ha scritto:
I was following this thread quite interested and my summary of it is what I did find out before also:
It seem to be quite hard to find people that are really interested in testing openSUSE (by various reasons).
So I really like your idea of creating such a testing team and would be glad if you/we do succeed with it. I think it would be worth to spend efforts into such a task and I'm willing to help. If there is anything I can do in enabling this team just let me know. You will get my full support.
Hi Holeger, sorry for my late answer! Thanks for the encouragement and for the interest in my idea. Indeed it seems hard to attract people to do that! Anyway, I still have some hope: I found some volunteer from the local Italian community (suseitalia.org) and they seem quite motivated to help. Anyway, we should really discuss on how to proceed in setting things up. Ideally it would be nice to start with alphas, but being the first time we try to do something like this, anytime is OK. At the moment I don't understand if there is interest for this idea on the Novell side though (Are you at N, btw?). I just had some quick talk with Zonker, who showed his interest, and who said he would have contacted Oliver Ries at suse.de to know what the existing infrastructure is. Still no answer. If you know something about that, it would be useful to talk about it too. :-) Let's talk soon, when you have time. Best, Alberto -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Hi Alberto,
Thanks for the encouragement and for the interest in my idea. Indeed it seems hard to attract people to do that! Anyway, I still have some hope: I found some volunteer from the local Italian community (suseitalia.org) and they seem quite motivated to help.
That sounds like a good starting point.
Anyway, we should really discuss on how to proceed in setting things up. Ideally it would be nice to start with alphas, but being the first time we try to do something like this, anytime is OK.
As far as I know there will not be any alpha and beta versions but something like milestones with 11.2 - so yes: starting to test with the first milestone make sense.
At the moment I don't understand if there is interest for this idea on the Novell side though (Are you at N, btw?). I just had some quick talk with Zonker, who showed his interest, and who said he would have contacted Oliver Ries at suse.de to know what the existing infrastructure is. Still no answer. If you know something about that, it would be useful to talk about it too. :-)
Let's talk soon, when you have time.
There is interest by Novell: Zonker already told you about that. Yes, I'm also working for SUSE/N (Oliver is my boss) and I'm quite interested as well. Which information do you like to have? I volunteered to take over the action item of the Testing/QA section of the CommunityWeek (http://en.opensuse.org/CommunityWeek). So maybe we can use that as kind of kick-off for the testing team. Best wishes, Holgi PS: I will be on vacation the next couple of days but should read my e-mails anyway. Maybe some answers will last a little bit longer. -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Hello, On Thu, Apr 09, 2009 at 08:47:37AM +0200, Holger Sickenberg wrote:
Anyway, we should really discuss on how to proceed in setting things up. Ideally it would be nice to start with alphas, but being the first time we try to do something like this, anytime is OK.
As far as I know there will not be any alpha and beta versions but something like milestones with 11.2 - so yes: starting to test with the first milestone make sense.
one of the main problems is (excuse my french) the management of it. As in what needs to be tested, how it should be tested or how it was tested, what has already been tested, how can we organize who tests what, how do test results make it back into Suse? How is the time line for opening Testopia to the outside, perhaps (initially) only for a selected "openSUSE test group"? Then we would have a test plan, would be able to avoid unneccessary double testing, and results would automatically find their way back to Suse QA.
I volunteered to take over the action item of the Testing/QA section of the CommunityWeek (http://en.opensuse.org/CommunityWeek). So maybe we can use that as kind of kick-off for the testing team.
That's a good idea. I'm particularly interested in the questions above, the organisation. Just doing isolated, random testing is not going to be very efficient. Regards, Rasmus -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
* Rasmus Plewe <rasmus@lst.de> [04-09-09 04:55]:
How is the time line for opening Testopia to the outside, perhaps (initially) only for a selected "openSUSE test group"? Then we would have a test plan, would be able to avoid unneccessary double testing, and results would automatically find their way back to Suse QA.
Maybe that should read: Opening TestSUSE ?? -- Patrick Shanahan Plainfield, Indiana, USA HOG # US1244711 http://wahoo.no-ip.org Photo Album: http://wahoo.no-ip.org/gallery2 Registered Linux User #207535 @ http://counter.li.org -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
On Thu, Apr 09, 2009 at 09:25:36AM -0400, Patrick Shanahan wrote:
* Rasmus Plewe <rasmus@lst.de> [04-09-09 04:55]:
How is the time line for opening Testopia to the outside, perhaps (initially) only for a selected "openSUSE test group"? Then we would have a test plan, would be able to avoid unneccessary double testing, and results would automatically find their way back to Suse QA.
Maybe that should read: Opening TestSUSE
The name of the tool used internally is Testopia, not TestSUSE. Regards, Rasmus -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
* Rasmus Plewe <rasmus@lst.de> [04-09-09 14:36]:
On Thu, Apr 09, 2009 at 09:25:36AM -0400, Patrick Shanahan wrote:
Maybe that should read: Opening TestSUSE
The name of the tool used internally is Testopia, not TestSUSE.
I understand that, but maybe it shouldn't be, afterall it is for openSUSE. A little brand recognition :^) -- Patrick Shanahan Plainfield, Indiana, USA HOG # US1244711 http://wahoo.no-ip.org Photo Album: http://wahoo.no-ip.org/gallery2 Registered Linux User #207535 @ http://counter.li.org -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Hi, maybe we should first set the team up, and then worry about how tools are called? ;-) Regards, A. Il giorno gio, 09/04/2009 alle 16.09 -0400, Patrick Shanahan ha scritto:
I understand that, but maybe it shouldn't be, afterall it is for openSUSE. A little brand recognition :^)
-- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Il giorno gio, 09/04/2009 alle 10.53 +0200, Rasmus Plewe ha scritto:
one of the main problems is (excuse my french) the management of it. As in what needs to be tested, how it should be tested or how it was tested, what has already been tested, how can we organize who tests what, how do test results make it back into Suse?
How is the time line for opening Testopia to the outside, perhaps (initially) only for a selected "openSUSE test group"? Then we would have a test plan, would be able to avoid unneccessary double testing, and results would automatically find their way back to Suse QA.
Yes. I agree. And I also like the idea of starting gradually, to see how it works and how many people takes part to this. Anyway, we can try to decide this during the community week. Best, Alberto -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Hi Holger, thanks for you reply! Il giorno gio, 09/04/2009 alle 08.47 +0200, Holger Sickenberg ha scritto:
As far as I know there will not be any alpha and beta versions but something like milestones with 11.2 - so yes: starting to test with the first milestone make sense.
OK.
At the moment I don't understand if there is interest for this idea on the Novell side though (Are you at N, btw?). I just had some quick talk with Zonker, who showed his interest, and who said he would have contacted Oliver Ries at suse.de to know what the existing infrastructure is. Still no answer. If you know something about that, it would be useful to talk about it too. :-)
Let's talk soon, when you have time.
There is interest by Novell: Zonker already told you about that. Yes, I'm also working for SUSE/N (Oliver is my boss) and I'm quite interested as well. Which information do you like to have?
Yup, Henne clarified the situation to me yesterday. Mainly I was interested to know what tools are available and some general idea of how testing works at SUSE.
I volunteered to take over the action item of the Testing/QA section of the CommunityWeek (http://en.opensuse.org/CommunityWeek). So maybe we can use that as kind of kick-off for the testing team.
Yeah. This sounds a good idea. I'm at work during that week, however it should not be overbusy, and I can find some time.
PS: I will be on vacation the next couple of days but should read my e-mails anyway. Maybe some answers will last a little bit longer.
No prob! Enjoy your vacation :-) Best, Alberto -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Hi Alberto,
As far as I know there will not be any alpha and beta versions but something like milestones with 11.2 - so yes: starting to test with the first milestone make sense.
OK.
It looks like the first milestone should be available around April, 23rd which is pretty soon. (see Coolo's mail: http://lists.opensuse.org/archive/opensuse-factory/2009-04/msg00052.html)
At the moment I don't understand if there is interest for this idea on the Novell side though (Are you at N, btw?). I just had some quick talk with Zonker, who showed his interest, and who said he would have contacted Oliver Ries at suse.de to know what the existing infrastructure is. Still no answer. If you know something about that, it would be useful to talk about it too. :-)
Let's talk soon, when you have time. There is interest by Novell: Zonker already told you about that. Yes, I'm also working for SUSE/N (Oliver is my boss) and I'm quite interested as well. Which information do you like to have?
Yup, Henne clarified the situation to me yesterday. Mainly I was interested to know what tools are available and some general idea of how testing works at SUSE.
Do you have that already or do you want me to tell something about the current way of testing at SUSE?
I volunteered to take over the action item of the Testing/QA section of the CommunityWeek (http://en.opensuse.org/CommunityWeek). So maybe we can use that as kind of kick-off for the testing team.
Yeah. This sounds a good idea. I'm at work during that week, however it should not be overbusy, and I can find some time.
Feel free to add your thoughts and ideas to the Testing/QA section of the wiki page mentioned above. Best wishes, Holgi -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Hi Holger, Il giorno ven, 10/04/2009 alle 11.20 +0200, Holger Sickenberg ha scritto:
Do you have that already or do you want me to tell something about the current way of testing at SUSE?
We did not talk about the current way of testing with Henne, so it would be great if you could tell me something :-)
Feel free to add your thoughts and ideas to the Testing/QA section of the wiki page mentioned above.
OK. Thanks, A. -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Hi Alberto,
Do you have that already or do you want me to tell something about the current way of testing at SUSE?
We did not talk about the current way of testing with Henne, so it would be great if you could tell me something :-)
Going not too much into details: current testing of openSUSE is divided in three parts: 1)Installation/Update Tests Test installing and updating from the former openSUSE version (for 11.1 it was 11.0) of every milestone (beta) version both on x86 and x86_64. (If time permits also installation tests on PlayStation 3) 2)Automated Kernel Test Running a subset of available Kernel Test Suites on every milestone version (x86/x86_64) 3)Feature Tests Based on the list of new features test if features are implemented and working as expected (see http://en.opensuse.org/Testing:Features_11.1) All test cases and results are stored in the tool "Testopia" (an enhancement to Bugzilla) which is unfortunately just accessable for people inside Novell. With the help of the openSUSE community (developers/testing team) I do see room for improvement for milestone testing in following areas: - test of installation/updates on a wider hardware base - better test coverage of feature tests - creating and executing of test cases for base system and applications (also useful for maintenance updates of packages during product lifetime) I hope this information is a good base for further discussions. Best wishes, Holgi -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Holger Sickenberg a écrit :
1)Installation/Update Tests Test installing and updating from the former openSUSE version (for 11.1 it was 11.0)
testing the install is pretty easy (need only to keep a partition free), testing *update* seems to me much more difficult, at least it would need some sort of HOWTO: howto backup previous install, just in case things go wrong or to be able to do it again (g4l?) Install should have a list of all the kind of hardware needing detailed tests use of automated HW list could allow to ask one defined tester who have one particular HW :-) all this seems mostly kernel directed? couldn't we also use the other distribution bugzilla to know what is wrong or share infos on a given kernel?
All test cases and results are stored in the tool "Testopia" (an enhancement to Bugzilla) which is unfortunately just accessable for people inside Novell.
this is odd and should be the first priority (at least open to test team or members)
- test of installation/updates on a wider hardware base - better test coverage of feature tests - creating and executing of test cases for base system and applications
true thanks jdd -- http://www.dodin.net http://valerie.dodin.org http://www.youtube.com/watch?v=t-eic8MSSfM http://www.facebook.com/profile.php?id=1412160445 -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Hi Holger, thanks for all the info. They're very useful to understand the process. Il giorno sab, 11/04/2009 alle 16.40 +0200, Holger Sickenberg ha scritto:
With the help of the openSUSE community (developers/testing team) I do see room for improvement for milestone testing in following areas: - test of installation/updates on a wider hardware base - better test coverage of feature tests - creating and executing of test cases for base system and applications (also useful for maintenance updates of packages during product lifetime)
Yes. I agree. Best regards, Alberto -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
Holger Sickenberg wrote:
Hi Alberto,
Hi guys, my name is Christian Hueller, i also work in the QA department at Novell/SUSE and also would like to offer help and information to a community quality group.
Do you have that already or do you want me to tell something about the current way of testing at SUSE?
We did not talk about the current way of testing with Henne, so it would be great if you could tell me something :-)
Going not too much into details: current testing of openSUSE is divided in three parts: 1)Installation/Update Tests Test installing and updating from the former openSUSE version (for 11.1 it was 11.0) of every milestone (beta) version both on x86 and x86_64. (If time permits also installation tests on PlayStation 3) 2)Automated Kernel Test Running a subset of available Kernel Test Suites on every milestone version (x86/x86_64) 3)Feature Tests Based on the list of new features test if features are implemented and working as expected (see http://en.opensuse.org/Testing:Features_11.1)
All test cases and results are stored in the tool "Testopia" (an enhancement to Bugzilla) which is unfortunately just accessable for people inside Novell.
We are working on opening it asap, however this might still be some time into the future. In case somebody is interested in the tool itself, one can find it's project pages here: http://www.mozilla.org/projects/testopia/.
With the help of the openSUSE community (developers/testing team) I do see room for improvement for milestone testing in following areas: - test of installation/updates on a wider hardware base - better test coverage of feature tests - creating and executing of test cases for base system and applications (also useful for maintenance updates of packages during product lifetime)
I hope this information is a good base for further discussions.
Let me add some more bits, especially areas where we are still expanding our efforts. o Benchmarking is becoming a real big item in QA. Kernel performance benchmarks showed some performance degradation in recent kernel releases which pointed to on the one hand still insufficient benchmarks and the need for some process to check for regression on the other hand. o Automated testing and coverage (outside the kernel) - currently mostly CLI based applications - next big item: GUI testing, since QT4 is used, LDTP (http://ldtp.freedesktop.org/wiki/) is becoming an option * Related to both above: Open Source Test Suites. There are plenty available, however reviewing and adopting them eats up time, so this also is an ongoing process. Here I see a lot of gain from more people simply looking and talking about them in a public discussion. o Documentation. It is there, though it could be better - as always ;) o much more I'm just not thinking of right now. There are three pages on opensuse.org right now (maybe more I'm not aware of), which I think can cause confusion or lead to missing information. I see these currently: http://en.opensuse.org/QA -> Redirects to http://en.opensuse.org/QA_Team, an Overview of QA personnel at Novell/SUSE http://opensuse.org/QA -> Empty page http://en.opensuse.org/Testing -> Testing page created around 11.1, mostly of 11.1 related test material My idea would be to use http://opensuse.org/QA as kind of entry point and then link to all other QA/Testing related pages from there, but thats just an suggestion, I do not want to tell you what to do but mere try to support you in the decisions that you make and give input where I have it. I'll send out some presentations and documentation if it is up to date and valid, just to give an better impression what QA is doing in here. If you have questions, please ask, we'll try to help the best we can. Have fun and thanks, Chris
Best wishes, Holgi
-- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: opensuse-testing+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-testing+help@opensuse.org
participants (11)
-
Alberto Passalacqua -
Andreas Jaeger -
Christian Hueller -
Dave Plater -
Holger Sickenberg -
jdd -
Martin Schlander -
Patrick Shanahan -
Rasmus Plewe -
Satoru Matsumoto -
Tilman Schmidt