[opensuse-project] Proposal for a community based testing team
Hello, I have thought for quite some time to the idea of building a community-based openSUSE testing team, and this seems to be the right time to discuss of it. The main idea is to create a group of stable testers to deeply test openSUSE functionalities on their machines, in order to catch annoying bugs earlier in the development stages, allowing the fixes to be done and tested again in a more timely and less rushed manner. It won't be easy, we need motivated volunteers and we have a lot to learn to do it the right way, but I think it is worth the effort, if we want to keep faith to the guiding principles, which states openSUSE aims to be "the most usable Linux distribution" (and not the most cutting edge one!). I summed up the motivations, a short problem analysis about the current status quo, and some proposal of solution in the attached slides so you can comment them (thanks suseROCKs for the slide template). All suggestions (and volunteers) are more than welcome, as well as some help from Novell people currently involved in testing/quality assurance. As I said we need to learn. ;-) With kind regards, Alberto
Mandag 23 februar 2009 19:33:42 skrev Alberto Passalacqua:
I have thought for quite some time to the idea of building a community-based openSUSE testing team
I'm not sure an organized team is necessary. But after the 11.1 fiasco, I plan to apply more of a systematic and methodic approach to testing personally. Previously I would basically just test that installation and hardware support/detection worked on my machines - and not much more than that, taking for granted that basic everyday stuff would work and be tested by someone else. But now I've started to compile a list of things I consider critical and my intention is to test everything in that list everytime. Basic things like cd- burning, kbluetooth, 1-click installation, updater applet, printing, home bank, wireless, multimedia, docx-documents etc. Hopefully the recent steps taken to improve the factory experience will mean more and earlier testing. And hopefully for non-SLE-base releases there'll be less post-feature-freeze experimentation and feature development, than was the case for 11.1. -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Thanks for your reply Martin, which is for now the only one. ;-) Basically what you want to do is what each member of the "team" would have to do in my opinion. The other task for this group should be to collect information on HOW to do testing and reporting bugs correctly, which necessarily has to involve some Novell people at the beginning, so we can learn what they need and write it down in a non-technical form that can be understood by more users. Feel free to share your list ;-) Regards, A. Il giorno mar, 24/02/2009 alle 17.55 +0100, Martin Schlander ha scritto:
Mandag 23 februar 2009 19:33:42 skrev Alberto Passalacqua:
I have thought for quite some time to the idea of building a community-based openSUSE testing team
I'm not sure an organized team is necessary. But after the 11.1 fiasco, I plan to apply more of a systematic and methodic approach to testing personally.
Previously I would basically just test that installation and hardware support/detection worked on my machines - and not much more than that, taking for granted that basic everyday stuff would work and be tested by someone else.
But now I've started to compile a list of things I consider critical and my intention is to test everything in that list everytime. Basic things like cd- burning, kbluetooth, 1-click installation, updater applet, printing, home bank, wireless, multimedia, docx-documents etc.
Hopefully the recent steps taken to improve the factory experience will mean more and earlier testing. And hopefully for non-SLE-base releases there'll be less post-feature-freeze experimentation and feature development, than was the case for 11.1.
-- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Hello, On Tuesday 24 February 2009 18:36:44 A. Passalacqua wrote (shortened):
The other task for this group should be to collect information on HOW to do testing and reporting bugs correctly, which necessarily has to involve some Novell people at the beginning, so we can learn what they need and write it down in a non-technical form that can be understood by more users.
Guys ... Holgi did a great job during the 11.1 testing phase and if you missed it ... no problem! For the next release we will be more visible! Holgi and Martin also created a IRC channel to help you with the tasks but only a few joined! Since this channel has been created I have been there and I still keep it alive. So if you want to learn something about QA or testing software then join: #opensuse-testing Greetings, daemon -- zero -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
On Wednesday 25 February 2009 01:31:20 Marco Michna wrote:
Hello,
On Tuesday 24 February 2009 18:36:44 A. Passalacqua wrote (shortened):
The other task for this group should be to collect information on HOW to do testing and reporting bugs correctly, which necessarily has to involve some Novell people at the beginning, so we can learn what they need and write it down in a non-technical form that can be understood by more users.
Guys ... Holgi did a great job during the 11.1 testing phase and if you missed it ... no problem! For the next release we will be more visible!
Holgi and Martin also created a IRC channel to help you with the tasks but only a few joined! Since this channel has been created I have been there and I still keep it alive. So if you want to learn something about QA or testing software then join:
#opensuse-testing
And we have an opensuse-testing mailing list - but it only saw very little feedback. So, I applaud this effort, we need to work together to create some team of testers that coordinates testing - in such a way that we now in the end what has been tested and what not. That will allow us to say that areas are stable - or not, Andreas -- Andreas Jaeger, Director Platform / openSUSE, aj@suse.de SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) Maxfeldstr. 5, 90409 Nürnberg, Germany GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
Martin Schlander a écrit :
I'm not sure an organized team is necessary. But after the 11.1 fiasco, I plan to apply more of a systematic and methodic approach to testing personally.
mediawiki uses the last dev version on the main wikipedia web site. Sure, Mediawiki is not as complex as openSUSE, but this is a direction to use. Could it be possible to have asap a working distro (no blocking bugs for the test team computers) to use on a dayly basis? most bugs are unexpected ones :-). It should be possible to keep testing say, beta 2 even if factory is still going on, it's not possible to do the same tests again and again (if not automatized) jdd -- http://www.dodin.net http://valerie.dodin.org http://www.youtube.com/watch?v=t-eic8MSSfM http://www.facebook.com/profile.php?id=1412160445 -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
On Tue, 24 Feb 2009, jdd wrote:
Could it be possible to have asap a working distro (no blocking bugs for the test team computers) to use on a dayly basis? most bugs are unexpected ones :-).
What prevents you from using FACTORY? (Serious question. If there is anything, we should address it.) Gerald -- Dr. Gerald Pfeifer E gp@novell.com SUSE Linux Products GmbH Director Inbound Product Mgmt T +49(911)74053-0 HRB 16746 (AG Nuremberg) openSUSE/SUSE Linux Enterprise F +49(911)74053-483 GF: Markus Rex -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
On Wed, Feb 25, 2009 at 10:30 AM, Gerald Pfeifer <gp@novell.com> wrote:
On Tue, 24 Feb 2009, jdd wrote:
Could it be possible to have asap a working distro (no blocking bugs for the test team computers) to use on a dayly basis? most bugs are unexpected ones :-).
What prevents you from using FACTORY?
(Serious question. If there is anything, we should address it.)
Gerald
Yes, agreed.It needs to be easier. Easier to download. Easier to update to. Easier to do updates, and those updates need to be more reliable (in terms, of actually downloading). Then the system should never be completely broken, and nor packages that aren't READY for *testing* be pushed. (put them in their own repo, until it's deemed that they're fine, and the maintainers are looking for *testing*). In short, try run factory as your main distro for a month -- and the problems testers have should be pretty obvious =)
-- Dr. Gerald Pfeifer E gp@novell.com SUSE Linux Products GmbH Director Inbound Product Mgmt T +49(911)74053-0 HRB 16746 (AG Nuremberg) openSUSE/SUSE Linux Enterprise F +49(911)74053-483 GF: Markus Rex -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
-- Eric Springer, PGP Fingerprint: 097D E98D 9278 FE86 2659 2959 DA9E 90BD F183 2F88 -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Am Mittwoch 25 Februar 2009 schrieb Eric Springer:
On Wed, Feb 25, 2009 at 10:30 AM, Gerald Pfeifer <gp@novell.com> wrote:
On Tue, 24 Feb 2009, jdd wrote:
Could it be possible to have asap a working distro (no blocking bugs for the test team computers) to use on a dayly basis? most bugs are unexpected ones :-).
What prevents you from using FACTORY?
(Serious question. If there is anything, we should address it.)
Gerald
Yes, agreed.It needs to be easier. Easier to download. Easier to update to. Easier to do updates, and those updates need to be more reliable (in terms, of actually downloading). Then the system should never be completely broken, and nor packages that aren't READY for *testing* be pushed. (put them in their own repo, until it's deemed that they're fine, and the maintainers are looking for *testing*). The system is never completely broken. And if there is a bug affecting many, it's our highest priority to push a fixed factory.
But e.g. a new kernel always bears a risk, same for new libzypp. We have a test suite and it succeeeds. This is of course no guarantee that it won't crash left and right ;(
In short, try run factory as your main distro for a month -- and the problems testers have should be pretty obvious =)
I do. Since 10.3 Greetings, Stephan -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
On Wed, Feb 25, 2009 at 04:03:57PM +1000, Eric Springer wrote:
On Wed, Feb 25, 2009 at 10:30 AM, Gerald Pfeifer <gp@novell.com> wrote:
What prevents you from using FACTORY?
(Serious question. If there is anything, we should address it.)
Gerald
Yes, agreed.It needs to be easier. Easier to download. Easier to update to. Easier to do updates, and those updates need to be more reliable (in terms, of actually downloading).
We are about to see significant improvement here; with changes implemented during the last weeks, and just hitting Factory now, Factory should be improved in several regards: * smaller deltas = less to download, due to "buildcompare" mechanism (comparing built packages with previous versions for actual changes) http://news.opensuse.org/2009/02/05/more-efficient-factory-development/ * http://download.opensuse.org/factory/ keeps the previous package version available for a grace period. This gives ongoing updates/downloads a chance to complete successfully, even though Factory is being republished E.g. in http://download.opensuse.org/factory/repo/oss/suse/i586/ you'll note the resulting package duplication * new, radically improved download backend in zypper/YaST, http://en.opensuse.org/Libzypp/Failover http://duncan.mac-vicar.com/blog/archives/507 using aria2c and metalinks, which offers several helpful features: - without user interaction, it deals with mirrors that are incomplete, halfway updated or broken in some way, by simply switching to another one - it can download faster, using parallel connections to multiple mirrors and maximizing bandwidth utilization No more worries about the "best mirror" - error checking of downloads is already while in transit; not after downloading everything and then trying to use it. * mirror selection with stronger exploitation network locality If a client is in the same network as a mirror, it will automatically be assigned to that one mirror (not just looking on country level): http://mirrorbrain.org/news_items/2.6_network_topological_mirror_selection A further, planned, change is to increase world-wide availability of Factory by tighter integrating it publishing and its mirror syncing. This altogether should make Factory tracking a pleasant experience. At least the downloading part, which is the first step to using it ;)
[...] Then the system should never be completely broken, and nor packages that aren't READY for *testing* be pushed. (put them in their own repo, until it's deemed that they're fine, and the maintainers are looking for *testing*).
In short, try run factory as your main distro for a month -- and the problems testers have should be pretty obvious =)
Amen. Peter -- Contact: admin@opensuse.org (a.k.a. ftpadmin@suse.com) #opensuse-mirrors on freenode.net Info: http://en.opensuse.org/Mirror_Infrastructure SUSE LINUX Products GmbH Research & Development -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Le mercredi 25 février 2009, à 16:52 +0100, Peter Poeml a écrit :
We are about to see significant improvement here; with changes implemented during the last weeks, and just hitting Factory now, Factory should be improved in several regards:
Just wondering out loud: there was also some discussion around using deltarpm by default, I believe. Any progress there? Vincent -- Les gens heureux ne sont pas pressés. -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
by the way, searching to dl the factory, I found only: Download openSUSE 11.1-RC1 so I will have to tweak to install factory? jdd -- http://www.dodin.net http://valerie.dodin.org http://www.youtube.com/watch?v=t-eic8MSSfM http://www.facebook.com/profile.php?id=1412160445 -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
jdd a écrit :
by the way, searching to dl the factory, I found only:
Download openSUSE 11.1-RC1
so I will have to tweak to install factory?
jdd
hope it's this one: http://download.opensuse.org/factory/iso/ warning: I don't write this to have help (I don't really need it :-), but to show the community based test system is broken -- http://www.dodin.net http://valerie.dodin.org http://www.youtube.com/watch?v=t-eic8MSSfM http://www.facebook.com/profile.php?id=1412160445 -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
jdd a écrit :
by the way, searching to dl the factory, I found only:
Download openSUSE 11.1-RC1
so I will have to tweak to install factory?
jdd and, by the way, the factory list is well hidden deep into the wiki.
If I remember, some month ago, it was very easy to go to testing from the wiki main page. No more. the faster link seems to be at the end of http://en.opensuse.org/Factory_Distribution, under the "bug report" section. jdd -- http://www.dodin.net http://valerie.dodin.org http://www.youtube.com/watch?v=t-eic8MSSfM http://www.facebook.com/profile.php?id=1412160445 -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
On Wed, Feb 25, 2009 at 05:33:38PM +0100, jdd wrote:
jdd a écrit :
by the way, searching to dl the factory, I found only:
Download openSUSE 11.1-RC1
so I will have to tweak to install factory?
jdd and, by the way, the factory list is well hidden deep into the wiki.
If I remember, some month ago, it was very easy to go to testing from the wiki main page. No more.
the faster link seems to be at the end of http://en.opensuse.org/Factory_Distribution, under the "bug report" section.
Yea, there are way to many "download pages" in the wiki. Some consolidation is needed. Anyway, the http://software.opensuse.org/developer page has been fixed recently, and no longer points to an old RC, it correctly sends to the factory iso directory on the download server now. Thanks, Peter -- "WARNING: This bug is visible to non-employees. Please be respectful!" SUSE LINUX Products GmbH Research & Development
trying to go ahead, don't you think the Virtualbox install should be rocksolid? this one anybody can test it and use it (may be it is already, but this was not the case in the past) jdd -- http://www.dodin.net http://valerie.dodin.org http://www.youtube.com/watch?v=t-eic8MSSfM http://www.facebook.com/profile.php?id=1412160445 -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
On Wed, 25 Feb 2009, Eric Springer wrote:
In short, try run factory as your main distro for a month -- and the problems testers have should be pretty obvious =)
I've been doing this for half of last year :-), including full distribution upgrades in hotel rooms overnight, and in general I found this surprisingly stable. For the last few weeks things have been somewhat, hmm, rough, though. That's something we need to get under control, I agree. As for your comments, I wonder whether we can make this somewhat more concrete, actionable?
Easier to download.
What is difficult on this front with current FACTORY? Any suggestions what you'd like to see changed? We now got smaller FACTORY updates, which is really, really nice.
Easier to update to. Easier to do updates
Personally, I'm finding "zypper up" and to a somewhat less degree "zypper dup" pretty easy. What are you looking for? A nicer GUI than we currently offer?
and those updates need to be more reliable (in terms, of actually downloading). Then the system should never be completely broken, and nor packages that aren't READY for *testing* be pushed.
That is a key item, absolutely. :-) Gerald -- Dr. Gerald Pfeifer E gp@novell.com SUSE Linux Products GmbH Director Inbound Product Mgmt T +49(911)74053-0 HRB 16746 (AG Nuremberg) openSUSE/SUSE Linux Enterprise F +49(911)74053-483 GF: Markus Rex -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Gerald Pfeifer a écrit :
On Wed, 25 Feb 2009, Eric Springer wrote:
In short, try run factory as your main distro for a month -- and the problems testers have should be pretty obvious =)
I've been doing this for half of last year :-), including full distribution upgrades in hotel rooms overnight, and in general I found this surprisingly stable.
is that possible right now? I failed to install factory from network with the factory mini cd right now (on virtualbox) jdd -- http://www.dodin.net http://valerie.dodin.org http://www.youtube.com/watch?v=t-eic8MSSfM http://www.facebook.com/profile.php?id=1412160445 -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
On Wednesday 25 February 2009 22:22:18 jdd wrote:
Gerald Pfeifer a écrit :
On Wed, 25 Feb 2009, Eric Springer wrote:
In short, try run factory as your main distro for a month -- and the problems testers have should be pretty obvious =)
I've been doing this for half of last year :-), including full distribution upgrades in hotel rooms overnight, and in general I found this surprisingly stable.
is that possible right now? I failed to install factory from network with the factory mini cd right now (on virtualbox)
I think this is exactly the difference between the two attempts of installing. As far as I understand Gerald (and this is exactly what I'm doing), we are running factory all the time. Which means in detail: we installed at some point in time (for example the machine I'm writing this on saw the last install somewhen around the 11.0 release) and since then I usually run either "zypper up" or "zypper dup" roughly twice a week to keep up. If this fails, or the result sounds too scary (like zypper dup trying to change the architecture of 100+ packages), I call up "yast2 sw_single" select the repositories, a package, context menu "all in this list" and then "update if newer version available" and look at the problems. This is a completely different issue than trying to install a system or image from scratch. Any severe bug in either the instsys or any critical yast2 configuration module can block the latter case, while you would not even see it in the former case. One is testing the system as such (my most frequented apps still run, my desktop still starts up during login, and so on) versus tesing the installation workflow and getting a running system. Both methods are very valuable, but substantially different and I can easily see the "install from scratch" method being broken much more frequently. For example if a kernel is updated and some out-of-tree module used on my machine does not build any longer, then the "update-the-system" variant will most likely end up not updating the kernel at all and keep that for the next time when that package (the foo-kmp) has been fixed. The "install-from-scratch" variant would end up with some hardware not working at all. The same is probably true for other subtrees of packages where one library was updated with a set of packages failing to build because of that and still requiring the old library name. (not trying to excuse any failure, just trying to explain here ...) -- with kind regards (mit freundlichem Grinsen), Ruediger Oertel (ro@novell.com,ro@suse.de,bugfinder@t-online.de) ---------------------------------------------------------------------- Linux MacBookRudi 2.6.29-rc5-1-default #1 SMP 2009-02-16 17:18:44 +0100 x86_64 Key fingerprint = 17DC 6553 86A7 384B 53C5 CA5C 3CE4 F2E7 23F2 B417 SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Ruediger Oertel a écrit :
is that possible right now? I failed to install factory from network with the factory mini cd right now (on virtualbox)
I think this is exactly the difference between the two attempts of installing. As far as I understand Gerald (and this is exactly what I'm doing), we are running factory all the time.
first: is it here (project) the place to have this discussion or is it better in factory? having an installable factory well documented in the wiki seems the mandatory thing to have. I don't understand why the install don't work on vortualbox (afaik anybody have the same hardware, there). This should be the bare minimum, no?? jdd NB: I installed via the 11.1 mini cd the minimal 11.1, modify the source to factory and zypper up. all is well, but I don't have yet tried ti boot :-( - I install kde4 before. is there any packman factory?? -- http://www.dodin.net http://valerie.dodin.org http://www.youtube.com/watch?v=t-eic8MSSfM http://www.facebook.com/profile.php?id=1412160445 -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 jdd wrote: [...]
is there any packman factory??
No, at Packman we don't build against Factory. We believe that it changes too often for that (which might be wrong, I don't think that any of us follows Factory that closely), and we don't have unlimited bandwidth nor enough build power to rebuild large parts of our repository every few days. It would also be very tricky to handle as we don't use OBS but trigger rebuilds manually instead (which is a feature when you have a single build server instead of a farm). cheers - -- -o) Pascal Bleser <pascal.bleser@opensuse.org> /\\ http://opensuse.org -- I took the green pill _\_v FOSDEM::7+8 Feb 2009, Brussels, http://fosdem.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iD8DBQFJqC4Fr3NMWliFcXcRAsbSAJ41Gl5gRGkYJ4LGEITv4fB62qRXMwCgsgae Ogvkf6Klr+iaX/PWx4l9ru4= =qPTV -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Pascal Bleser a écrit :
jdd wrote: [...]
is there any packman factory??
No, at Packman we don't build against Factory.
I asked this because testing need day to day use and that pacman is nearly mandatory for any uses :-) (thanks for that :-) But given the kind of apps pacman have, is this build necessary, or wil the ordinary 11.1 work (mostly) with factory tree?? thanks jdd NB: essentially libdvdcss and lame mp3? -- http://www.dodin.net http://valerie.dodin.org http://www.youtube.com/watch?v=t-eic8MSSfM http://www.facebook.com/profile.php?id=1412160445 -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 jdd wrote:
Pascal Bleser a écrit :
jdd wrote: [...]
is there any packman factory?? No, at Packman we don't build against Factory.
I asked this because testing need day to day use and that pacman is nearly mandatory for any uses :-) (thanks for that :-)
I don't know. For me, factory is _not_ for daily use, it's for testing. I really don't see how we (packman) could keep up with the frequent changes in factory.
But given the kind of apps pacman have, is this build necessary, or will the ordinary 11.1 work (mostly) with factory tree??
I have no idea. Could work. Depends on the versions of packages in factory I guess.
NB: essentially libdvdcss and lame mp3?
Mind you, we don't and we won't provide libdvdcss. - -- -o) Pascal Bleser <pascal.bleser@opensuse.org> /\\ http://opensuse.org -- I took the green pill _\_v FOSDEM::7+8 Feb 2009, Brussels, http://fosdem.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iD8DBQFJqzQhr3NMWliFcXcRAgGOAKCykXt1R1GOYSS9NEqERDHFrFnh0gCdHw9M Egg23l/RJpcr3x3xTFeta8k= =8A4z -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Am Mittwoch 25 Februar 2009 schrieb jdd:
Gerald Pfeifer a écrit :
On Wed, 25 Feb 2009, Eric Springer wrote:
In short, try run factory as your main distro for a month -- and the problems testers have should be pretty obvious =)
I've been doing this for half of last year :-), including full distribution upgrades in hotel rooms overnight, and in general I found this surprisingly stable.
is that possible right now? I failed to install factory from network with the factory mini cd right now (on virtualbox)
Do read my mails - factory isos are broken. A new kernel always changes things ;( Greetings, Stephan -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Stephan Kulow a écrit :
Do read my mails - factory isos are broken. A new kernel always changes things
why have one then?? I don't understand jdd -- http://www.dodin.net http://valerie.dodin.org http://www.youtube.com/watch?v=t-eic8MSSfM http://www.facebook.com/profile.php?id=1412160445 -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Gerald Pfeifer a écrit :
On Tue, 24 Feb 2009, jdd wrote:
Could it be possible to have asap a working distro (no blocking bugs for the test team computers) to use on a dayly basis? most bugs are unexpected ones :-).
What prevents you from using FACTORY?
(Serious question. If there is anything, we should address it.)
the problem is that factory is always changing, adding new bugs in the time it remove old ones, so it's not possible to have the same platform for all the test team. How can one validate a bug? and any people can't even install, and the system is unusable for day to day use. only day to day use can show most of the bugs. I think there are two kinds of bug: * the hardware bugs, related to the hardware can only be found with a very large community. For this it should be enough to have some sort of live cd with minimal install system * the day to day working bugs (that may also be related to hardware) that can only be found by real work. These are less distro related (they are often related to new apps, like mozilla and flash, kdenlive bugs... many others). For these we should have a static platform with only security updates if any (probably better with no updates), with the core apps, but this platform must be really installable jdd -- http://www.dodin.net http://valerie.dodin.org http://www.youtube.com/watch?v=t-eic8MSSfM http://www.facebook.com/profile.php?id=1412160445 -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
On Streda 25 Február 2009 08:25:07 jdd wrote:
Gerald Pfeifer a écrit :
On Tue, 24 Feb 2009, jdd wrote:
Could it be possible to have asap a working distro (no blocking bugs for the test team computers) to use on a dayly basis? most bugs are unexpected ones :-).
What prevents you from using FACTORY?
(Serious question. If there is anything, we should address it.)
the problem is that factory is always changing, adding new bugs in the time it remove old ones, so it's not possible to have the same platform for all the test team. How can one validate a bug?
One thing I was trying to figure out is how to keep those bugs against Factory valid for longer time. I want to avoid situation where a bug against Factory is closed because nobody is able to reproduce it anymore, or even have no idea how to reproduce it. One suggestion I have is to use libzypp information. Do you know that if you do 'zypper dup', libzypp will always store a testcase for this run? (I hope to get it to keep more than one in future, but this is a start). Plus we have the history now. Testcase contains a snapshot of the repositories as well as the system package-wise. So, it also contains information which packages were used when the bug was triggered. What we miss is an easy way how to dig that data and possibly to recreate the situation. Stano -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
On Wed, 25 Feb 2009, Stanislav Visnovsky wrote:
Testcase contains a snapshot of the repositories as well as the system package-wise. So, it also contains information which packages were used when the bug was triggered.
What we miss is an easy way how to dig that data and possibly to recreate the situation.
It also would be nice to make those testcases easier to handle for users reporting an issue. Right now one has to upload four different files. Any chance this gets consolidated into one? Gerald -- Dr. Gerald Pfeifer E gp@novell.com SUSE Linux Products GmbH Director Inbound Product Mgmt T +49(911)74053-0 HRB 16746 (AG Nuremberg) openSUSE/SUSE Linux Enterprise F +49(911)74053-483 GF: Markus Rex -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Am Thursday 26 February 2009 23:18:24 schrieb Gerald Pfeifer:
It also would be nice to make those testcases easier to handle for users reporting an issue. Right now one has to upload four different files. Any chance this gets consolidated into one?
man 1 tar ? Greetings, Stephan -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
On Fri, 27 Feb 2009, Stephan Kulow wrote:
It also would be nice to make those testcases easier to handle for users reporting an issue. Right now one has to upload four different files. Any chance this gets consolidated into one? man 1 tar ?
Not a problem for me :-), but it would be good to have a consistent approach to this, and why not have the tool do this to begin with? Gerald -- Dr. Gerald Pfeifer E gp@novell.com SUSE Linux Products GmbH Director Inbound Product Mgmt T +49(911)74053-0 HRB 16746 (AG Nuremberg) openSUSE/SUSE Linux Enterprise F +49(911)74053-483 GF: Markus Rex -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Am Dienstag 24 Februar 2009 schrieb Martin Schlander:
Mandag 23 februar 2009 19:33:42 skrev Alberto Passalacqua:
I have thought for quite some time to the idea of building a community-based openSUSE testing team
I'm not sure an organized team is necessary. But after the 11.1 fiasco, I plan to apply more of a systematic and methodic approach to testing personally.
I would think you need a "team", so that people know where to head if they want to help in testing more than installation. Not everyone likes jumping on IRC. Greetings, Stephan -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Hi Coolo,
[...]
I would think you need a "team", so that people know where to head if they want to help in testing more than installation. Not everyone likes jumping on IRC.
There is opensuse-testing@opensuse.org as a good starting point to coordinate such a testing effort. Best wishes, Holgi -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
On Wednesday 25 February 2009 09:55:26 am Holger Sickenberg wrote:
Hi Coolo,
[...]
I would think you need a "team", so that people know where to head if they want to help in testing more than installation. Not everyone likes jumping on IRC.
There is opensuse-testing@opensuse.org as a good starting point to coordinate such a testing effort.
Also: http://en.opensuse.org/Testing_Team can be good place to report on duty ;-) Do we need introduction of interests, skills and needs? IMHO, yes, and good place would be: http://en.opensuse.org/Testing_Team/<user_nick> linked from Testing Team page. Current jobs, as I can see, would be: Testing features. Testing factory. Testing updates. Notifying people of changes on communication media [1]. Asking for help with testing. Scanning lists for reports that need developers attention. (or I'm on a BugBuster side already?) [1] media: mail lists: opensuse, opensuse-factory, opensuse-kde, opensuse-gnome, forums: forums.opensuse.org news: ? -- Regards, Rajko -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Am Mittwoch 25 Februar 2009 schrieb Holger Sickenberg:
Hi Coolo,
[...]
I would think you need a "team", so that people know where to head if they want to help in testing more than installation. Not everyone likes jumping on IRC.
There is opensuse-testing@opensuse.org as a good starting point to coordinate such a testing effort.
I know and you know. Many others need to know - and writing it in the beta announcement doesn't help ;( Greetings, Stephan -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Stephan Kulow a écrit :
There is opensuse-testing@opensuse.org as a good starting point to coordinate such a testing effort.
I know and you know. Many others need to know - and writing it in the beta announcement doesn't help ;(
frankly, there are too many mailing lists with similar scope. This should be cleared on the wiki jdd -- http://www.dodin.net http://valerie.dodin.org http://www.youtube.com/watch?v=t-eic8MSSfM http://www.facebook.com/profile.php?id=1412160445 -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Am Donnerstag 26 Februar 2009 schrieb jdd:
Stephan Kulow a écrit :
There is opensuse-testing@opensuse.org as a good starting point to coordinate such a testing effort.
I know and you know. Many others need to know - and writing it in the beta announcement doesn't help ;(
frankly, there are too many mailing lists with similar scope. This should be cleared on the wiki
No idea what this has to do with the current topic though: factory development and status of factory is clearly belonging to opensuse-factory Greetings, Stephan -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Stephan Kulow a écrit :
Am Donnerstag 26 Februar 2009 schrieb jdd:
Stephan Kulow a écrit :
There is opensuse-testing@opensuse.org as a good starting point to coordinate such a testing effort. I know and you know. Many others need to know - and writing it in the beta announcement doesn't help ;( frankly, there are too many mailing lists with similar scope. This should be cleared on the wiki
No idea what this has to do with the current topic though: factory development and status of factory is clearly belonging to opensuse-factory
is not the current topic about development? The wiki is, right now, a mess about testing, factory and so on. I see now at least three lists about development: testing, factory, project. what is the good one? is factory list only for programmers or also for testers? why did this discussion start on project and was not directed to testing? do testers need also to subscribe project and factory and opensuse and wiki and... whatever? could we be practical? why test openSUSE should be so complicated? jdd -- http://www.dodin.net http://valerie.dodin.org http://www.youtube.com/watch?v=t-eic8MSSfM http://www.facebook.com/profile.php?id=1412160445 -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Moin, On Thursday 26 February 2009, jdd wrote:
Stephan Kulow a écrit :
Am Donnerstag 26 Februar 2009 schrieb jdd:
Stephan Kulow a écrit :
There is opensuse-testing@opensuse.org as a good starting point to coordinate such a testing effort.
I know and you know. Many others need to know - and writing it in the beta announcement doesn't help ;(
frankly, there are too many mailing lists with similar scope. This should be cleared on the wiki
No idea what this has to do with the current topic though: factory development and status of factory is clearly belonging to opensuse-factory
is not the current topic about development? Let's come back to the topic. Alberto made a very good proposal at least imo. And the discussion showed that different people made differenct experience - good and not so good ones. It shows again that we're very good in offering everything - you just need to know where or even better know an omniscient person who knows it.
The wiki is, right now, a mess about testing, factory and so on. And exactly here there is THE starting point. We need to sort the stuff and make it easy to reach (or even intuitive) and then of course explain testing and give guidance. And of course gather feedback to improve testing.
Alberto perhaps Holgi and Deamon are a good starting point for the creation of a testing team. Best M
I see now at least three lists about development: testing, factory, project. what is the good one?
is factory list only for programmers or also for testers? why did this discussion start on project and was not directed to testing? do testers need also to subscribe project and factory and opensuse and wiki and... whatever?
could we be practical? why test openSUSE should be so complicated?
jdd
-- http://www.dodin.net http://valerie.dodin.org http://www.youtube.com/watch?v=t-eic8MSSfM http://www.facebook.com/profile.php?id=1412160445
-- Michael Löffler, Product Management Email: michl@suse.de Phone: +49 911 74053-376 SUSE LINUX Products GmbH - Nürnberg - AG Nürnberg - HRB 16746 - GF: Markus Rex -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
is not the current topic about development?
Actually the topic is "testing", not "development". They are related, but not exactly the same ;-)
The wiki is, right now, a mess about testing, factory and so on.
I see now at least three lists about development: testing, factory, project. what is the good one?
-project -> is not about development. It is about the whole openSUSE project, its organization, initiatives, and so on. -factory -> is about development, distribution status, new stuff. -testing -> is to discuss about testing
why did this discussion start on project and was not directed to testing?
For two reasons: a) I didn't know of -testing b) because it is a "new" initiative for the project.
do testers need also to subscribe project and factory and opensuse and wiki and... whatever?
could we be practical? why test openSUSE should be so complicated?
Well, I often say there are too many communication media, but in this specific case, I find the separation useful and practical to keep things in order. Factory has a high number of posts, and also -project. The information about testing would be lost among a lot of other messages, so the -testing ML has a perfect reason to exist for me. Regards, A. -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
On Mon, Feb 23, 2009 at 6:33 PM, Alberto Passalacqua <alberto.passalacqua@tin.it> wrote:
Hello,
I have thought for quite some time to the idea of building a community-based openSUSE testing team, and this seems to be the right time to discuss of it.
The main idea is to create a group of stable testers to deeply test openSUSE functionalities on their machines, in order to catch annoying bugs earlier in the development stages, allowing the fixes to be done and tested again in a more timely and less rushed manner.
It won't be easy, we need motivated volunteers and we have a lot to learn to do it the right way, but I think it is worth the effort, if we want to keep faith to the guiding principles, which states openSUSE aims to be "the most usable Linux distribution" (and not the most cutting edge one!).
I summed up the motivations, a short problem analysis about the current status quo, and some proposal of solution in the attached slides so you can comment them (thanks suseROCKs for the slide template).
All suggestions (and volunteers) are more than welcome, as well as some help from Novell people currently involved in testing/quality assurance. As I said we need to learn. ;-)
With kind regards, Alberto
I have tried testing factory and also official beta/alpha releases, but huh... they were barely installable not to mention usable. And that's in a virtual machine where we are talking basic hardware setup and not something new and fancy. I see you have this issue pointed out in the presentation. -- Kind Regards, Ivan N. Zlatev -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Ivan N. Zlatev a écrit :
I have tried testing factory and also official beta/alpha releases, but huh... they were barely installable not to mention usable. And that's in a virtual machine where we are talking basic hardware setup and not something new and fancy. I see you have this issue pointed out in the presentation.
could it be possible to define a virtualbox image to test? that is a preconfigured system. So we could test the working, not the install, and use it in dayly work (most modern computers can work with a virtual box) jdd -- http://www.dodin.net http://valerie.dodin.org http://www.youtube.com/watch?v=t-eic8MSSfM http://www.facebook.com/profile.php?id=1412160445 -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
Hi Alberto,
I have thought for quite some time to the idea of building a community-based openSUSE testing team, and this seems to be the right time to discuss of it.
The main idea is to create a group of stable testers to deeply test openSUSE functionalities on their machines, in order to catch annoying bugs earlier in the development stages, allowing the fixes to be done and tested again in a more timely and less rushed manner.
It won't be easy, we need motivated volunteers and we have a lot to learn to do it the right way, but I think it is worth the effort, if we want to keep faith to the guiding principles, which states openSUSE aims to be "the most usable Linux distribution" (and not the most cutting edge one!).
I summed up the motivations, a short problem analysis about the current status quo, and some proposal of solution in the attached slides so you can comment them (thanks suseROCKs for the slide template). [...] All suggestions (and volunteers) are more than welcome, as well as some help from Novell people currently involved in testing/quality assurance. As I said we need to learn. ;-)
Thanks a lot for bringing up this topic. I think you are right - a Community testing team would help the openSUSE project a lot. I'm responsible for openSUSE testing in Novell and - as some other mentioned before - we already spent some efforts to make testing more easier for the Community: - http://en.opensuse.org/Testing It's the entrance point for testing of openSUSE and includes a "openSUSE 11.1 Feature Test Plan". - opensuse-testing@opensuse.org The mailing list to discuss and coordinate testing efforts. - FreeNode #opensuse-testing An IRC channel to directly get in touch with other testers. - openSUSE 11.1 Feature Testing Day on Thursday November, 5th 2008 So you can see there are already some activities in the testing area. Nevertheless there could be much more and I'm willing to help you and all other Community members that are also willing to help. So feel free to get in touch with me either directly or on own of the mailing lists. Best wishes, Holgi -- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-project+help@opensuse.org
participants (17)
-
Alberto Passalacqua -
Andreas Jaeger -
Eric Springer -
Gerald Pfeifer -
Holger Sickenberg -
Ivan N. Zlatev -
jdd -
Marco Michna -
Martin Schlander -
Michael Loeffler -
Pascal Bleser -
Peter Poeml -
Rajko M. -
Ruediger Oertel -
Stanislav Visnovsky -
Stephan Kulow -
Vincent Untz