Mailinglist Archive: opensuse-factory (1324 mails)

< Previous Next >
Re: [opensuse-factory] Will we have additional repositories for Leap as we have currently?
  • From: Claudio Freire <klaussfreire@xxxxxxxxx>
  • Date: Mon, 19 Oct 2015 19:02:51 -0300
  • Message-id: <>
On Mon, Oct 19, 2015 at 5:59 PM, Richard Brown <RBrownCCB@xxxxxxxxxxxx> wrote:
OpenQA only does
basic testing, you'll have to recognize, some things depend on people
testing, and some use cases get none of that.

openQA does pretty damn advanced testing, and I challenge you to find
something which openQA absolutely cannot test. Multi machine,
multipath, HA, real hardware, you name it, openQA can do it, or we
want to find a way to make it do it.

The biggest limitation is not the technology, but that someone has to
care enough about an issue to write a test for it

That's not that hard to do -

And there is lots of examples to learn from -

Even when there's the technical possibility of testing, it's not always done.

See what's happening with i586.

Wasting hardware testing hundreds of scenarios on an architecture a
minority, a shrinking minority, of users use isn't an effective use of

I agree, that's why I say OpenQA can only do limited (I said basic,
but I should have used "limited") testing.

The cost-benefit balance stops being positive after some point.

You really think if someone wrote a few thousand tests covering every
workflow for the top 90% of users, it'd actually be run? There would
probably not be enough resources to do that amount of testing.

We're running a few hundred different scenarios, each including 20-30
tests, so yes, we're already at the few thousand test mark..

And it's already at its limits, which is, as I understand, why i586
tests are being nixed.

And nobody did write that many tests. And no, it's not that easy. I
read OpenQA test scripts and they read like chinese to me, and I
consider myself quite capable of reading code in any language.

Really? it's not that hard..starting at line 10 I'll decode into plain
english (the rest is boilerplate stuff, as described in the docs)

10 - load the X11 application called gedit
11 - check the screen is showing gedit launched
12 - type the string "If you can see this text gedit is working.\n"
13 - wait 2 seconds
14 - check the screen is showing the text
15 - wait 2 seconds
16 - press alt-f4
17 - wait 2 seconds
18 - press alt-w
19 - sleep 2 seconds

Yeah, it's the needles that confuse me really.

I may give it another try, if I find something worth testing that
isn't being tested already.
To unsubscribe, e-mail: opensuse-factory+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: opensuse-factory+owner@xxxxxxxxxxxx

< Previous Next >
Follow Ups