Mailinglist Archive: opensuse-factory (1324 mails)

< Previous Next >
Re: [opensuse-factory] Will we have additional repositories for Leap as we have currently?
  • From: Richard Brown <RBrownCCB@xxxxxxxxxxxx>
  • Date: Mon, 19 Oct 2015 22:59:49 +0200
  • Message-id: <CAA0b23x048WhLjGgsoRQ=b4L58H2_TXz-rL3gq5yRPSE6KzRTQ@mail.gmail.com>
OpenQA only does
basic testing, you'll have to recognize, some things depend on people
testing, and some use cases get none of that.

openQA does pretty damn advanced testing, and I challenge you to find
something which openQA absolutely cannot test. Multi machine,
multipath, HA, real hardware, you name it, openQA can do it, or we
want to find a way to make it do it.

The biggest limitation is not the technology, but that someone has to
care enough about an issue to write a test for it

That's not that hard to do -
https://github.com/os-autoinst/openQA/blob/master/docs/WritingTests.asciidoc

And there is lots of examples to learn from -
https://github.com/os-autoinst/os-autoinst-distri-opensuse/tree/master/tests

Even when there's the technical possibility of testing, it's not always done.

See what's happening with i586.

Wasting hardware testing hundreds of scenarios on an architecture a
minority, a shrinking minority, of users use isn't an effective use of
hardware..sure

But we still are going to keep an i586 'canary' or two around to keep
some attention on that old platform.

If people would like to donate more testing hardware, they can contact
donations@xxxxxxxxxxxx, we'll gladly take it and integrate it into the
openQA system.

You really think if someone wrote a few thousand tests covering every
workflow for the top 90% of users, it'd actually be run? There would
probably not be enough resources to do that amount of testing.

We're running a few hundred different scenarios, each including 20-30
tests, so yes, we're already at the few thousand test mark..

And nobody did write that many tests. And no, it's not that easy. I
read OpenQA test scripts and they read like chinese to me, and I
consider myself quite capable of reading code in any language.

https://openqa.opensuse.org/tests/94150/modules/gedit/steps/1/src

Really? it's not that hard..starting at line 10 I'll decode into plain
english (the rest is boilerplate stuff, as described in the docs)

10 - load the X11 application called gedit
11 - check the screen is showing gedit launched
12 - type the string "If you can see this text gedit is working.\n"
13 - wait 2 seconds
14 - check the screen is showing the text
15 - wait 2 seconds
16 - press alt-f4
17 - wait 2 seconds
18 - press alt-w
19 - sleep 2 seconds

Even the most complicated tests all share a common heritage with
simple ones like this example..just as we want to delve a little
deeper, things sometimes get a little bit more fanciful..but that's
the same with all programming :)
--
To unsubscribe, e-mail: opensuse-factory+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: opensuse-factory+owner@xxxxxxxxxxxx

< Previous Next >