Mailinglist Archive: yast-devel (36 mails)

< Previous Next >
Re: [yast-devel] Article on Testing
Dne 6.8.2018 v 13:00 Ancor Gonzalez Sosa napsal(a):

...

To be more concrete, the following spec tests the rendered HTML, the
Javascript behavior, the database update and the mailer (and all the
interactions between those parts) by just describing the user actions
and the application responses to them. Just describing the scenario like
"go here", "click on that button", "this should appear".

https://github.com/openSUSE/travel-support-program/blob/master/spec/features/comments_spec.rb

We are far from having something similar in YaST, that can be written
and maintained so easily and that can be run with a single command in
some seconds. For proper integration, we need to reproduce a whole Linux
system with some setup, to interact with the UI, to inspect the real
changes in the system... We have nothing of that,

Not in YaST directly, but we have openQA.

But the disadvantage is that

a) It takes ages to get an ISO with your change for the test.
b) You cannot easily run it locally or in CI as a part of the development
process.
c) It's hard to setup some scenarios, e.g. I can simply simulate a network issue
in the registration module unit test by raising the Net::HTTPNotFound
exception,
in an integration test I'd need a testing proxy server.
d) We cannot test too many scenarios as openQA is slow and does not have
shortcuts.
You cannot just test the storage proposal for scenario X, you have to go
through
full installation from the boot to the end. And that takes some time.

So for me the unit tests are cheap and quick way for testing, we just need
to find the right balance between the unit tests and the integration tests
(openQA).


On testing only the common cases but not the exotic ones
========================================================

I partially disagree here. I really believe that thinking about the
boundary scenarios and the corner cases while writing tests for them
helps a lot to write better code from the very beginning and to reduce
errors.


You usually test the happy path already during development, I believe
we get only few bugs that something obvious does not work. I guess most
of the bugs we get are caused by some strange hardware, complicated setup,
users do unusual or even crazy things...

So I consider it important to cover also the exotic cases.


--
Ladislav Slezák
YaST Developer

SUSE LINUX, s.r.o.
Corso IIa
Křižíkova 148/34
18600 Praha 8
--
To unsubscribe, e-mail: yast-devel+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: yast-devel+owner@xxxxxxxxxxxx

< Previous Next >
References