Mailinglist Archive: yast-devel (38 mails)

< Previous Next >
Re: [yast-devel] Article on Testing
On 07/24/2018 12:41 PM, Arvin Schnell wrote:

Hi,

I read the article [1] and some of the references about testing
and found it interesting and also disillusioning.

ciao Arvin

[1]
https://blog.usejournal.com/lean-testing-or-why-unit-tests-are-worse-than-you-think-b6500139a009

Good article, I fully agree with most of the points.

But it seems to be written in the scope of modern web development.
That's a much more friendly environment for integration testing and
other practices.

On integration tests vs unit tests
==================================

The article is right here. Before joining the the YaST Team, I seldom
wrote unit tests for the components. Only for those which implemented
tricky logic (like the BootRequirementsChecker, to show a YaST-oriented
example). The biggest part of my tests were integration tests...

But that was in the web development world, in which writing integration
tests is a piece of cake. For example, in my last Rails (pre-YaST)
project I could tests the whole stack as easily as this:

https://github.com/openSUSE/travel-support-program/tree/master/spec/features

To be more concrete, the following spec tests the rendered HTML, the
Javascript behavior, the database update and the mailer (and all the
interactions between those parts) by just describing the user actions
and the application responses to them. Just describing the scenario like
"go here", "click on that button", "this should appear".

https://github.com/openSUSE/travel-support-program/blob/master/spec/features/comments_spec.rb

We are far from having something similar in YaST, that can be written
and maintained so easily and that can be run with a single command in
some seconds. For proper integration, we need to reproduce a whole Linux
system with some setup, to interact with the UI, to inspect the real
changes in the system... We have nothing of that, so we fallback to unit
testing, which admittedly implies a bigger level of mocking and provides
a way lower ROI.

On testing only the common cases but not the exotic ones
========================================================

I partially disagree here. I really believe that thinking about the
boundary scenarios and the corner cases while writing tests for them
helps a lot to write better code from the very beginning and to reduce
errors.

After all, the errors in the "happy path" are obvious enough to be found
easily0, even manually. One of the things I like more about testing is
the exercise of challenging your code with different situations.

Of course, that implies an approach to testing based on scenarios and
BDD, not just repeating the implementation with the RSpec syntax.
Because when we write tests that are suspiciously similar to the
implementation we are indeed just pleasing the code coverage stat with
no real gain.

Unit tests and the ossification of the internal structure
==========================================================

I agree with the main point here. But again, that depends on the context
for which the article is written. If you are just writing a web
application to be used via browser, then you don't want the API of your
components to get "ossified". You only care about the user-visible behavior.

But an important part of the work in YaST is about writing APIs to be
consumed in several parts of the project. Usually in other YaST modules
living in another repository. And you really want those APIs to be ossified.

The smaller internal classes and methods should be tested indirectly,
using the more external API instead of calling every single method
explicitly in the unit tests, but they should still be unit-tested.

Cheers.
--
Ancor González Sosa
YaST Team at SUSE Linux GmbH
--
To unsubscribe, e-mail: yast-devel+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: yast-devel+owner@xxxxxxxxxxxx

< Previous Next >
Follow Ups