Mailinglist Archive: yast-devel (144 mails)

< Previous Next >
Re: [yast-devel] Requirements for Yast testing framework
Dne 12.8.2013 16:04, Ladislav Slezak napsal(a):
in a different thread we are discussing the possible frameworks for writing
Although it's nice to have some preferred framework and style, we should
collect our requirements first to see what we need or expect from it.


I think there is one decision missing which would translate into a requirement:

What style do we want to write the tests in?

In Ruby world, there are currently two dominant styles:

* Test::Unit-like
* RSpec-like

The most important distinction is that Test::Unit-like frameworks use regular classes and methods to write tests, RSpec-like frameworks use structured blocks.

I think we should choose one style, as mixing both in one codebase would only lead to confusion and inconsistency. I also think that we should choose RSpec-like style, as it leads to better structured and documented tests.

In the next paragraphs, I will assume that RSpec-like style is the preferred one and thus its support is a requirement. If someone disagrees with this assumption, please say so.

Now, a second requirement I feel is missing becomes important:

If possible, use what other Ruby developers use.

This is extremely important. Choosing a widely used framework means it will be more stable, supported, documented, etc. than the other alternatives. There will be a large body of knowledge available when we get into trouble. Its development will also probably progress faster.

Most importantly of all, using a widely used framework will make it easy for outside contributors to hack YaST becaue they won't need to learn a new thing (most likely). I think this is in line with the philosophy which was also behind the Ruby translation.


If one looks at the existing frameworks, it's clear that for RSpec-like style the main contenders are minitest and RSpec.

For minitest, I see one advantage -- it is being bundled with Ruby. While it is used by many projects, I have yet to see one where its RSpec-like way of writing tests is used. To me, it looks like people treat it mostly as a replacement of Test::Unit. This is a disadvantage.

RSpec seems to be the default choice in the Ruby world at the moment and for me, it is a clear winner in the "use what other Ruby developers use" category. It also offers *only* RSpec-like tests, which means they are treated in a first-class way. It is not just one of several offered options like in case of minitest. I see that as a an advantage (would you rather buy your bike in Tesco, or in a specialized bike shop?).

My conclusion is: Both minitest and RSpec seem good choices, but for the reasons stated above, I think RSpec is a better one.


BTW, we should get to a firm decision soon, as the testing framework decision blocks new development. If the question isn't resolved here soon, it will get resolved "organically" (= everyone doing what he wants). I don't think anybody wants that.

David Majda
SUSE Studio developer
To unsubscribe, e-mail: yast-devel+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: yast-devel+owner@xxxxxxxxxxxx

< Previous Next >