Hi, Dne 9.10.2013 10:31, Josef Reidinger napsal(a):
Hi, When I play with jenkins setup I face one issue with rspec modules that is newly written ( thanks gabi, vmoravec and mfilka for being explorer ). Issue is that some modules have heavy dependencies and idea of jenkins is to have minimal stable installation so it doesn't need regular updates like old one.
I think that the proposals for mocking strategies below should be considered not only based on the jenkis use case but as a general approach to testing of yast module code. As a yast dev I would expect minimal development dependencies which should not be yast module specific. Currently we have ruby, yast-ruby-bindings, yast-rake; these are fine as they are general and common for all yast packages/modules. For running unit tests I would not expect to have anything else installed on my development machine. This is practical as I only have a single development environment. I think that the same should be valid for the jenkins node; for running the unit tests jenkins is actually our public development environment which shouldn't be different from the private one. The jenkins building process is a different context which must not determine our testing workflow. If we consider mocking all yast stuff difficult or time consuming, we should probably unify it into some package (yast-test ?) and reuse it to be DRY. Maintainers of the the yast modules would be providing the mocking API.
There is four possible solutions with this issue:
1) mock everything outside of your module.
Pros: - testing only your module - more presure to stable API - only ruby-bindings on jenkins
Cons: - in todays modules it can be a lot of work to mock all stuff, so it can discourage developers from writting tests
2) mock everything except yast2
Making yast2 a development dependency makes yast development more complex as it is now as you must know which yast2 version you need just to run the unit tests. If this version is not available for your dev machine, you must step back and mock it anyway.
Pros: - tests also yast2 which almost doesn't have external dependencies like SCR calls to system - less work as mocking only non-framework dependencies ( like mock users in ldap module )
Cons: - yast2 must be on jenkins and it effective break idea to have one node to test all maintenance branches
3) install everything on jenkins node
Pros: - no mocking
Cons: - need to keep it uptodate ( to be honest then we need different volunteer to maintain yast jenkins node )
4) skip running rspec tests before osc:build and run it only during osc build as part of test if package build
Pros: - no dependencies, even ruby bindings is not part, only yast-rake
Cons: - if rspec test fail, then it will be later in build, so no fail fast
My preffered solution is 4 because it reduce maintain of yast jenkins node and contra is not so big.
I vote for this, we'll see if we will get bitten byt his ;) vlado
Lets discussion start. If there is no objections I change yast-rake to not run test:unit before osc:build at the end of week.
Josef