[yast-devel] Speedup unit tests: parallel_tests gem and parallel Travis jobs
Hi all, Short version: Install the latest ruby2.5-rubygem-parallel_tests ruby2.5-rubygem-parallel_tests packages from YaST:Head OBS and enjoy lightning fast "rake test:unit" in yast2-storage-ng :-) (The "devel_yast" pattern has been adapted, running "zypper dup" should be enough.) Long version: I have spent some time speeding up the unit tests in yast2-storage-ng package. The reason was that running all tests took several minutes, about 3 minutes even on a quite fast CPU. So after doing even a small change you would have to wait several minutes to see whether you did not break something by mistake. The main bottleneck was the sequential start of the tests, so even on a quad core CPU only one CPU was actually used. I have added parallel_test gem [1] support to "rake test:unit" and turned that on in yast2-storage-ng package. (By default the standard sequential RSpec is used.) It uses all available CPUs and scales almost linearly, the speed up is about `nproc` times. A similar approach I have used in Travis, originally we used license check, Rubocop, yardoc, unit tests, package build, ... everything sequentially in one Travis job. Fortunately Travis supports defining multiple jobs which run independently and are started in parallel. I have split the Travis work in yast2-storage-ng into three groups: unit tests, rubocop and the rest (yardoc, package build...). If you want to use a similar solution in your package then check the documentation I wrote about this: the Travis [2], the unit tests [3]. Some real numbers to see the improvement: - Running "rake test:unit" locally: from 2:47s to 0:38s (speedup ~4.3x on a quad core CPU with HT on) - Similar speed up in "rake osc:build" (it just additionally prepares the chroot) - OBS package build: from 323s-505s to 102s-235s (checked several builds, it highly depends on the speed of the worker) - Travis speed up: from 8-10 minutes to 3-4 minutes (parallel jobs + parallel tests) Enjoy! [1] https://github.com/grosser/parallel_tests [2] http://yastgithubio.readthedocs.io/en/latest/travis-integration/#parallel-bu... [3] http://yastgithubio.readthedocs.io/en/latest/how-to-write-tests/#parallel-te... -- Best Regards Ladislav Slezák Yast Developer ------------------------------------------------------------------------ SUSE LINUX, s.r.o. e-mail: lslezak@suse.cz Lihovarská 1060/12 tel: +420 284 028 960 190 00 Prague 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 18.06.2018 11:06, Ladislav Slezak wrote:
Short version:
Install the latest ruby2.5-rubygem-parallel_tests ruby2.5-rubygem-parallel_tests packages from YaST:Head OBS and enjoy lightning fast "rake test:unit" in yast2-storage-ng:-)
Test results: rake test:unit in yast-storage-ng took 22:30 (yes, 22 min 30 sec!) on my machine before that. Using parallel tests reduced this to 4:50. The code and the tests were running on a local XFS filesystem, not on NFS. Moving the y2log from my NFS home to that local XFS reduced it to 0:37 (!!). Redirecting the y2log to /dev/null reduced it to 0:31 (!!). This is as simple as export Y2LOG_FILE=/dev/null rake test:unit or, for just that one call, Y2LOG_FILE=/dev/null rake test:unit This is really worthwhile. Kind regards -- Stefan Hundhammer <shundhammer@suse.de> YaST Developer SUSE Linux GmbH GF: Felix Imendörffer, Jane Smithard, Graham Norton; HRB 21284 (AG Nürnberg) Maxfeldstr. 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Thu, Aug 16, 2018 at 04:33:56PM +0200, Stefan Hundhammer wrote:
Test results:
rake test:unit in yast-storage-ng took 22:30 (yes, 22 min 30 sec!) on my machine before that.
Using parallel tests reduced this to 4:50. The code and the tests were running on a local XFS filesystem, not on NFS.
Moving the y2log from my NFS home to that local XFS reduced it to 0:37 (!!).
Redirecting the y2log to /dev/null reduced it to 0:31 (!!).
This is as simple as
export Y2LOG_FILE=/dev/null
Y2SLOG_FILE is the right variable (who knows why).
rake test:unit
or, for just that one call,
Y2LOG_FILE=/dev/null rake test:unit
This is really worthwhile.
Thanks, that is indeed worthwhile. In general I would expect a testsuite to generate a logfile per test in the local directory. Otherwise finding the log for a single test seems almost impossible. ciao Arvin -- Arvin Schnell, <aschnell@suse.com> Senior Software Engineer, Research & Development SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
V Mon, 3 Sep 2018 12:31:52 +0000 Arvin Schnell <aschnell@suse.com> napsáno:
On Thu, Aug 16, 2018 at 04:33:56PM +0200, Stefan Hundhammer wrote:
Test results:
rake test:unit in yast-storage-ng took 22:30 (yes, 22 min 30 sec!) on my machine before that.
Using parallel tests reduced this to 4:50. The code and the tests were running on a local XFS filesystem, not on NFS.
Moving the y2log from my NFS home to that local XFS reduced it to 0:37 (!!).
Redirecting the y2log to /dev/null reduced it to 0:31 (!!).
This is as simple as
export Y2LOG_FILE=/dev/null
Y2SLOG_FILE is the right variable (who knows why).
rake test:unit
or, for just that one call,
Y2LOG_FILE=/dev/null rake test:unit
This is really worthwhile.
Thanks, that is indeed worthwhile.
In general I would expect a testsuite to generate a logfile per test in the local directory. Otherwise finding the log for a single test seems almost impossible.
ciao Arvin
Just hint. When I need to inspect specific test, I usually do something like rm ~/.y2log and then rspec <my_test.rb>:<line of failing test> so e.g. rspec bootloader_test.rb:140 adn this writes only logs for the specific test. It works also for e.g. context block. Josef -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
participants (4)
-
Arvin Schnell
-
Josef Reidinger
-
Ladislav Slezak
-
Stefan Hundhammer