Mailinglist Archive: yast-devel (43 mails)

< Previous Next >
[yast-devel] Using Docker at Travis

Hi all,

I found an alternative way how to build packages at Travis.
Instead of using Ubuntu packages it we can use Docker
and run a custom Docker image. (Actually it runs inside that
Ubuntu VM, but that does not matter much, Docker it lightweight
and should not add much overhead.)

I started with a simple example in libyui-qt [1], initially
to solve the issue with not building pull requests.
(mvidner already tried to solve that but without success...)

Then improved it a bit and applied in the registration
module to see how it really works [2]. The improvement was
that now we could even build the RPM package and try
installing it. That's was not possible in Ubuntu at all.


Later today a problem in the ntp-client package appeared:
it required a newer augeas package. To make Travis pass
we would need to find or port a newer package to Ubuntu...
Quite tricky, isn't it?

So I adapted the patch from the registration package
and switched the ntp-client package to Docker as well [3].
And the tests passed!

So it turns out that switching to Docker at Travis really
helps.


Docker Advantages
==================

- Does not need packages built for Ubuntu (less maintenance
for us, less troubles with building/publishing the packages
by OBS)

- Closer matches our target environment (the same tooling)

- We can even build the rpm package from the sources (using
plain `rpmbuild` to make it simple), that was not possible
with Ubuntu, i.e. the `*.spec` file issues will be found now

- Can build Pull Requests, in Jenkins it's not trivial

- You can easily reproduce the build locally, just run the
same commands as in the `.travis.yml` file. You download
and use the same image as Travis does, that makes debugging
much easier.

Disadvantages
=============

- A bit longer build, but not much (about +1-2 minutes in the
few cases have so far). It could be improved by building our
custom Docker image(s) with pre-installed YaST build
dependencies, see below.


Note: we need to keep the Jenkins workers as we cannot send submit
requests to build service (at least not to the internal one, in
theory sending to the public OBS should be possible).

BTW I tried running "osc build" in Docker and it failed with several
"mount" errors, I guess it is trying to mount /proc and similar fs
and fails. Docker is not a virtualization, it's a container,
i.e. chroot on steroids.


Ideas for Enhancement
=====================

1. Share the code or configuration - all Docker based builds
look very similar so far, we should somehow DRY it...

2. Build custom YaST Docker images, I think 2 images (or so)
should be enough, one for C++ based yast packages, one for Ruby
ones[*]. Maybe some more for special cases, but we do not need to build
a separate image for each package, if you need just few extra packages
"zypper install" is your friend...

I have no clue about the storage-ng requirements, maybe we could use
the same YaST images. But even adding few storage-ng images should
not be a problem.

More details:

- The docker images can be built and uploaded by Travis, this is
even mentioned in the official documentation [4].

- Travis supports cron based jobs as well, they are started regularly
regardless a new commit was pushed or not [5]. With this feature
we could build a fresh image daily. (I think it is possible to
trigger a build manually if needed sooner. At least for commit
based builds it works.)


Ideas? Comments?




[1] https://github.com/libyui/libyui-qt/pull/66/files
[2] https://github.com/yast/yast-registration/pull/289
[3] https://github.com/yast/yast-ntp-client/pull/66
[4] https://docs.travis-ci.com/user/docker/#Pushing-a-Docker-Image-to-a-Registry
[5] https://docs.travis-ci.com/user/cron-jobs/

[*] We could check the dependency graphs

(https://lizards.opensuse.org/wp-content/uploads/2016/09/yast_deps_after.png)
and see if there are some groups of packages or to find the
set which is enough for most packages...

--
Ladislav Slezák
YaST Developer

SUSE LINUX, s.r.o.
Corso IIa
Křižíkova 148/34
18600 Praha 8
--
To unsubscribe, e-mail: yast-devel+unsubscribe@xxxxxxxxxxxx
To contact the owner, e-mail: yast-devel+owner@xxxxxxxxxxxx

< Previous Next >
Follow Ups