Hi all, I found an alternative way how to build packages at Travis. Instead of using Ubuntu packages it we can use Docker and run a custom Docker image. (Actually it runs inside that Ubuntu VM, but that does not matter much, Docker it lightweight and should not add much overhead.) I started with a simple example in libyui-qt [1], initially to solve the issue with not building pull requests. (mvidner already tried to solve that but without success...) Then improved it a bit and applied in the registration module to see how it really works [2]. The improvement was that now we could even build the RPM package and try installing it. That's was not possible in Ubuntu at all. Later today a problem in the ntp-client package appeared: it required a newer augeas package. To make Travis pass we would need to find or port a newer package to Ubuntu... Quite tricky, isn't it? So I adapted the patch from the registration package and switched the ntp-client package to Docker as well [3]. And the tests passed! So it turns out that switching to Docker at Travis really helps. Docker Advantages ================== - Does not need packages built for Ubuntu (less maintenance for us, less troubles with building/publishing the packages by OBS) - Closer matches our target environment (the same tooling) - We can even build the rpm package from the sources (using plain `rpmbuild` to make it simple), that was not possible with Ubuntu, i.e. the `*.spec` file issues will be found now - Can build Pull Requests, in Jenkins it's not trivial - You can easily reproduce the build locally, just run the same commands as in the `.travis.yml` file. You download and use the same image as Travis does, that makes debugging much easier. Disadvantages ============= - A bit longer build, but not much (about +1-2 minutes in the few cases have so far). It could be improved by building our custom Docker image(s) with pre-installed YaST build dependencies, see below. Note: we need to keep the Jenkins workers as we cannot send submit requests to build service (at least not to the internal one, in theory sending to the public OBS should be possible). BTW I tried running "osc build" in Docker and it failed with several "mount" errors, I guess it is trying to mount /proc and similar fs and fails. Docker is not a virtualization, it's a container, i.e. chroot on steroids. Ideas for Enhancement ===================== 1. Share the code or configuration - all Docker based builds look very similar so far, we should somehow DRY it... 2. Build custom YaST Docker images, I think 2 images (or so) should be enough, one for C++ based yast packages, one for Ruby ones[*]. Maybe some more for special cases, but we do not need to build a separate image for each package, if you need just few extra packages "zypper install" is your friend... I have no clue about the storage-ng requirements, maybe we could use the same YaST images. But even adding few storage-ng images should not be a problem. More details: - The docker images can be built and uploaded by Travis, this is even mentioned in the official documentation [4]. - Travis supports cron based jobs as well, they are started regularly regardless a new commit was pushed or not [5]. With this feature we could build a fresh image daily. (I think it is possible to trigger a build manually if needed sooner. At least for commit based builds it works.) Ideas? Comments? [1] https://github.com/libyui/libyui-qt/pull/66/files [2] https://github.com/yast/yast-registration/pull/289 [3] https://github.com/yast/yast-ntp-client/pull/66 [4] https://docs.travis-ci.com/user/docker/#Pushing-a-Docker-Image-to-a-Registry [5] https://docs.travis-ci.com/user/cron-jobs/ [*] We could check the dependency graphs (https://lizards.opensuse.org/wp-content/uploads/2016/09/yast_deps_after.png) and see if there are some groups of packages or to find the set which is enough for most packages... -- Ladislav Slezák YaST Developer SUSE LINUX, s.r.o. Corso IIa Křižíkova 148/34 18600 Praha 8 -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org