CI (GitHub Actions/Travis) broken due to Glibc upgrade :-(
Hi, TLDR: Travis/Github Actions are broken right now, ignore the failures :-/ The Tumbleweed distro upgraded Glibc to version 2.33. This uses a new syscall (faccessat2) which requires a newer kernel (5.8+) and some support in Docker. Unfortunately these are not present in the Ubuntu workers running at GitHub or Travis so the result is many EPERM exceptions or not found file errors. The bad part is that we cannot do much about it :-( Either TW disables that syscall or Github/Travis upgrades the workers to support that. The only workaround would be to use the Leap 15.3 as the base distro for CI. So far there should not big differences so that should work fine, at least for some time. We could use it until the issue is resolved, later we can switch back to TW. BTW the same problem also happens in other distros and in other systems (even WSL has been reported). This clearly shows the disadvantage of containers. Even if we bundle our complete openSUSE user space stack (down to glibc) we still rely on the host kernel (Ubuntu in this case) and this might cause some incompatibilities... :-/ Some links: https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/F... https://github.com/microsoft/WSL/issues/6562 https://docs.docker.com/engine/release-notes/#security-1 -- Ladislav Slezák YaST Developer SUSE LINUX, s.r.o. Corso IIa Křižíkova 148/34 18600 Praha 8
On 2/19/21 11:10 AM, Ladislav Slezak wrote:
The only workaround would be to use the Leap 15.3 as the base distro for CI. So far there should not big differences so that should work fine, at least for some time. We could use it until the issue is resolved, later we can switch back to TW.
We can also go back to "self-baked solution" @home, but that's IMO a step back. This problem will not last forever. But it will most probably happen again one day. Similarly, Flatpak on my 15.2 reported to be too old for official Chromium.
BTW the same problem also happens in other distros and in other systems (even WSL has been reported). This clearly shows the disadvantage of containers. Even if we bundle our complete openSUSE user space stack (down to glibc) we still rely on the host kernel (Ubuntu in this case) and this might cause some incompatibilities... :-/
While I'm not a "containers everywhere" guy, this is IMO an exception and advantages of using containers (for our use-case) are still much bigger than disadvantages. Anyway, we've just finished SUSE Workshop where Containers were an important topic. I believe that this issue deserves being recorded there... Anyone who could summarize the problem there, please? Thx Lukas -- Lukas Ocilka, Systems Mgmt & Consulting Team Leader & YaST Product Owner SLE Department, SUSE Linux
Dne 19. 02. 21 v 11:10 Ladislav Slezak napsal(a):
Hi,
TLDR: Travis/Github Actions are broken right now, ignore the failures :-/
[...]
The only workaround would be to use the Leap 15.3 as the base distro for CI.
OK, I applied the workaround, all Docker images have been rebased to Leap 15.3. The content is the same, there should be no big difference, maybe some slightly different versions of some core packages (like glibc ;-)). The YaST packages are taken from YaST:Head as usually... Just restart the build to make your pull request green! :-) If you still see some problem with the new images just ping me. Ladislav -- Ladislav Slezák YaST Developer SUSE LINUX, s.r.o. Corso IIa Křižíkova 148/34 18600 Praha 8
Dne 22. 02. 21 v 18:24 Ladislav Slezak napsal(a):
Dne 19. 02. 21 v 11:10 Ladislav Slezak napsal(a):
Hi,
TLDR: Travis/Github Actions are broken right now, ignore the failures :-/
[...]
The only workaround would be to use the Leap 15.3 as the base distro for CI.
OK, I applied the workaround, all Docker images have been rebased to Leap 15.3.
The Docker environment in GitHub Actions has been updated and now should work properly with glibc-2.33 in the containers. I have reverted back all YaST "master" Docker images to use openSUSE Tumbleweed again. If you see any problem after the switch just ping me. -- Ladislav Slezák YaST Developer SUSE LINUX, s.r.o. Corso IIa Křižíkova 148/34 18600 Praha 8
participants (2)
-
Ladislav Slezak -
Lukas Ocilka