[Bug 1186256] New: qemu-linux-user: hardcoded binfmt handler doesn't play well with containers
https://bugzilla.suse.com/show_bug.cgi?id=1186256 Bug ID: 1186256 Summary: qemu-linux-user: hardcoded binfmt handler doesn't play well with containers Classification: openSUSE Product: openSUSE Tumbleweed Version: Current Hardware: Other OS: Other Status: NEW Severity: Normal Priority: P5 - None Component: KVM Assignee: kvm-bugs@suse.de Reporter: martin.wilck@suse.com QA Contact: qa-bugs@suse.de Found By: --- Blocker: --- Created attachment 849481 --> https://bugzilla.suse.com/attachment.cgi?id=849481&action=edit Proposed patch for qemu-binfmt-conf.sh Since abbc0ce ("qemu-binfmt-conf: use qemu-ARCH-binfmt"), qemu-binfmt-conf.sh under openSUSE automatically replaces the default qemu binfmt wrapper "qemu-$ARCH" with "qemu-$ARCH-binfmt" in order to ensure that argv[0] is preserved; qemu-$ARCH-binfmt is a link to qemu-binfmt, which is just a simple wrapper that mangles argv to achieve the desired result. This is a SUSE-specific modification which isn't used upstream. This approach is inconvenient in some situations. In particular for running foreign-arch containers, it's useful to use the binfmt_misc "F" ("fix binary") flag to pre-load the qemu wrapper in the kernel. That way, foreign-arch containers can be run just like native containers, without having to bind-mount interpreters into the container. But that's impossible with the SUSE binfmt wrapper that needs to exec() a different (native) executable. In the openSUSE default mode of qemu-binfmt-conf.sh, the user needs to bind-mount both the -binfmt executable and the actual emulator into the container:
$ podman run -it --rm \ -v /usr/bin/qemu-ppc64le-binfmt:/usr/bin/qemu-ppc64le-binfmt \ -v /usr/bin/qemu-ppc64le:/usr/bin/qemu-ppc64le \ ppc64le/busybox uname -m ppc64le
Otherwise, he gets
$ podman run -t --rm ppc64le/busybox uname -m standard_init_linux.go:219: exec user process caused: no such file or directory
If qemu-binfmt-conf.sh is used with the --persistent flag, qemu-ppc64le-binfmt is loaded into the kernel, but qemu-ppc64le must still be bind-mounted. If qemu-ppc64le was used directly as persistent binfmt_misc helper, it would be sufficient to run the container as if it was a native one:
$ podman run -it --rm ppc64le/busybox uname -m ppc64le
I can see why it makes sense to try to preserve argv[0], but for me at least, the "foreign container" use case is more important. Therefore I'd like to be able to switch the behavior of the qemu binfmt_misc helper back to the upstream default. So far I've worked around the issue by simply using the upstream container "docker.io/multiarch/qemu-user-static", but I'd like to be able to do this easily with openSUSE on-board tools. The attached patch allows the user to override the default "-binfmt" suffix by running "qemu-binfmt-conf.sh --qemu-suffix ''". (Note: "qemu-binfmt-conf.sh -F ''" doesn't work, that's a different issue). -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1186256
Martin Wilck
https://bugzilla.suse.com/show_bug.cgi?id=1186256
Martin Wilck
https://bugzilla.suse.com/show_bug.cgi?id=1186256
https://bugzilla.suse.com/show_bug.cgi?id=1186256#c1
Martin Wilck
https://bugzilla.suse.com/show_bug.cgi?id=1186256
https://bugzilla.suse.com/show_bug.cgi?id=1186256#c2
--- Comment #2 from Martin Wilck
https://bugzilla.suse.com/show_bug.cgi?id=1186256
https://bugzilla.suse.com/show_bug.cgi?id=1186256#c3
--- Comment #3 from Martin Wilck
https://bugzilla.suse.com/show_bug.cgi?id=1186256
Jos� Ricardo Ziviani
https://bugzilla.suse.com/show_bug.cgi?id=1186256
https://bugzilla.suse.com/show_bug.cgi?id=1186256#c4
--- Comment #4 from Jos� Ricardo Ziviani
https://bugzilla.suse.com/show_bug.cgi?id=1186256
https://bugzilla.suse.com/show_bug.cgi?id=1186256#c5
--- Comment #5 from Martin Wilck
https://bugzilla.suse.com/show_bug.cgi?id=1186256
Matej Cepl
https://bugzilla.suse.com/show_bug.cgi?id=1186256
https://bugzilla.suse.com/show_bug.cgi?id=1186256#c7
--- Comment #7 from Martin Wilck
https://bugzilla.suse.com/show_bug.cgi?id=1186256
https://bugzilla.suse.com/show_bug.cgi?id=1186256#c8
--- Comment #8 from Jos� Ricardo Ziviani
Jos�,
we're not there yet because an upstream bot rejected my -F patch (comment 2) because of a style issue which was definitely not my fault. The overlong line was there before my patch already. I never got this reply (spam folder? no idea), so I was also never able to fix this non-issue.
https://lists.gnu.org/archive/html/qemu-devel/2021-05/msg06012.html
I'll re-post the patch and cc you. I'd be glad if you could pull it into opensuse before upstream gets to it.
Hello Martin, Sure, I'll add it here. By the way, your -F patch is in Factory, should be available in this next update. Thanks -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1186256
Claudio Fontana
https://bugzilla.suse.com/show_bug.cgi?id=1186256
Jos� Ricardo Ziviani
https://bugzilla.suse.com/show_bug.cgi?id=1186256
https://bugzilla.suse.com/show_bug.cgi?id=1186256#c13
--- Comment #13 from Martin Wilck
https://bugzilla.suse.com/show_bug.cgi?id=1186256
https://bugzilla.suse.com/show_bug.cgi?id=1186256#c14
--- Comment #14 from Martin Wilck
https://bugzilla.suse.com/show_bug.cgi?id=1186256
https://bugzilla.suse.com/show_bug.cgi?id=1186256#c15
--- Comment #15 from Martin Wilck
https://bugzilla.suse.com/show_bug.cgi?id=1186256
Dario Faggioli
https://bugzilla.suse.com/show_bug.cgi?id=1186256
https://bugzilla.suse.com/show_bug.cgi?id=1186256#c18
Martin Wilck
https://bugzilla.suse.com/show_bug.cgi?id=1186256
https://bugzilla.suse.com/show_bug.cgi?id=1186256#c22
Martin Wilck
Mmm... I also see this:
virt136:~ # ls /usr/bin/qemu-ppc64le* -l -rwxr-xr-x 1 root root 3940664 Dec 6 14:35 /usr/bin/qemu-ppc64le lrwxrwxrwx 1 root root 11 Dec 6 14:33 /usr/bin/qemu-ppc64le-binfmt -> qemu-binfmt
This is the normal SUSE setup. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1186256
https://bugzilla.suse.com/show_bug.cgi?id=1186256#c24
Dario Faggioli
participants (1)
-
bugzilla_noreply@suse.com