[Bug 1217306] New: distrobox stuck at starting container
https://bugzilla.suse.com/show_bug.cgi?id=1217306 Bug ID: 1217306 Summary: distrobox stuck at starting container Classification: openSUSE Product: openSUSE Distribution Version: Leap 15.6 Hardware: Other OS: Other Status: NEW Severity: Normal Priority: P5 - None Component: Containers Assignee: containers-bugowner@suse.de Reporter: lubos.kocman@suse.com QA Contact: qa-bugs@suse.de Target Milestone: --- Found By: --- Blocker: --- Created attachment 870843 --> https://bugzilla.suse.com/attachment.cgi?id=870843&action=edit few seconds of strace -p 5212 2>&1 | tee /tmp/strace-out.log Following happened on Leap 15.6 lkocman@localhost:~/Workspace> distrobox enter mirror Error: no such container mirror Cannot find container mirror Creating the container with command: /usr/bin/distrobox-create -i registry.opensuse.org/opensuse/distrobox:latest -n mirror Trying to pull registry.opensuse.org/opensuse/distrobox:latest... Getting image source signatures Copying blob 942e611a39d7 done Copying blob 1517c1ea8bb6 done Copying config 8372decf75 done Writing manifest to image destination 8372decf752e74665fcb94d31b9b378dd1790233ab36b03d37f63dac462e0fbc Creating 'mirror' using image registry.opensuse.org/opensuse/distrobox:latest [ OK ] Distrobox 'mirror' successfully created. To enter, run: distrobox enter mirror Container mirror is not running. Starting container mirror run this command to follow along: podman logs -f mirror Starting container... # Stuck here for 30 minutes or so lkocman@localhost:~/Workspace> ps aux | grep distro lkocman 5210 0.0 0.0 11096 3328 pts/0 S+ 08:13 0:00 /bin/sh /usr/bin/distrobox enter mirror lkocman 5212 2.1 0.0 11224 3840 pts/0 S+ 08:13 0:53 /bin/sh /usr/bin/distrobox-enter mirror lkocman 27424 0.0 0.0 8212 2304 pts/1 S+ 08:56 0:00 grep --color=auto distro Strace seems that the program itself is busy Not sure if this isn't an issue rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=27782, si_uid=1000, si_status=0, si_utime=2, si_stime=0} --- wait4(-1, 0x7ffc3cd5bbd0, WNOHANG, NULL) = -1 ECHILD (No child processes) rt_sigreturn({mask=[]}) = 0 openat(AT_FDCWD, "/home/lkocman/.cache/.mirror.fifo", O_RDONLY) = 3 fcntl(0, F_GETFD) = 0 fcntl(0, F_DUPFD, 10) = 10 fcntl(0, F_GETFD) = 0 fcntl(10, F_SETFD, FD_CLOEXEC) = 0 dup2(3, 0) = 0 close(3) = 0 ioctl(0, TCGETS, 0x7ffc3cd5c4f0) = -1 ENOTTY (Inappropriate ioctl for device) -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c1 Lubos Kocman <lubos.kocman@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|Normal |Major --- Comment #1 from Lubos Kocman <lubos.kocman@suse.com> --- I have seen this behavior on two 15.5 laptops as well. I have not noticed it until now on my 15.6 installation, and I use distrobox on daily basis, but I mostly re-use my existing 4 or similar contianers, and rarely create news ones. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c2 --- Comment #2 from Lubos Kocman <lubos.kocman@suse.com> --- Workaround: Killing the process and entering again seem to just work. Creating 'mirror' using image registry.opensuse.org/opensuse/distrobox:latest [ OK ] Distrobox 'mirror' successfully created. To enter, run: distrobox enter mirror Container mirror is not running. Starting container mirror run this command to follow along: podman logs -f mirror Starting container... ^C lkocman@localhost:~/Workspace> distrobox enter mirror lkocman@mirror:~/Workspace> -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 Dirk Mueller <dmueller@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|containers-bugowner@suse.de |dfaggioli@suse.com -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c3 --- Comment #3 from Lubos Kocman <lubos.kocman@suse.com> --- Interestingly it seems to happen also for existing distrobox containers lkocman@localhost:/etc/openvpn> distrobox enter osc Container osc is not running. Starting container osc run this command to follow along: podman logs -f osc Starting container... ^C lkocman@localhost:/etc/openvpn> distrobox enter osc lkocman@osc:/run/host/etc/openvpn> -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c4 Danish Prakash <danish.prakash@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |danish.prakash@suse.com --- Comment #4 from Danish Prakash <danish.prakash@suse.com> --- Hi Lubos, which podman and distrobox version are you running? I came across this issue upstream[1][2] that has the fixes merged and discussed. Given you're running Leap, can you try configuring your podman logging driver to "file" in containers.conf? [1] - https://github.com/89luca89/distrobox/issues/286 [2] - https://github.com/89luca89/distrobox/issues/287 -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c5 --- Comment #5 from Dario Faggioli <dfaggioli@suse.com> --- (In reply to Danish Prakash from comment #4)
Hi Lubos, which podman and distrobox version are you running? I came across this issue upstream[1][2] that has the fixes merged and discussed. Given you're running Leap, can you try configuring your podman logging driver to "file" in containers.conf?
[1] - https://github.com/89luca89/distrobox/issues/286 [2] - https://github.com/89luca89/distrobox/issues/287
Leap seems to have distrobox 1.4.2, which should have the fixes. But, yes, it's worth checking if the problem could be related to the podman logging configuration. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c6 --- Comment #6 from Lubos Kocman <lubos.kocman@suse.com> --- lkocman@localhost:/etc/openvpn> rpm -qa | grep distrobox distrobox-bash-completion-1.5.0.2-bp156.1.2.noarch distrobox-1.5.0.2-bp156.1.2.noarch -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c7 --- Comment #7 from Lubos Kocman <lubos.kocman@suse.com> --- Confirming that it still happens lkocman@localhost:/etc/openvpn> distrobox enter tw Container tw is not running. Starting container tw run this command to follow along: podman logs -f tw Starting container... -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c8 Dario Faggioli <dfaggioli@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lubos.kocman@suse.com Flags| |needinfo?(lubos.kocman@suse | |.com) --- Comment #8 from Dario Faggioli <dfaggioli@suse.com> --- So, I just installed a Tumbleweed VM, and it seems to work there:
dario@localhost:~> distrobox enter test Error: no such container test Creating the container test Trying to pull registry.opensuse.org/opensuse/distrobox:latest... Getting image source signatures Copying blob 3f501f5dfd08 done | Copying blob 2c46e69ca941 done | Copying config 98f2e40d0f done | Writing manifest to image destination 98f2e40d0f804ddcf8a55d3e8e8905f46bf6c439ead61ac0ed17b4ca02e8cd0f Creating 'test' using image registry.opensuse.org/opensuse/distrobox:latest [ OK ] Distrobox 'test' successfully created. To enter, run:
distrobox enter test
Starting container... [ OK ] Installing basic packages... [ OK ] Setting up devpts mounts... [ OK ] Setting up read-only mounts... [ OK ] Setting up read-write mounts... [ OK ] Setting up host's sockets integration... [ OK ] Integrating host's themes, icons, fonts... [ OK ] Setting up package manager exceptions... [ OK ] Setting up rpm exceptions... [ OK ] Setting up distrobox profile... [ OK ] Setting up sudo... [ OK ] Setting up groups... [ OK ] Setting up users... [ OK ] Setting up skel... [ OK ]
Container Setup Complete! dario@test:~>
Version of distrobox in Tumbleweed is the latest:
dario@localhost:~> distrobox --version distrobox: 1.6.0.1
In /usr/share/containers/containers.conf I have
# Logging driver for the container. Available options: k8s-file and journald. # log_driver = "journald"
So, this seems to be either an old distrobox issue or an (old?) podman issue. I'm not sure when I'll be able to spend time debugging distrobox 1.4 or 1.5 on Leap. In fact, if we're still in time (are we, Lubos?), we probably should push 1.6.0.1 to, at least, Leap 15.6. If it's podman, then I don't really know... :-( -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c9 Lubos Kocman <lubos.kocman@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(lubos.kocman@suse | |.com) | --- Comment #9 from Lubos Kocman <lubos.kocman@suse.com> --- lkocman@localhost:/etc/openvpn> grep log_driver /usr/share/containers/containers.conf log_driver = "journald" lkocman@localhost:/etc/openvpn> rpm -q podman distrobox podman-4.7.2-150600.1.3.x86_64 distrobox-1.5.0.2-bp156.1.2.noarch If we could update distrobox to 1.6, that would be great. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c10 --- Comment #10 from Lubos Kocman <lubos.kocman@suse.com> --- https://build.opensuse.org/request/show/1132491 -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c11 Lubos Kocman <lubos.kocman@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |needinfo?(dfaggioli@suse.co | |m) CC| |dfaggioli@suse.com --- Comment #11 from Lubos Kocman <lubos.kocman@suse.com> --- lkocman@localhost:~> distrobox enter tw + version=1.5.0.2 + '[' 2 -eq 0 ']' ++ dirname /usr/bin/distrobox + distrobox_path=/usr/bin + distrobox_command=enter + shift + case "${distrobox_command}" in + /usr/bin/distrobox-enter tw Container tw is not running. Starting container tw run this command to follow along: podman logs -f tw Starting container... # still the same ... but isn't this the problem? lkocman@localhost:~> podman logs -f tw Error: using --follow with the journald --log-driver but without the journald --events-backend (file) is not supported lkocman@localhost:~> -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c12 --- Comment #12 from Lubos Kocman <lubos.kocman@suse.com> --- I see that people complained about this before https://github.com/containers/podman/issues/18987 -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c13 Dario Faggioli <dfaggioli@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(dfaggioli@suse.co | |m) | --- Comment #13 from Dario Faggioli <dfaggioli@suse.com> --- (In reply to Lubos Kocman from comment #12)
I see that people complained about this before https://github.com/containers/podman/issues/18987
Yeah, they also mention:
As the error says this is not supported you have to change your event logger to journald or the container logger to k8s-file. You can set this in containers.conf, see log_driver and events_logger.
Which is what I meant when I said it might be a podman issue. distrobox should have a workaround for that, and I honestly don't see why it would not work... But, anyway, let's first check how things are with the newest release of distrobox in place. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c14 --- Comment #14 from Lubos Kocman <lubos.kocman@suse.com> --- I do agree Dario. I do have an update request for docker too, I'll check if there are any requests for SLES (aside for SLEM) to update docker. The problem that I see is that next container host (SLEM 6.X) is already ALP based, so there is an extra effort to ensure updates land also in codestream 15. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c15 --- Comment #15 from Lubos Kocman <lubos.kocman@suse.com> --- https://jira.suse.com/browse/PED-7483 -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c16 --- Comment #16 from Lubos Kocman <lubos.kocman@suse.com> --- Interestingly this issue still happens on 15.6 Beta (I'm using podman) lkocman@localhost:~> rpm -qa | egrep "distrobox|podman|docker|container" libcontainers-common-20230913-150600.1.1.noarch distrobox-1.6.0.1-bp156.1.1.noarch docker-24.0.5_ce-150600.200.47.x86_64 podman-4.7.2-150600.1.37.x86_64 docker-bash-completion-24.0.5_ce-150600.200.47.noarch containerd-1.7.10-150000.106.1.x86_64 libcontainers-default-policy-20230913-150600.1.1.noarch docker-buildx-0.11.2-bp156.1.8.x86_64 docker-rootless-extras-24.0.5_ce-150600.200.47.noarch distrobox-bash-completion-1.6.0.1-bp156.1.1.noarch lkocman@localhost:~> distrobox enter tw Starting container... -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c17 Alexandre Vicenzi <alexandre.vicenzi@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |alexandre.vicenzi@suse.com --- Comment #17 from Alexandre Vicenzi <alexandre.vicenzi@suse.com> --- I believe this is related to log driver Bug 1199871. I can still reproduce distrobox creation getting stuck. While digging it for Bug 1221278 I noticed it was stuck due to a failed attempt to follow the log to check if the container had started/finished setting up. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c18 --- Comment #18 from Lubos Kocman <lubos.kocman@suse.com> --- Not on my machine any more :-) Or at lest I do get different error, but it's no longer stuck. So imho this is resolved, and we should open a new issue. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c19 Alexandre Vicenzi <alexandre.vicenzi@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |needinfo?(lubos.kocman@suse | |.com) --- Comment #19 from Alexandre Vicenzi <alexandre.vicenzi@suse.com> --- (In reply to Lubos Kocman from comment #18)
Not on my machine any more :-) Or at lest I do get different error, but it's no longer stuck. So imho this is resolved, and we should open a new issue.
Which OS, podman, and distrobox version you have? I can re-test to ensure it got fixed since the last time I tried. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c20 Lubos Kocman <lubos.kocman@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(lubos.kocman@suse | |.com) | --- Comment #20 from Lubos Kocman <lubos.kocman@suse.com> --- Leap 15.6 / x86_64 lkocman@localhost:~> rpm -q distrobox podman docker distrobox-1.7.2.1-bp156.1.1.noarch podman-4.8.3-150500.3.9.1.x86_64 package docker is not installed -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1217306 https://bugzilla.suse.com/show_bug.cgi?id=1217306#c21 Alexandre Vicenzi <alexandre.vicenzi@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |WORKSFORME Status|NEW |RESOLVED --- Comment #21 from Alexandre Vicenzi <alexandre.vicenzi@suse.com> --- (In reply to Lubos Kocman from comment #18)
Not on my machine any more :-) Or at lest I do get different error, but it's no longer stuck. So imho this is resolved, and we should open a new issue.
I'm not able to reproduce it as well. Closing this as suggested. Feel free to create a new issue if you see fit. -- You are receiving this mail because: You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠suse.com