[opensuse-factory] Systemd regression in recent snapshot?
I'm sorry to bother the list with this but I didn't get any response on the forum. I believe there is a regression in the recent systemd update. I installed tumbleweed fresh about two weeks ago on the then-current snapshot, and everything was working great, until a few days ago when I updated. When systemd was updated, it didn't seem to restart. (243/531) Installing: systemd-232-2.1.x86_64 .......................................[done] Additional rpm output: Failed to reload daemon: Failed to activate service 'org.freedesktop.systemd1': timed out ... (274/531) Installing: udev-232-2.1.x86_64 ..........................................[done] Additional rpm output: Failed to reload daemon: Failed to activate service 'org.freedesktop.systemd1': timed out Failed to reload daemon: Failed to activate service 'org.freedesktop.systemd1': timed out Failed to try-restart systemd-udevd-control.socket: Connection timed out See system logs and 'systemctl status systemd-udevd-control.socket' for details. Failed to try-restart systemd-udevd-kernel.socket: Failed to activate service 'org.freedesktop.systemd1': timed out See system logs and 'systemctl status systemd-udevd-kernel.socket' for details. Failed to try-restart systemd-udevd.service: Failed to activate service 'org.freedesktop.systemd1': timed out See system logs and 'systemctl status systemd-udevd.service' for details. I was then unable to shutdown, and when I restarted the boot hung after USB device detection. After a period of time I saw messages: [103s] [sytemd-journald] Failed to send WATCHDOG=1 notification message: Connection refused [223s] ]systemd-journald] Failed to send WATCHDOG=1 notification message: Transport endpoint is not connected (Here's a screenshot: http://i.imgur.com/w4iSS3l.jpg) I then tried the 2017-02-06 snapshot (and again tonight I tried the 02-07 snapshot) and both result in the same message and unsuccessful boot. I initially believed this issue to be related to recent Xorg or AMDGPU issues but blacklisting both radeon and AMDGPU and setting runlevel to 3, all results in the same issue. I thought perhaps I could boot the rescue disk and chroot, then downgrade the systemd package with zypper. But, I'm also unable to boot the rescue system on the snapshot. It hangs at the message: "Starting udev Wait for Complete Device Initialization..." So, I would like to enter a bug, but I'm unsure of how to troubleshoot this any further, or get back to a booting system. Thanks for any help! -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Hi, On 02/09/2017 05:00 AM, Jason DeRose wrote:
I'm sorry to bother the list with this but I didn't get any response on the forum.
I believe there is a regression in the recent systemd update.
I installed tumbleweed fresh about two weeks ago on the then-current snapshot, and everything was working great, until a few days ago when I updated. When systemd was updated, it didn't seem to restart.
(243/531) Installing: systemd-232-2.1.x86_64 .......................................[done] Additional rpm output: Failed to reload daemon: Failed to activate service 'org.freedesktop.systemd1': timed out
It might have happened if systemd was already dead before the package update... It would have been nice to look at the content of the journal otherwise it's pretty hard to guess why systemd crashed. This might happen if your system was running out of memory.
I was then unable to shutdown, and when I restarted the boot hung after USB device detection. After a period of time I saw messages:
[103s] [sytemd-journald] Failed to send WATCHDOG=1 notification message: Connection refused [223s] ]systemd-journald] Failed to send WATCHDOG=1 notification message: Transport endpoint is not connected
It looks like systemd is dead again.
I then tried the 2017-02-06 snapshot (and again tonight I tried the 02-07 snapshot) and both result in the same message and unsuccessful boot. I initially believed this issue to be related to recent Xorg or AMDGPU issues but blacklisting both radeon and AMDGPU and setting runlevel to 3, all results in the same issue.
I would suggest to: - try to boot with the debug logs enabled (append "debug printk.devkmsg=on to the kernel command line via the grub menu) and try to find something relevant - try also to boot with "systemd.confirm_spawn=1" (append it to the kernel command line and keep the debug logs enabled). This will allow to start all services interactively during the boot process. This might allow you to locate the failure more easily. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Thu, Feb 9, 2017, at 03:58 AM, Franck Bui wrote:
I would suggest to:
- try to boot with the debug logs enabled (append "debug printk.devkmsg=on to the kernel command line via the grub menu) and try to find something relevant
- try also to boot with "systemd.confirm_spawn=1" (append it to the kernel command line and keep the debug logs enabled). This will allow to start all services interactively during the boot process. This might allow you to locate the failure more easily.
I am getting some additional output now. Systemd does seem to start initially, but either it or plymouth seems to be dying: http://imgur.com/uo9T3Ev.jpg Additional output: Core dump to /usr/lib/systemd/systemd-coredump ... plymouthd pipe failed Is there some way to pause after each page of output so that I can examine the output that has scrolled off the screen? Also, is there any way to have my keyboard load sooner? I have a USB keyboard that doesn't seem to load until late in the boot process and I can't respond to the prompt until then, having to wait until they time out. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Hello! 09.02.2017 16:39, Jason DeRose пишет:
On Thu, Feb 9, 2017, at 03:58 AM, Franck Bui wrote:
I would suggest to:
- try to boot with the debug logs enabled (append "debug printk.devkmsg=on to the kernel command line via the grub menu) and try to find something relevant
- try also to boot with "systemd.confirm_spawn=1" (append it to the kernel command line and keep the debug logs enabled). This will allow to start all services interactively during the boot process. This might allow you to locate the failure more easily. I am getting some additional output now. Systemd does seem to start initially, but either it or plymouth seems to be dying:
http://imgur.com/uo9T3Ev.jpg Additional output: Core dump to /usr/lib/systemd/systemd-coredump ... plymouthd pipe failed
Does it similar to https://bugzilla.opensuse.org/show_bug.cgi?id=1012142 ? -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Thu, Feb 9, 2017, at 09:48 AM, Mikhail Kasimov wrote:
09.02.2017 16:39, Jason DeRose пишет:
On Thu, Feb 9, 2017, at 03:58 AM, Franck Bui wrote:
I would suggest to:
- try to boot with the debug logs enabled (append "debug printk.devkmsg=on to the kernel command line via the grub menu) and try to find something relevant
- try also to boot with "systemd.confirm_spawn=1" (append it to the kernel command line and keep the debug logs enabled). This will allow to start all services interactively during the boot process. This might allow you to locate the failure more easily. I am getting some additional output now. Systemd does seem to start initially, but either it or plymouth seems to be dying:
http://imgur.com/uo9T3Ev.jpg Additional output: Core dump to /usr/lib/systemd/systemd-coredump ... plymouthd pipe failed
Does it similar to https://bugzilla.opensuse.org/show_bug.cgi?id=1012142 ?
I don't think so - the output is different. The more I mess with it I don't actually think it's plymouth. I can skip plymouthd in the interactive boot mode and it doesn't seem to make an difference. The system still won't boot. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 02/09/2017 03:39 PM, Jason DeRose wrote:
Did you enable the debug log ("debug printk.devkmsg=on") ?
Additional output: Core dump to /usr/lib/systemd/systemd-coredump ... plymouthd pipe failed
It will be simpler if you disable plymouth (add "plymouth.enable=0" to the kernel command line too IIRC)
Is there some way to pause after each page of output so that I can examine the output that has scrolled off the screen?
Not that I'm aware off. If you have a serial port you can try the serial console. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Fri, Feb 10, 2017, at 02:34 AM, Franck Bui wrote:
On 02/09/2017 03:39 PM, Jason DeRose wrote:
Did you enable the debug log ("debug printk.devkmsg=on") ?
Additional output: Core dump to /usr/lib/systemd/systemd-coredump ... plymouthd pipe failed
It will be simpler if you disable plymouth (add "plymouth.enable=0" to the kernel command line too IIRC)
Is there some way to pause after each page of output so that I can examine the output that has scrolled off the screen?
Not that I'm aware off.
If you have a serial port you can try the serial console.
Yes I enabled the debug log. The extra output is cleared though after it starts detecting USB devices. If I enable the spawn prompts I can capture more of it. Here is everything I was able to capture: http://i.imgur.com/Bvj5svh.jpg http://i.imgur.com/Xz1i2aA.jpg http://i.imgur.com/Y736RFv.jpg Disabling plymouth with the kernel command line does not make any difference in regard to allowing the system to boot. Neither does blacklisting amdgpu and radeon drivers and setting runlevel to 3. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
participants (3)
-
Franck Bui
-
Jason DeRose
-
Mikhail Kasimov