[Bug 787228] New: systemd: kexec is stuck after network shutdown
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c0 Summary: systemd: kexec is stuck after network shutdown Classification: openSUSE Product: openSUSE 12.2 Version: Final Platform: Other OS/Version: Other Status: NEW Severity: Normal Priority: P5 - None Component: Basesystem AssignedTo: fcrozat@suse.com ReportedBy: jslaby@suse.com QAContact: qa-bugs@suse.de Found By: --- Blocker: --- When I try to kexec to a new kernel, most of time, systemd gets stuck waiting for something infinitely. It usually happens after vpn and network is shut down. Nothing happens if I do sysrq-e, sysrq-i (send TERM and KILL to everybody). I can only press ctrl-alt-del. But then system will reboot instead of kexec. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c1 --- Comment #1 from Frederic Crozat <fcrozat@suse.com> 2012-11-26 13:08:21 UTC --- I could be interesting to boot with systemd.log_level=debug systemd.log_target=console to see what is going on. I can't reproduce the issue, even after enabling VPN.. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c2 --- Comment #2 from Jiri Slaby <jslaby@suse.com> 2012-12-07 14:03:10 UTC --- (In reply to comment #1)
I could be interesting to boot with systemd.log_level=debug systemd.log_target=console to see what is going on.
I can't reproduce the issue, even after enabling VPN..
It looks like not bound to VPN. It spits out that there is a dependency problem with some service. Should kill -54 1 kill -59 1 do the job above at runtime? As it doesn't seem to have any effect? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c3 --- Comment #3 from Frederic Crozat <fcrozat@suse.com> 2012-12-07 14:13:56 UTC --- hmm, according to documentation and if I still know how to count: debug => kill -56 1 console output => kill -61 1 -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c4 --- Comment #4 from Jiri Slaby <jslaby@suse.com> 2012-12-07 18:27:48 UTC --- (In reply to comment #3)
hmm, according to documentation and if I still know how to count: debug => kill -56 1
SIGRTMIN+22 /usr/include/asm/signal.h: ... #define SIGRTMIN 32 32+22=54
console output => kill -61 1
SIGRTMIN+27 27+32=59 Right? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c5 --- Comment #5 from Jiri Slaby <jslaby@suse.com> 2012-12-07 18:37:24 UTC --- (In reply to comment #4)
Right?
Nope, got it now. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c6 --- Comment #6 from Jiri Slaby <jslaby@suse.com> 2013-01-09 11:53:08 UTC --- This happens also with poweroff. However I never recall to enable debug+console before kexec/poweroff. Anyway, what I see now is ntpd cannot be stopped for some reason (Stopping ntpd .. [FAILED]) and systemd says it is a poweroff dependency failure. Then network is shut down and it blocks. I'll try to remeber to enable debug+console next time. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c7 --- Comment #7 from Jiri Slaby <jslaby@suse.com> 2013-01-12 20:19:50 UTC --- Created an attachment (id=520032) --> (http://bugzilla.novell.com/attachment.cgi?id=520032) screen when stuck This is how it looks. ntpd looks suspicious there... -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c8 --- Comment #8 from Jiri Slaby <jslaby@suse.com> 2013-02-04 15:37:26 UTC --- (In reply to comment #7)
This is how it looks. ntpd looks suspicious there...
It's not ntpd. When I disable it the issue still occurs. It happens only when network connection is activated. Then any of kexec, poweroff and reboot gets stuck. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c9 --- Comment #9 from Jiri Slaby <jslaby@suse.com> 2013-02-04 15:38:26 UTC --- Forgot to add that there is a plenty of messages when debug is enabled and nothing relevant in there. What should I be looking for? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c10 --- Comment #10 from Frederic Crozat <fcrozat@suse.com> 2013-02-04 16:18:45 UTC --- could you try to do the procedure described in http://freedesktop.org/wiki/Software/systemd/Debugging#Diagnosing_Shutdown_P... to get a full trace of what is going on ? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c11 --- Comment #11 from Jiri Slaby <jslaby@suse.com> 2013-02-04 16:22:29 UTC --- (In reply to comment #10)
could you try to do the procedure described in http://freedesktop.org/wiki/Software/systemd/Debugging#Diagnosing_Shutdown_P... to get a full trace of what is going on ?
Ok. Regarding the first step in there, CTRL+ALT+DEL forces reboot to proceed even if it was stuck. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c12 --- Comment #12 from Jiri Slaby <jslaby@suse.com> 2013-02-04 16:54:34 UTC --- Created an attachment (id=523233) --> (http://bugzilla.novell.com/attachment.cgi?id=523233) debug log of hung reboot (In reply to comment #10)
to get a full trace of what is going on ?
Here you go. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c13 --- Comment #13 from Frederic Crozat <fcrozat@suse.com> 2013-02-04 17:16:03 UTC --- in the attached trace, did you cancel the shutdown or anything like that ? There is something suspiscious : [49911.355122] systemd[1]: sys-devices-virtual-net-tun0.device changed plugged -> dead [49911.362057] systemd[1]: Accepted connection on private bus. [49911.362625] systemd[1]: Got D-Bus request: org.freedesktop.systemd1.Manager.RestartUnit() on /org/freedesktop/systemd1 [49911.362652] systemd[1]: Trying to enqueue job dnsmasq.service/restart/replace [49911.362999] systemd[1]: Installed new job dnsmasq.service/restart as 1925 [49911.363007] systemd[1]: Job dbus.socket/stop finished, result=canceled [49911.363016] systemd[1]: Installed new job dbus.socket/start as 1928 [49911.363022] systemd[1]: Job sysinit.target/stop finished, result=canceled [49911.363031] systemd[1]: Installed new job sysinit.target/start as 1929 [49911.363037] systemd[1]: Job local-fs.target/stop finished, result=canceled [49911.363043] systemd[1]: Installed new job local-fs.target/start as 1930 [49911.363048] systemd[1]: Job boot-efi.mount/stop finished, result=canceled [49911.363054] systemd[1]: Installed new job boot-efi.mount/start as 1931 [49911.363060] systemd[1]: Installed new job fsck@dev-disk-by\x2did-ata\x2dINTEL_SSDSA2M080G2GC_CVPO0175040N080JGN\x2dpart1.service/start as 1932 [49911.363066] systemd[1]: Job umount.target/start finished, result=canceled [49911.363074] systemd[1]: Job reboot.service/start finished, result=dependency [49911.363390] systemd[1]: Job reboot.target/start finished, result=dependency [49911.363399] systemd[1]: Job reboot.target/start failed with result 'dependency'. [49911.363404] systemd[1]: Job reboot.service/start failed with result 'dependency'. It looks like turning off the vpn is restarting dnsmasq (which was already off), which is restarting a number of services. Could you try replacing "/etc/init.d/dnsmasq restart" by "/etc/init.d/dnsmasq try-restart" in /etc/openvpn/client.down ? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c14 --- Comment #14 from Jiri Slaby <jslaby@suse.com> 2013-02-04 21:05:24 UTC --- (In reply to comment #13)
in the attached trace, did you cancel the shutdown or anything like that ?
No, no, it is what it does w/o my intervention.
It looks like turning off the vpn is restarting dnsmasq (which was already off), which is restarting a number of services.
Could you try replacing "/etc/init.d/dnsmasq restart" by "/etc/init.d/dnsmasq try-restart" in /etc/openvpn/client.down ?
Yeah, that fixed it. Should we update this document: https://wiki.innerweb.novell.com/index.php/Services_Team/Policies/openVPN/cl... ? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c15 Frederic Crozat <fcrozat@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution| |FIXED --- Comment #15 from Frederic Crozat <fcrozat@suse.com> 2013-02-05 08:33:49 UTC --- (In reply to comment #14)
(In reply to comment #13)
It looks like turning off the vpn is restarting dnsmasq (which was already off), which is restarting a number of services.
Could you try replacing "/etc/init.d/dnsmasq restart" by "/etc/init.d/dnsmasq try-restart" in /etc/openvpn/client.down ?
Yeah, that fixed it. Should we update this document: https://wiki.innerweb.novell.com/index.php/Services_Team/Policies/openVPN/cl... ?
Done. closing as "fixed" -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c16 --- Comment #16 from Jiri Slaby <jslaby@suse.com> 2013-02-05 08:37:06 UTC --- Neat, so I can reboot after a half year :). Thanks. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c17 --- Comment #17 from Frederic Crozat <fcrozat@suse.com> 2013-02-22 15:51:19 UTC --- just for the record, upstream has just fixed this issue properly, by creating transactions which can't be cancelled automatically (only with a command), for stuff like reboot, shutdown : a service being started at shutdown would no longer stop reboot transaction.. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c18 --- Comment #18 from Jiri Slaby <jslaby@suse.com> 2013-04-02 15:05:48 UTC --- *** Bug 812541 has been marked as a duplicate of this bug. *** http://bugzilla.novell.com/show_bug.cgi?id=812541 -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c19 Joschi Brauchle <joschibrauchle@gmx.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |joschibrauchle@gmx.de --- Comment #19 from Joschi Brauchle <joschibrauchle@gmx.de> 2013-04-12 14:41:54 UTC --- Just a quick question: will there be or has there been update for 12.3 including this fix? I have just had a situation where this bug prevented a reboot on ~30 machines :-( -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=787228 https://bugzilla.novell.com/show_bug.cgi?id=787228#c20 --- Comment #20 from Frederic Crozat <fcrozat@suse.com> 2013-04-12 14:54:00 UTC --- (In reply to comment #19)
Just a quick question: will there be or has there been update for 12.3 including this fix? I have just had a situation where this bug prevented a reboot on ~30 machines :-(
No, it can't be backported. You have to find which service is being started at shutdown and prevent that. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠novell.com