https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c zj jia changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |zjjia@suse.com AssignedTo|bnc-team-screening@forge.pr |werner@suse.com |ovo.novell.com | -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c2 Danny van Delft changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |NEW InfoProvider|d.a.van.delft@gmail.com | --- Comment #2 from Danny van Delft 2011-11-15 09:38:42 UTC --- I wish I knew, but can't determine. I tried with running a top first on another VT than the one I initiate the reboot. Although I could switch back to the VT with top, it gave no usable output at the hang. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c4 Danny van Delft changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |NEW InfoProvider|d.a.van.delft@gmail.com | --- Comment #4 from Danny van Delft 2011-11-15 10:56:42 UTC --- Gives no output whatsoever, even if explicitly stdout redirected to /dev/tty10. Looks like this place is not reached during halt. I've done some other experimenting, by enabling sysrq. Probably due to keyboard issues in combination with vmware, I can't trigger the sysrq from the keyboard, but with something like while : ; do echo t > /proc/sysrq-trigger;usleep 100000;done at least some output appears on tty10. Helas, at the moment of the hang, the net result is only the header, like .. SysRq : Show State is shown. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c6 Danny van Delft changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |NEW InfoProvider|d.a.van.delft@gmail.com | --- Comment #6 from Danny van Delft 2011-11-15 11:17:59 UTC --- Hold on, something interesting. If booted with kernel-default (i.s.o. kernel-desktop), the same hang occurs at shutdown, but at least your ps lines in /etc/init.d/halt give some output after a while. Initially it looks as if the machine hangs at the same point, with the high CPU usage, but after a couple of seconds the ps output appears. After the ps loop of 60s output stops, the machine reboots. So the problem only occurs with kernel-desktop, even though with kernel-default it still has a period of high CPU-activity. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c8 --- Comment #8 from Danny van Delft 2011-11-15 11:28:10 UTC --- (In reply to comment #5)

Hmmm ... or the redirect to /dev/console results in invisible output. Now try to add

set -x

after the line

set +e

in /etc/init.d/halt ... let us see what happens upto the very last message

Done the same with kernel-default, now different output appears. Looks like another codepath, which succeeds, is taken in comparison to kernel-desktop. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c10 --- Comment #10 from Danny van Delft 2011-11-15 12:41:10 UTC --- Yes, apart from comments /etc/init.d/halt.local is empty. In addition to comment 7 above, sometimes the shutdown does show some output from /etc/init.d/halt. However, it has never shutdown completely, as it does do when using kernel-default. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c12 --- Comment #12 from Danny van Delft 2011-11-15 15:00:44 UTC --- Could be that desktop-kernel is not usable for a VM. However when running under systemd shutdown goes well, even with kernel-desktop. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c14 --- Comment #14 from Danny van Delft 2011-11-15 16:21:24 UTC --- FYI, I tried the following: on a VT: sleep 60 ; echo e > /proc/sysrq-trigger ; sleep 10 ; echo i > /proc/sysrq-trigger ; sleep 10 ; echo b > /proc/sysrq-trigger on another V: reboot The system hung as "expected", but at least on the first VT the last echo did its work: the system rebooted. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c16 --- Comment #16 from Lars Müller 2011-12-04 16:54:52 CET --- Created an attachment (id=465781) --> (http://bugzilla.novell.com/attachment.cgi?id=465781) Output of the script started before calling reboot The script called this commands: #!/bin/bash test -e reboot_counter && \ mv reboot_counter reboot_counter.old reboot_counter=0 while [ 1 ]; do reboot_counter=$((reboot_counter+1)) echo -e "\nCOUNTER: ${reboot_counter}" >>reboot_counter ps xuaf >>reboot_counter sleep 1 done -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c18 --- Comment #18 from Lars Müller 2011-12-04 17:27:27 CET --- The reboot took 331 seconds. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c20 Dr. Werner Fink changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |NEEDINFO InfoProvider| |lmuelle@suse.com --- Comment #20 from Dr. Werner Fink 2011-12-05 08:00:19 UTC --- Guess: kernel 3.0 and higher cause trouble with blogd ... check how large/huge /var/log/boot.msg and /var/log/boot.omsg are. Also an strace of blogd would be interesting -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c23 Lars Müller changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |NEW InfoProvider|lmuelle@suse.com | --- Comment #23 from Lars Müller 2011-12-05 20:35:49 CET --- Created an attachment (id=465956) --> (http://bugzilla.novell.com/attachment.cgi?id=465956) strace of blogd while reboot -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c25 --- Comment #25 from Lars Müller 2011-12-06 11:40:20 CET --- The final message got displayed very quickly. In about 10 up to 15 seconds after I called to reboot. By reading the reported output again I noticed a difference I saw between different monitored reboots. Sometimes I did not saw the 'Running /etc/init.d/halt.local' output. I guess it got delayed too. But it might also never got printed. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c27 Dr. Werner Fink changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |NEEDINFO InfoProvider| |lmuelle@suse.com --- Comment #27 from Dr. Werner Fink 2012-03-01 08:49:50 UTC --- (In reply to comment #26) Lars? Why do you think this? A syou can see from my comment #24 this is a kernel bug. Or why does this never happen with a none VM system? I've several systems around here using SysVinit on 12.1 and those systems do *not* hang. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c29 Andreas Jaeger changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |matz@suse.com --- Comment #29 from Andreas Jaeger 2012-03-01 09:23:50 UTC --- Michael, could you have a look as well, please? Werner, you might want to ask on the kernel list for help. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c31 Michael Matz changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|Normal |Critical --- Comment #31 from Michael Matz 2012-03-01 14:52:11 UTC --- blogd is multithreaded, hence the above strace doesn't reveal much information as it contains only info from the main thread. Please do the strace again, this time with -f (and if you are at it also with -tt). With the information herein we can't blame the kernel or glibc. In particular we can't rule out that blogd doesn't simply contain a dead lock situation, that is only triggered under VMs (for whatever reasons, timing or otherwise). -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c33 Lars Müller changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |NEW InfoProvider|lmuelle@suse.com | --- Comment #33 from Lars Müller 2012-03-02 12:47:27 CET --- Remove need info till I got the information how to collect the data even on a remote system. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c35 --- Comment #35 from Matthias Pfafferodt 2012-03-03 15:43:48 UTC --- The vm's do not hang if the shutdown is done directly after they were started. But if I do an update and reboot them after several days of operation it happens. Kernel is after today 3.2.9-12-xen (from tumbleweed). -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c37 --- Comment #37 from Matthias Pfafferodt 2012-03-27 18:58:09 UTC --- I did reboot the VMs today and could not reproduce this. I will check this again if another reboot is needed. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c39 Matthias Pfafferodt changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |NEW InfoProvider|syntron@web.de | --- Comment #39 from Matthias Pfafferodt 2012-05-06 20:35:21 UTC --- I found a way to reproduce this bug: - start a computer with an nfs mount (I also have yp and autofs) - start the script 'nfs.sh' _on_ the nfs mount (I suspect it creates open file handles) - start the script 'nohup.sh' which calls 'blogd.sh' which calls strace on blogd on a real hard drive to create the log file - shutdown the computer and wait :-( As said above, I suspect open file handles on the nfs mount which case something like an endless loop Two examples of the strace output are attached. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c41 --- Comment #41 from Matthias Pfafferodt 2012-05-06 20:41:58 UTC --- Created an attachment (id=489696) --> (http://bugzilla.novell.com/attachment.cgi?id=489696) start the script blogd.sh with nohup -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c43 --- Comment #43 from Matthias Pfafferodt 2012-05-06 20:59:12 UTC --- Created an attachment (id=489698) --> (http://bugzilla.novell.com/attachment.cgi?id=489698) strace (run 1) the last two lines are repeated till I hard reset the computer -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c45 Dr. Werner Fink changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |NEEDINFO InfoProvider| |syntron@web.de --- Comment #45 from Dr. Werner Fink 2012-05-07 07:12:37 UTC --- (In reply to comment #39) For NFS shares as well as for any other network based shared filessystem: If the server of such an share is not reachable then any system call like e.g. stat(2) will run into a deadlock. I've added a lot of workarounds in the last few years to avoid system call calls like stat(2) but even by using readlink(2): if the kernel does not have any information cached it will ask the server of the share and this also lead to a deadlock. This has nothing todo with sysvinit its self. It may depend on the execution order at shuthdown (that is that all network connections should go down *after* the the network based share is unmounted). That is you should check the dependcy order of the boot scripts and enforece that any process on or with the network based shares are stopped *before* the network for those shares will switched off. The last loop you are seeing from the blogd in your strace logs are simply a waiting blogd for more input on /dev/console but it will wait a long time if nothing happens on /dev/console due to a system call waiting on a not eachable server of a network based share. IMHO the initial report does not mention a network based share. That is your report does not belong to this bug, does it? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c47 Dr. Werner Fink changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |NEEDINFO InfoProvider| |d.a.van.delft@gmail.com --- Comment #47 from Dr. Werner Fink 2012-05-07 11:54:21 UTC --- (In reply to comment #46) Please try out an update to latest sysvinit done for factory. It could be that you see a duplicate of bug #731563 -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c49 --- Comment #49 from Jeff Mahoney 2012-05-07 09:21:03 EDT --- (In reply to comment #45)

(In reply to comment #39)

For NFS shares as well as for any other network based shared filessystem:

If the server of such an share is not reachable then any system call like e.g. stat(2) will run into a deadlock. I've added a lot of workarounds in the last few years to avoid system call calls like stat(2) but even by using readlink(2): if the kernel does not have any information cached it will ask the server of the share and this also lead to a deadlock.

To be fair, this isn't an actual deadlock. It's a network dependency that will be met when the server comes back. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c51 --- Comment #51 from Matthias Pfafferodt 2012-05-07 19:36:39 UTC --- (In reply to comment #45)

(In reply to comment #39)

For NFS shares as well as for any other network based shared filessystem:

If the server of such an share is not reachable then any system call like e.g. stat(2) will run into a deadlock. I've added a lot of workarounds in the last few years to avoid system call calls like stat(2) but even by using readlink(2): if the kernel does not have any information cached it will ask the server of the share and this also lead to a deadlock.

This has nothing todo with sysvinit its self. It may depend on the execution order at shuthdown (that is that all network connections should go down *after* the the network based share is unmounted). That is you should check the dependcy order of the boot scripts and enforece that any process on or with the network based shares are stopped *before* the network for those shares will switched off.

This means that using nfs mounted home (nis and autofs) I will have to ensure that all processes accessing the nfs share are killed before doing a shutdown? The server was online at the time of the shutdown so this is not the reason. Thus, the order of the shutdown scripts should be changed? I have to say, that I do not understand how the boot scripts network-remotefs and network interact.

IMHO the initial report does not mention a network based share. That is your report does not belong to this bug, does it?

Yes, my last report does depend on using an nfs share and also in my first post all linux instances used to have an nfs share mounted. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c53 --- Comment #53 from Marius Tomaschewski 2012-05-08 12:48:24 UTC --- About network & network-remotefs -- the start is as follows: network -- base networking, without ppp, wlan, (NM, ) ... nfs & smb -- network filesystems on top of (base) network network-remotefs -- rest of the network stuff that may be on nfs-/usr including NetworkMananger (sysvinit only) <remotefs depending services> the stop is another way around. So when <remotefs depending services> dependencies are correct and the programs are stopped/cleaned up correctly (e.g. desktop services, dbus servers, ...) the shutdown should work fine... You can try to set some of these variables: $ grep ^CONNECTION /etc/sysconfig/network/config CONNECTION_SHOW_WHEN_IFSTATUS="no" CONNECTION_CHECK_BEFORE_IFDOWN="no" CONNECTION_CLOSE_BEFORE_IFDOWN="no" CONNECTION_UMOUNT_NFS_BEFORE_IFDOWN="no" CONNECTION_SEND_KILL_SIGNAL="no" (In reply to comment #51)

This means that using nfs mounted home (nis and autofs) I will have to ensure that all processes accessing the nfs share are killed before doing a shutdown?

Not exactly you, but the nfs/autofs/remomte-fs-depending... service scripts. So the question is, what is still running at the time nfs and network go down. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c55 --- Comment #55 from Ludwig Nussel 2012-06-21 09:31:00 CEST --- deadlock in blogd. One thread called pthread_yield() in a loop with a lock held while the main thread tried to acquire the lock to cancel the thread. You may want to test the fix in home:lnussel:branches:openSUSE:12.1:Update/sysvinit -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c57 --- Comment #57 from Ludwig Nussel 2012-06-21 13:29:02 CEST --- it got turned into a maintenance update already, it's now staged in openSUSE:Maintenance:567 -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c59 Bernhard Wiedemann changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bwiedemann@suse.com --- Comment #59 from Bernhard Wiedemann 2012-06-24 19:21:06 CEST --- I have seen the same sysvinit / blogd hang in Beta 2 / Factory today, so the fix will be needed there and for 12.2, too. And I found that omitting "killproc -SYS /sbin/blogd" in /etc/init.d/rc also helped. I guess this is one of those scripts that are ignored by systemd which explains why it does not hang there. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c61 --- Comment #61 from Bernhard Wiedemann 2012-06-25 11:00:10 CEST --- This is an autogenerated message for OBS integration: This bug (730193) was mentioned in https://build.opensuse.org/request/show/125924 Factory / sysvinit -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=730193 https://bugzilla.novell.com/show_bug.cgi?id=730193#c63 Swamp Workflow Management changed: What |Removed |Added ---------------------------------------------------------------------------- Status Whiteboard| |maint:released:sle11-sp2:48 | |038 --- Comment #63 from Swamp Workflow Management 2012-06-26 20:03:34 UTC --- Update released for: sysvinit, sysvinit-debuginfo, sysvinit-debugsource Products: SLE-DEBUGINFO 11-SP2 (i386, ia64, ppc64, s390x, x86_64) SLE-DESKTOP 11-SP2 (i386, x86_64) SLE-SERVER 11-SP2 (i386, ia64, ppc64, s390x, x86_64) SLES4VMWARE 11-SP2 (i386, x86_64) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.