[opensuse] What could cause "ps afx" to block? And "reboot" to block?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, it has happened to me a few times, each time I doing different things as far as I can remember. Suddenly a process hangs. CPU is about iddle, disk is iddling. But some processes do not start. A simple "ps afx" never finishes, and ^C it doesn't work. Subsequently, a reboot also hangs, does nothing. I have to hit the power button hard, more than 4 seconds. There was nothing in the log at the time. dmesg showed nothing different. However, looking at the /var/log/messages file now, I do see entries, happening when I try to reboot on console, seeing nothing, then hitting the power button. But no problem logged, as far as I can see. <3.6> 2015-09-28 13:27:19 minas-tirith dhclient 9038 - - DHCPACK from 192.168.42.129 <3.6> 2015-09-28 13:27:19 minas-tirith dhclient 9038 - - bound to 192.168.42.209 -- renewal in 1614 seconds. <3.6> 2015-09-28 13:27:19 minas-tirith avahi-daemon 1799 - - Joining mDNS multicast group on interface enp0s29f7u1.IPv4 with address 192.168.42.209. <3.6> 2015-09-28 13:27:19 minas-tirith avahi-daemon 1799 - - New relevant interface enp0s29f7u1.IPv4 for mDNS. <3.6> 2015-09-28 13:27:19 minas-tirith avahi-daemon 1799 - - Registering new address record for 192.168.42.209 on enp0s29f7u1.IPv4. <0.4> 2015-09-28 13:27:19 minas-tirith kernel - - - [64437.252819] perf samples too long (5049 > 5000), lowering kernel.perf_event_max_sample_rate to 25000 <3.5> 2015-09-28 13:27:20 minas-tirith dbus 1798 - - [system] Activating via systemd: service name='org.freedesktop.nm_dispatcher' unit='dbus-org.freedesktop.nm-dispatcher.service' <3.6> 2015-09-28 13:27:20 minas-tirith systemd 1 - - Starting Network Manager Script Dispatcher Service... <3.5> 2015-09-28 13:27:21 minas-tirith dbus 1798 - - [system] Successfully activated service 'org.freedesktop.nm_dispatcher' <3.6> 2015-09-28 13:27:21 minas-tirith systemd 1 - - Started Network Manager Script Dispatcher Service. <3.6> 2015-09-28 13:27:36 minas-tirith systemd 1 - - Time has been changed <3.6> 2015-09-28 13:28:13 minas-tirith dnsmasq 2073 - - reading /etc/resolv.conf <3.6> 2015-09-28 13:28:13 minas-tirith dnsmasq 2073 - - using nameserver 192.168.42.129#53 <3.6> 2015-09-28 13:28:13 minas-tirith dnsmasq 2073 - - using nameserver 8.8.4.4#53 <3.6> 2015-09-28 13:28:13 minas-tirith dnsmasq 2073 - - using nameserver 8.8.8.8#53 <3.6> 2015-09-28 13:27:39 minas-tirith systemd 1 - - message repeated 3 times: [ Time has been changed] <3.6> 2015-09-28 13:30:02 minas-tirith systemd 1 - - Starting Session 115 of user cer. <3.6> 2015-09-28 13:30:02 minas-tirith systemd 1 - - Started Session 115 of user cer. <3.6> 2015-09-28 13:30:02 minas-tirith systemd 1 - - Starting Session 114 of user root. <3.6> 2015-09-28 13:30:02 minas-tirith systemd 1 - - Started Session 114 of user root. ... (fetching mail happened about here) <3.6> 2015-09-28 13:41:13 minas-tirith systemd 1 - - Starting Session 116 of user root. <4.6> 2015-09-28 13:41:13 minas-tirith systemd-logind 1820 - - New session 116 of user root. <3.6> 2015-09-28 13:41:13 minas-tirith systemd 1 - - Started Session 116 of user root. <10.5> 2015-09-28 13:41:13 minas-tirith login - - - ROOT LOGIN ON tty1 <3.6> 2015-09-28 13:41:35 minas-tirith systemd 1 - - Starting Getty on tty2... <3.6> 2015-09-28 13:41:35 minas-tirith systemd 1 - - Started Getty on tty2. <10.5> 2015-09-28 13:41:41 minas-tirith login - - - FAILED LOGIN 1 FROM tty2 FOR root, Authentication failure <10.5> 2015-09-28 13:41:48 minas-tirith login - - - FAILED LOGIN 2 FROM tty2 FOR root, Authentication failure <3.6> 2015-09-28 13:41:59 minas-tirith systemd 1 - - Starting Session 117 of user root. <3.6> 2015-09-28 13:41:59 minas-tirith systemd 1 - - Started Session 117 of user root. <4.6> 2015-09-28 13:41:59 minas-tirith systemd-logind 1820 - - New session 117 of user root. <10.5> 2015-09-28 13:41:59 minas-tirith login - - - ROOT LOGIN ON tty2 <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping /sys/devices/virtual/block/dm-0. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - message repeated 5 times: [ Stopping /sys/devices/virtual/block/dm-0.] <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Session 117 of user root. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopped Session 117 of user root. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Session 116 of user root. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopped Session 116 of user root. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Session 3 of user cer. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopped Session 3 of user cer. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Session 1 of user lightdm. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopped Session 1 of user lightdm. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping user-494.slice. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Removed slice user-494.slice. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Sound Card. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopped target Sound Card. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Stop Read-Ahead Data Collection 10s After Completed Startup. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopped Stop Read-Ahead Data Collection 10s After Completed Startup. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Disk Manager... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Daemon for power management... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping RealtimeKit Scheduling Policy Service... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping User Manager for 1000... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Console Manager... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping User Manager for 0... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Accounts Service... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Bluetooth service... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Authorization Manager... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Graphical Interface. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopped target Graphical Interface. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping LSB: X Display Manager... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Plays a welcome sound when target multi-user is reached... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopped Plays a welcome sound when target multi-user is reached. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Multi-User System. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopped target Multi-User System. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping LSB: Start the hddtemp daemon... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping LSB: irqbalance daemon providing irq balancing on MP-machines... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping LSB: Supports the direct execution of binary formats.... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Command Scheduler... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping CUPS Printing Service... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Initialize hardware monitoring sensors... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping OpenSSH Daemon... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Avahi mDNS/DNS-SD Stack... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Daemonized version of spamassassin... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Xinetd A Powerful Replacement For Inetd... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Execution Queue Daemon... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Self Monitoring and Reporting Technology (SMART) Daemon... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping System Logging Service... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping SuSEfirewall2 phase 2... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Dovecot IMAP/POP3 email server... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Modem Manager... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Login Service... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping /etc/init.d/after.local Compatibility... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopped /etc/init.d/after.local Compatibility. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Login Prompts. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopped target Login Prompts. <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Getty on tty2... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Stopping Getty on tty1... <3.6> 2015-09-28 13:42:46 minas-tirith systemd 1 - - Starting Store Sound Card State... <3.6> 2015-09-28 13:39:57 minas-tirith xinetd 2048 - - EXIT: nntp status=0 duration=573(sec) <3.4> 2015-09-28 13:42:46 minas-tirith xinetd 2048 - - Exiting... <5.6> 2015-09-28 13:42:50 minas-tirith rsyslogd - - - [origin software="rsyslogd" swVersion="7.4.7" x-pid="9340" x-info="http://www.rsyslog.com"] exiting on signal 15. 2015-09-28 13:42:52+02:00 - Halting the system now =========================================== uptime: 13:42pm up 3 days 10:25, 31 users, load average: 25.31, 14.49, 7.18 2015-09-28 13:45:57+02:00 - Booting the system now ================================================================================ Linux minas-tirith 3.11.10-29-desktop #1 SMP PREEMPT Thu Mar 5 16:24:00 UTC 2015 (338c513) x86_64 x86_64 x86_64 GNU/Linux <5.6> 2015-09-28 13:45:58 minas-tirith rsyslogd - - - [origin software="rsyslogd" swVersion="7.4.7" x-pid="1524" x-info="http://www.rsyslog.com"] start This is a laptop running current 13.1. I restored from hibernation about 28 minutes before. - -- Cheers Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlYJLw4ACgkQja8UbcUWM1zydQD/WFQsbQqwcvQepQ9m2/L/PyFg ssKsUSyBSWImlbYMIucA/30qigEI/cigjCry+zISO1enrRBcr/TJaqF1Do84ga47 =fDeO -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Mon, Sep 28, 2015 at 3:14 PM, Carlos E. R.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Hi,
it has happened to me a few times, each time I doing different things as far as I can remember. Suddenly a process hangs. CPU is about iddle, disk is iddling. But some processes do not start. A simple "ps afx" never finishes, and ^C it doesn't work. Subsequently, a reboot also hangs, does nothing. I have to hit the power button hard, more than 4 seconds.
Most likely process is waiting uniterruptibly on completion of some kernel activity or on some resource. It can be "work as designed" - e.g. NFS hard mount waiting for a server to respond. Or it could be a bug. In general the only way to get some information is to use SysRq to trigger stack print (SysRq-t and -l) and/or panic to generate dump, although the latter needs someone to actually analyze it. If it happens often, you could setup netconsole so that you can capture console output of stack ptint. This would be good starting point for a bug report. Locking information (SysRq-d) may be of interest too. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 El 2015-09-28 a las 15:36 +0300, Andrei Borzenkov escribió:
On Mon, Sep 28, 2015 at 3:14 PM, Carlos E. R. <> wrote:
it has happened to me a few times, each time I doing different things as far as I can remember. Suddenly a process hangs. CPU is about iddle, disk is iddling. But some processes do not start. A simple "ps afx" never finishes, and ^C it doesn't work. Subsequently, a reboot also hangs, does nothing. I have to hit the power button hard, more than 4 seconds.
Most likely process is waiting uniterruptibly on completion of some kernel activity or on some resource. It can be "work as designed" - e.g. NFS hard mount waiting for a server to respond. Or it could be a bug.
No external mounts this time. No local network at all, in fact. Some days ago, yes. If the system still remembers that session :-?
In general the only way to get some information is to use SysRq to trigger stack print (SysRq-t and -l) and/or panic to generate dump, although the latter needs someone to actually analyze it. If it happens often, you could setup netconsole so that you can capture console output of stack ptint. This would be good starting point for a bug report. Locking information (SysRq-d) may be of interest too.
Netconsole... That would be difficult. I believe it is started as a kernel boot parameter. This is a laptop, and the problem may take weeks to reappear, so I can't ensure the machine be at the same network as the netconsole. Even no network at all, single machine, depends where I are at the time. However, syslog was able to write things. So, if there is a command that dumps that information to syslog, it should work. I'm trying to find what is the SysRq key combo on this Compaq laptop. I think it is the "fin/pet sis" key, together with the "fn" key. So far, I got nothing. Ok, got it. It is ctrl-alf-fn-SysReq-t: you have to be a hand twister! I get this in syslog: <0.6> 2015-09-28 15:45:27 minas-tirith kernel - - - [ 7270.503624] SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(V) show-blocked-tasks(w) dump-ftrace-buffer(z) <0.6> 2015-09-28 15:45:27 minas-tirith kernel - - - [ 7270.618162] SysRq : This sysrq operation is disabled. <0.6> 2015-09-28 15:45:28 minas-tirith kernel - - - [ 7270.802517] SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(V) show-blocked-tasks(w) dump-ftrace-buffer(z) <0.6> 2015-09-28 15:45:28 minas-tirith kernel - - - [ 7271.221314] SysRq : This sysrq operation is disabled. <0.6> 2015-09-28 15:45:28 minas-tirith kernel - - - [ 7271.601518] SysRq : This sysrq operation is disabled. <0.6> 2015-09-28 15:46:01 minas-tirith kernel - - - [ 7303.809116] SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(V) show-blocked-tasks(w) dump-ftrace-buffer(z) <0.6> 2015-09-28 15:46:01 minas-tirith kernel - - - [ 7303.878108] SysRq : This sysrq operation is disabled. Are you sure the openSUSE kernel is built with sysreq support? :-? - -- Cheers Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlYJRicACgkQja8UbcUWM1xfKgD9Ge98uG6n2hUEKwZTfSGE5JsP jkg8qkjcsh6XWwsvHEcBAJt++HsaaPf7WIIqWf383XNCdwJhosrrhFVw86d6NRax =kdl+ -----END PGP SIGNATURE-----
On Mon, Sep 28, 2015 at 4:52 PM, Carlos E. R.
Are you sure the openSUSE kernel is built with sysreq support? :-?
You may need to enable it cat /proc/sys/kernel/sysrq echo 1 >/proc/sys/kernel/sysrq You can also trigger it without twisting fingers echo t > /proc/sysrq-trigger ... etc -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Content-ID:
On Mon, Sep 28, 2015 at 4:52 PM, Carlos E. R. <> wrote:
Are you sure the openSUSE kernel is built with sysreq support? :-?
You may need to enable it
cat /proc/sys/kernel/sysrq
176
echo 1 >/proc/sys/kernel/sysrq
You can also trigger it without twisting fingers
echo t > /proc/sysrq-trigger ... etc
Fantastic! It works, at least now. Thanks :-) I'll write a script with the entire sequence, so that I don't need to remember. - -- Cheers Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlYJSsgACgkQja8UbcUWM1xqAAD8DK4ronLF8uTBeehA1W1JHBuv g6zY7VYUThaWX9BdCt4A/REVo76QMwruUd55wMbG5ORom8wv/XQ7sM+WufD6WfuF =Ud/f -----END PGP SIGNATURE-----
participants (2)
-
Andrei Borzenkov -
Carlos E. R.