http://bugzilla.suse.com/show_bug.cgi?id=1003342 Dr. Werner Fink changed: What |Removed |Added ---------------------------------------------------------------------------- Blocker|--- |Yes Found By|--- |Development -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1003342 http://bugzilla.suse.com/show_bug.cgi?id=1003342#c2 Neil Brown changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |needinfo?(werner@suse.com) --- Comment #2 from Neil Brown ---

What exactly should the nfs-server-generator do?

If you give it three directories (they can all be /tmp) it will create a directory "nfs-server.service.d" in the first directory, and a file "order-with-mounts.conf" in that directory. This serves as a "drop-in" for systemd to extend the nfs-server.service unit file. It will read /etc/fstab and /etc/exports (and /etc/exports.d/*) and write out two sorts of directives: 1/ "RequiresMountsFor=DIR" for every directory that is exported by /etc/exports 2/ "Before=DIR.mount" for every nfs (or nfs4) mount point listed in /etc/fstab. This ensures that nfsd is started after all the filesystems it exports are mounted, and before and NFS filesystems are mounted. Presumably the hang that you notice is caused by one of those causing systemd to wait for something that will never happen. Could you please run /usr/lib/systemd/system-generators/nfs-server-generator /tmp /tmp /tmp and then attach /etc/fstab /etc/exports and /tmp/nfs-server.service.d/order-with-mounts.conf to this bug. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1003342 http://bugzilla.suse.com/show_bug.cgi?id=1003342#c4 --- Comment #4 from Dr. Werner Fink --- Created attachment 696293 --> http://bugzilla.suse.com/attachment.cgi?id=696293&action=edit /etc/fstab -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1003342 http://bugzilla.suse.com/show_bug.cgi?id=1003342#c6 --- Comment #6 from Dr. Werner Fink --- Created attachment 696297 --> http://bugzilla.suse.com/attachment.cgi?id=696297&action=edit /tmp/nfs-server.service.d/order-with-mounts.conf Why this strange path ... IMHO it should be /run/systemd/system/nfs-server.service.d/order-with-mounts.conf or /run/systemd/generator/nfs-server.service.d/order-with-mounts.conf -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1003342 http://bugzilla.suse.com/show_bug.cgi?id=1003342#c7 --- Comment #7 from Neil Brown ---

Why this strange path ... IMHO it should be

/run/systemd/system/nfs-server.service.d/order-with-mounts.conf

True, and it would be if you had passed "/run/systemd/system" as the argument to nfs-server-generator. I think systemd passes /run/systemd/generator. /tmp was just for testing. There is nothing unusual in your files. The only effect of having nfs-server-generator present is that nfs-server.service won't start until /usr/src is mounted, and that would have been the case anyway. So it really should have no net effect. How certain are you that this program is the cause of the failure to boot? Did you try putting the program back and observe the failure return? Presumably the boot is getting past the initrd/dracut stage? Can you get logs of what systemd thought was happening (no, i don't know off hand how to do that). Very confusing... -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1003342 http://bugzilla.suse.com/show_bug.cgi?id=1003342#c9 --- Comment #9 from Dr. Werner Fink --- Currently I'm not able to reproduce this on a VM. And dead locking my workstation takes some effort to get it back again due to this INTEL evaluation BIOS of this evaluation MB ;) -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1003342 http://bugzilla.suse.com/show_bug.cgi?id=1003342#c11 --- Comment #11 from Dr. Werner Fink --- On a virtual Leap 42.1 I see (without hang): d174:~ # journalctl -b 0 | grep nfs-s Oct 12 08:21:18 linux-nzqz systemd[1]: /usr/lib/systemd/system-generators/nfs-server-generator exited successfully. Oct 12 08:21:18 linux-nzqz systemd[1]: Installed new job nfs-server.service/start as 194 Oct 12 08:21:03 linux-nzqz nfs-server-generator[371]: Failed to resolve boole.suse.de Oct 12 08:21:03 linux-nzqz nfs-server-generator[371]: Failed to resolve boole.suse.de Oct 12 08:21:03 linux-nzqz nfs-server-generator[371]: Failed to resolve localhost.suse.de Oct 12 08:21:03 linux-nzqz nfs-server-generator[371]: Failed to resolve localhost.suse.de Oct 12 08:21:02 linux-nzqz systemd[1]: Spawned /usr/lib/systemd/system-generators/nfs-server-generator as 371 Oct 12 08:22:12 d174 systemd[1]: nfs-server.service changed dead -> start-pre Oct 12 08:22:13 d174 systemd[1]: Child 1685 belongs to nfs-server.service Oct 12 08:22:13 d174 systemd[1]: nfs-server.service: control process exited, code=exited status=0 Oct 12 08:22:13 d174 systemd[1]: nfs-server.service got final SIGCHLD for state start-pre Oct 12 08:22:13 d174 systemd[1]: nfs-server.service changed start-pre -> start Oct 12 08:22:13 d174 systemd[1]: Child 1697 belongs to nfs-server.service Oct 12 08:22:13 d174 systemd[1]: nfs-server.service: main process exited, code=exited, status=0/SUCCESS Oct 12 08:22:13 d174 systemd[1]: nfs-server.service changed start -> exited Oct 12 08:22:13 d174 systemd[1]: Job nfs-server.service/start finished, result=done Oct 12 08:22:13 d174 systemd[1]: nfs-server.service: cgroup is empty Oct 12 08:22:13 d174 systemd[1]: nfs-server.service: cgroup is empty Interesting is that the file /run/systemd/generator/nfs-server.service.d/order-with-mounts.conf shows # Automatically generated by nfs-server-generator [Unit] Before= usr-src-werner.mount but the file with /tmp three times argument to the generator # Automatically generated by nfs-server-generator [Unit] RequiresMountsFor=/usr/lib/crash/export Before= usr-src-werner.mount with my test partition /usr/lib/crash and its mount export /usr/lib/crash/export ... -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1003342 http://bugzilla.suse.com/show_bug.cgi?id=1003342#c13 --- Comment #13 from Dr. Werner Fink --- Hmmm ... the difference between my working horse and the virtual system is the host line in /etc/nsswitch.conf, that is on my work station I use hosts: db [NOTFOUND=continue] files mdns_minimal [NOTFOUND=continue] dns whereas on the virtual host hosts: files mdns_minimal [NOTFOUND=return] dns ... I guess that the nfs-server-generator on my workstation simply locks at name resolution. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1003342 http://bugzilla.suse.com/show_bug.cgi?id=1003342#c17 --- Comment #17 from Dr. Werner Fink --- Created attachment 696979 --> http://bugzilla.suse.com/attachment.cgi?id=696979&action=edit /tmp/nfs-generator.log Here, some point I've detected. First of all, nfs-server-generator can not detect host nor domain in this early excution time ... but what really hurst is that at this point it is not guaranteed that we have a working syslog socket. That is: nfs-server-generator does hang on sendto of a local UNIX domain socket sendto(4, "<11>Oct 12 14:02:32 nfs-server-generator: Failed to resolve d136.suse.de", 72, MSG_NOSIGNAL, NULL, 0 and waits upto the point where I've kill it from the emergency shell I've spawned on the ninth virtual console. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1003342 http://bugzilla.suse.com/show_bug.cgi?id=1003342#c19 --- Comment #19 from Dr. Werner Fink --- OK getmntent is harmless, but openlog as well as (__v)syslog(_chk) and getaddrinfo and co. are not -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1003342 http://bugzilla.suse.com/show_bug.cgi?id=1003342#c21 --- Comment #21 from Dr. Werner Fink --- With this patch I see now d174:~ # cat /run/systemd/generator/nfs-server.service.d/order-with-mounts.conf # Automatically generated by nfs-server-generator [Unit] RequiresMountsFor=/usr/lib/crash/export -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1003342 http://bugzilla.suse.com/show_bug.cgi?id=1003342#c23 --- Comment #23 from Neil Brown --- Created attachment 697117 --> http://bugzilla.suse.com/attachment.cgi?id=697117&action=edit Patch that I will propose upstream. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1003342 http://bugzilla.suse.com/show_bug.cgi?id=1003342#c25 --- Comment #25 from Thomas Schäfer --- I informed one of my best experts about this bug. His statement (I hope that I correctly translate): He would change the nfs-server unit. Instead of network.target network-online.target. (https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/ ) Then DNS and other things should work. Roll back this patch: https://build.opensuse.org/package/view_file/openSUSE:Leap:42.1/nfs-utils/00... One thing I still do not understand - on my system nfs-server.service was disabled at that time. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1003342 http://bugzilla.suse.com/show_bug.cgi?id=1003342#c27 --- Comment #27 from Neil Brown ---

He would change the nfs-server unit. Instead of network.target network-online.target.

I disagree. We really do want nfsd to start even if the network isn't online yet. There could be a problem because nfs-server.service runs exportfs, and that will fail if DNS lookup isn't working. I think the correct fix for that would be to change exportfs to work correctly when DNS is not available - I don't think there is a reason why it shouldn't work.

One thing I still do not understand - on my system nfs-server.service was disabled at that time.

The problem is being caused by a generator. systemd generators are run unconditionally so the nfs-server genetator is run whether nfs-server.service is enabled or not (so changing nfs-server to start a bit later wouldn't stop the generator from misbehaving). I'm confident that we have found the correct resolution in changing the generator not to use DNS or syslog. Thanks. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1003342 http://bugzilla.suse.com/show_bug.cgi?id=1003342#c29 Neil Brown changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |maint-coord@suse.de Flags| |needinfo?(maint-coord@suse. | |de) --- Comment #29 from Neil Brown ---

I'm wondering whether we should introduce a default timeout for generators as a safety measure.

Possibly a good idea. Something to take up with the systemd developers. I have created an nfs-utils package which fixes these issues. It is in home:neilbrown:branches:SUSE:SLE-12-SP1:Update/nfs-utils and I have submitted a maintenance request for SLE-12-SP1. The same update should go to Leap:42.1 Maintenance: the fix for bsc#994468 can cause the system to hang on boot, depending on particular details of configuration. So the next nfs-utils update might need to be a slightly higher priority. I need to submit an update for 12-SP2 as well. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1003342 Swamp Workflow Management changed: What |Removed |Added ---------------------------------------------------------------------------- Whiteboard| maint:planned:update |maint:planned:update | |ibs:running:3350:moderate -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1003342 Swamp Workflow Management changed: What |Removed |Added ---------------------------------------------------------------------------- Whiteboard|ibs:running:3350:moderate |ibs:running:3350:moderate | |obs:running:5750:moderate -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1003342 Swamp Workflow Management changed: What |Removed |Added ---------------------------------------------------------------------------- Whiteboard|ibs:running:3350:moderate |obs:running:5750:moderate |obs:running:5750:moderate | -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1003342 Swamp Workflow Management changed: What |Removed |Added ---------------------------------------------------------------------------- Whiteboard|obs:running:5750:moderate | -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.suse.com/show_bug.cgi?id=1003342 http://bugzilla.suse.com/show_bug.cgi?id=1003342#c35 --- Comment #35 from Thomas Schäfer --- Name : nfs-client Version : 1.3.0 Release : 26.1 Architecture: x86_64 Install Date: Mo 31 Okt 2016 14:06:48 CET works for me. -- You are receiving this mail because: You are on the CC list for the bug.