[opensuse] sistemad and _netdev (working?) [WAS: The fight continues - sistemad troubleshooting?]
On Wed, Mar 9, 2016 at 4:09 PM, Greg Freemyer
The "_netdev" is important.
I'm testing now. How long should the _netdev mounts take to mount?
After reboot I immediately logged in: the NFS mount is in place but the _netdev mount of /srv/sftp is not.
It has been 20 minutes since reboot, so even the 15 minute point has passed? I'll keep watching.
I don't recall using _netdev before, so this could be my config error, but I think I've followed the advice I got. It's now been 3 hours since I rebooted with these lines in my fstab: --- 10.200.3.230:/mnt/pacers1/kvm672/kvm672 /srv_new nfs rw,relatime,vers=3 0 0 /srv_new/sftp-container-large /srv/sftp ext4 _netdev,loop 0 0 /srv_new/portal_backup_container /home/portal_backup/portal_backup ext4 _netdev,loop 0 0 --- The _netdev mounts were only attempted once and that was too soon. Another bug? The NFS mount happened during the boot process, but seconds BEFORE the 2 mounts with _netdev in the args. === You can see the NFS mount happened at 15:29:35 ===
systemctl status srv_new.mount srv_new.mount - /srv_new Loaded: loaded (/etc/fstab) Active: active (mounted) since Wed 2016-03-09 15:29:35 EST; 3h 28min ago Where: /srv_new What: 10.200.3.230:/mnt/pacers1/kvm672/kvm672 Process: 1053 ExecMount=/bin/mount -n -t nfs -o rw,relatime,vers=3 10.200.3.230:/mnt/pacers1/kvm672/kvm672 /srv_new (code=exited, status=0/SUCCESS) ===
Note that the _netdev mount attempts were 5 seconds before that: === Output from systemd status for srv-sftp.mount === # systemctl status srv-sftp.mount srv-sftp.mount - /srv/sftp Loaded: loaded (/etc/fstab) Active: failed (Result: exit-code) since Wed 2016-03-09 15:29:30 EST; 3h 25min ago Where: /srv/sftp What: /srv_new/sftp-container-large Process: 1028 ExecMount=/bin/mount -n -t ext4 -o _netdev,loop /srv_new/sftp-container-large /srv/sftp (code=exited, status=32) Mar 09 15:29:30 cloud1 mount[1028]: mount: /srv_new/sftp-container-large: failed to setup loop device: No such file or directory ===================== Here's what I think is the only relevant part of the journal log: === Mar 09 15:29:30 cloud1 kernel: loop: module loaded Mar 09 15:29:30 cloud1 mount[1028]: mount: /srv_new/sftp-container-large: failed to setup loop device: No such file or directory Mar 09 15:29:30 cloud1 mount[1027]: mount: /srv_new/portal_backup_container: failed to setup loop device: No such file or directory Mar 09 15:29:30 cloud1 systemd[1]: Failed to mount /home/portal_backup/portal_backup. Mar 09 15:29:30 cloud1 systemd[1]: Dependency failed for Remote File Systems. Mar 09 15:29:30 cloud1 systemd[1]: Failed to mount /srv/sftp. === This is openSUSE 13.2 with current patches. Thanks Greg -- Greg Freemyer www.IntelligentAvatar.net -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 03/09/2016 07:07 PM, Greg Freemyer wrote:
I don't recall using _netdev before, so this could be my config error, but I think I've followed the advice I got.
It's now been 3 hours since I rebooted with these lines in my fstab:
--- 10.200.3.230:/mnt/pacers1/kvm672/kvm672 /srv_new nfs rw,relatime,vers=3 0 0 /srv_new/sftp-container-large /srv/sftp ext4 _netdev,loop 0 0 /srv_new/portal_backup_container /home/portal_backup/portal_backup ext4 _netdev,loop 0 0 ---
The _netdev mounts were only attempted once and that was too soon. Another bug?
The NFS mount happened during the boot process, but seconds BEFORE the 2 mounts with _netdev in the args.
The "_netdev" is for network devices only. It should be on the NFS line and not the "loop" lines. You've got it exactly the wrong way round there. The real issue is making the "loop" entries dependent on the NFS. As you say
Note that the _netdev mount attempts were 5 seconds before that:
That pretty much guarantees it will fail! -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
10.03.2016 06:01, Anton Aylward пишет:
On 03/09/2016 07:07 PM, Greg Freemyer wrote:
I don't recall using _netdev before, so this could be my config error, but I think I've followed the advice I got.
It's now been 3 hours since I rebooted with these lines in my fstab:
--- 10.200.3.230:/mnt/pacers1/kvm672/kvm672 /srv_new nfs rw,relatime,vers=3 0 0 /srv_new/sftp-container-large /srv/sftp ext4 _netdev,loop 0 0 /srv_new/portal_backup_container /home/portal_backup/portal_backup ext4 _netdev,loop 0 0 ---
The _netdev mounts were only attempted once and that was too soon. Another bug?
The NFS mount happened during the boot process, but seconds BEFORE the 2 mounts with _netdev in the args.
The "_netdev" is for network devices only. It should be on the NFS line and not the "loop" lines. You've got it exactly the wrong way round there.
No, it was exactly correct. _netdev is needed in this case because otherwise systemd does not know that filesystem is related to network and must be attempted after network is configured. For NFS it knows it already so no _netdev is really needed.
The real issue is making the "loop" entries dependent on the NFS.
Normally I expect that it order filesystems automatically. But looking in code, it skips this dependency for network mounts. Not sure why, this is also in current upstream code. It should be possible to force dependencies still.
As you say
Note that the _netdev mount attempts were 5 seconds before that:
That pretty much guarantees it will fail!
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 03/09/2016 10:30 PM, Andrei Borzenkov wrote:
The "_netdev" is for network devices only. It should be on the NFS line and not the "loop" lines. You've got it exactly the wrong way round there.
No, it was exactly correct. _netdev is needed in this case because otherwise systemd does not know that filesystem is related to network and must be attempted after network is configured. For NFS it knows it already so no _netdev is really needed.
I submit that his timing from the logs supports my case. The use of _netdev on the loop entries breaks the normal dependency that I and Carlos, in Message-ID: <56E0E757.4000803@telefonica.net> Date: Thu, 10 Mar 2016 04:17:43 +0100 address. I agree with him when he says:
I think you should try this way:
10.200.3.230:/mnt/pacers1/kvm672/kvm672 /srv_new nfs rw,relatime,vers=3,_netdev 0 1 /srv_new/sftp-container-large /srv/sftp ext4 loop 0 2 /srv_new/portal_backup_container /home/portal_backup/portal_backup ext4 loop 0 2
-- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-03-10 01:07, Greg Freemyer wrote:
It's now been 3 hours since I rebooted with these lines in my fstab:
--- 10.200.3.230:/mnt/pacers1/kvm672/kvm672 /srv_new nfs rw,relatime,vers=3 0 0 /srv_new/sftp-container-large /srv/sftp ext4 _netdev,loop 0 0 /srv_new/portal_backup_container /home/portal_backup/portal_backup ext4 _netdev,loop 0 0 ---
I think you should try this way: 10.200.3.230:/mnt/pacers1/kvm672/kvm672 /srv_new nfs rw,relatime,vers=3,_netdev 0 1 /srv_new/sftp-container-large /srv/sftp ext4 loop 0 2 /srv_new/portal_backup_container /home/portal_backup/portal_backup ext4 loop 0 2 man mount _netdev The filesystem resides on a device that requires network access (used to prevent the system from attempting to mount these filesystems until the network has been enabled on the system). I thought that "_netdev" was exclusive of "nfs", but the manual doesn't say that. However, the "device" for your loop mounts will not appear till after the nfs share is mounted, anyway, so _netdev would not be needed. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
10.03.2016 03:07, Greg Freemyer пишет:
The _netdev mounts were only attempted once and that was too soon. Another bug?
May be. For some reasons systemd explicitly does *not* add dependency on "fs_spec" for network mount points in this case. I.e. normally if you have /foo/bar /mnt ... it would implicitly add RequiresMoountsFor=/foo/bar /mnt that orders it after any mount point that contains them. But for network the former is skipped. It is sure something to at least ask upstream. Current systemd supports x-systemd.requires-mounts-for fstab option. As you have 13.2 which is too old, you can instead add configuration snippet to do the same. See below.
The NFS mount happened during the boot process, but seconds BEFORE the 2 mounts with _netdev in the args.
=== You can see the NFS mount happened at 15:29:35 ===
systemctl status srv_new.mount srv_new.mount - /srv_new Loaded: loaded (/etc/fstab) Active: active (mounted) since Wed 2016-03-09 15:29:35 EST; 3h 28min ago Where: /srv_new What: 10.200.3.230:/mnt/pacers1/kvm672/kvm672 Process: 1053 ExecMount=/bin/mount -n -t nfs -o rw,relatime,vers=3 10.200.3.230:/mnt/pacers1/kvm672/kvm672 /srv_new (code=exited, status=0/SUCCESS) ===
Note that the _netdev mount attempts were 5 seconds before that:
=== Output from systemd status for srv-sftp.mount === # systemctl status srv-sftp.mount srv-sftp.mount - /srv/sftp Loaded: loaded (/etc/fstab) Active: failed (Result: exit-code) since Wed 2016-03-09 15:29:30 EST; 3h 25min ago Where: /srv/sftp What: /srv_new/sftp-container-large Process: 1028 ExecMount=/bin/mount -n -t ext4 -o _netdev,loop /srv_new/sftp-container-large /srv/sftp (code=exited, status=32)
Mar 09 15:29:30 cloud1 mount[1028]: mount: /srv_new/sftp-container-large: failed to setup loop device: No such file or directory =====================
OK, so mkdir -p /etc/systemd/system/srv-sftp.mount.d cat > /etc/systemd/system/srv-sftp.mount.d/need_srv_new.conf << EOF [Unit] RequiresMountsFor=/srv_new/sftp-container-large EOF
Here's what I think is the only relevant part of the journal log:
=== Mar 09 15:29:30 cloud1 kernel: loop: module loaded Mar 09 15:29:30 cloud1 mount[1028]: mount: /srv_new/sftp-container-large: failed to setup loop device: No such file or directory Mar 09 15:29:30 cloud1 mount[1027]: mount: /srv_new/portal_backup_container: failed to setup loop device: No such file or directory Mar 09 15:29:30 cloud1 systemd[1]: Failed to mount /home/portal_backup/portal_backup. Mar 09 15:29:30 cloud1 systemd[1]: Dependency failed for Remote File Systems. Mar 09 15:29:30 cloud1 systemd[1]: Failed to mount /srv/sftp. ===
This is openSUSE 13.2 with current patches.
Thanks Greg -- Greg Freemyer www.IntelligentAvatar.net
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 03/09/2016 10:46 PM, Andrei Borzenkov wrote:
10.03.2016 03:07, Greg Freemyer пишет:
The _netdev mounts were only attempted once and that was too soon. Another bug?
May be. For some reasons systemd explicitly does *not* add dependency on "fs_spec" for network mount points in this case. I.e. normally if you have
/foo/bar /mnt ...
it would implicitly add
RequiresMoountsFor=/foo/bar /mnt
I note from systemd.unit(5) RequiresMountsFor= Takes a space-separated list of absolute paths. Automatically adds dependencies of type Requires= and After= for all mount its required to access the specified path.
that orders it after any mount point that contains them.
But for network the former is skipped. It is sure something to at least ask upstream.
Or experiment to see what happens with specific versions of systemd. It's easy enough for Greg to experiment with. He can set the various combinations we've discussed each in turn and see what the generator produces by looking in /var/run/systemd/generator at the .mount files. I have mountable /home /anton /Photographs /By-year /2015 The mount units produced by the geneerator for each of those has an explicit "After". I would expect when the generator runs on the correctly configured fstab (as per earlier mail about _netdev not being on the loop entries as Carlos documents) to see something like that, perhaps a "Requires" . -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (4)
-
Andrei Borzenkov -
Anton Aylward -
Carlos E. R. -
Greg Freemyer