[opensuse] Systemd parallel startup/shutdown and dependencies (This is not a rant).
Hi all. Slowly getting my head around Systemd and how to "control" it. I understand the idea of speeding up boot/shutdown by taking advantage of modern processor/system capabilities by parallelising startup and shutdown wherever possible, but this does lead to some potential race conditions and I'm not sure there's a generic "patch" that can fix all of them since some are very system-specific. With SysV Init, one could fairly easily control the order in which services were started (at boot) and killed (at shutdown) simpy by changing the Sxx or Kxx prefix of the scripts in /etc/rc.d/init.d (or /etc/init.d, depending on your distro), however this assumes that the entire startup process is serialised. The same approach doesn't work with systemd, leading to one unintended side effect that I (and others, judging by open bug reports for other distros) have been dealing with. One example - mounting remote file systems at boot time, especially Samba shares defined in /etc/samba/cifstab. With the default systemd configuration, the remote-fs and remote-fs-pre services (required by the remote-fs target) don't have the network.service listed in their "Requires=" parameter in the unit file; thus, in the default state the system can (and does) try to mount remote file systems before the network is started, but apparently only if you're bringing up the network using ifup instead of NetworkManager. [To my mind, NetworkManager is great on a laptop or other mobile system but makes no sense on a desktop or server system where the network configuration is static.] Furthermore, the system also takes down the network before trying to unmount the remote file systems, resulting in delayed shutdown until the unmount job times out. The fix for the boot problem was easy; add "Requires=network.service" to the remote-fs.target unit file (in /usr/lib/systemd/system). This did not fix the shutdown problem though - the network interface was shut down before the remote file-systems were umounted, even though the network.service apparently wasn't. I also had to add "Requires=network@eth0.service" to the remote-fs.service (and, for good measure, to the remote-fs-pre.target file, because it seems cifs.service may be started from remote-fs-pre.target rather than remote- fs.target). Adding a dependency on the specific (and in my case, only) network interface ensures that the remote filesystems are unmounted before the interface is taken down and fixes the shutdown issue. Herein lies the problem - creating a dependency on a specific network interface requires that the interface name be known when the unit file is created; there is no "network@eth0.service" file - on disk it exists as "network@.service" - the interface name is appended at runtime. Now, consistent naming of interfaces is obviously important here but this is what makes it system-specific, because under the new scheme the interface names are system-specific (unless you override them to use the traditional names as my system does). That's why I'm not sure that a generic fix is possible, because it is so specific to each individual system. That means that if you are going to be mounting remote file systems at boot time, you'd better get familiar with systemd unit files because it's going to require fixing on a system-by-system basis. That, I think, fits the definition of a broken design. It may have been a "corner case" that wasn't considered by the designers of systemd, I don't know, but there's got to be a better way. Changing the way the system handles the network startup/shutdown (to avoid the need to have dependencies on specific interfaces) is probably an area that needs addressing. Do I feel a bug report coming on? -- ============================================================== Rodney Baker VK5ZTV rodney.baker@iinet.net.au ============================================================== -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
В Thu, 19 Jun 2014 10:38:35 +0930 Rodney Baker <rodney.baker@iinet.net.au> пишет:
Do I feel a bug report coming on?
https://bugzilla.novell.com/show_bug.cgi?id=883565 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Mon, 23 Jun 2014 14:45:04 Andrey Borzenkov wrote:
В Thu, 19 Jun 2014 10:38:35 +0930 Rodney Baker <rodney.baker@iinet.net.au> пишет:
Do I feel a bug report coming on?
Thanks, Andrey - I raised that bug report after posting the OP to the mailing list last Thursday, only the OP took until today to show up on the list. Maybe (due to a previous thread) mails containing systemd in the subject were moderated? Thanks for your work on this one. :) -- ============================================================== Rodney Baker VK5ZTV rodney.baker@iinet.net.au ============================================================== -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 19/06/2014 03:08, Rodney Baker a écrit :
With SysV Init, one could fairly easily control the order in which services were started (at boot) and killed (at shutdown) simpy by changing the Sxx or Kxx prefix of the scripts in /etc/rc.d/init.d (or /etc/init.d, depending on your distro),
this was no more true, at least for openSUSE and for years, now. There already was requires of some sort inside the init scripts. my understanding is that systemd was build to solve the dependency problems these init scripts are making, specially in our days where devices are hotplugged. example, the mysql script (still present in 13.1) ### BEGIN INIT INFO # Provides: mysql # Required-Start: $network $remote_fs # Required-Stop: $network $remote_fs # Default-Start: 3 5 # (...) this reminds me of the very ancient days of the HP-41C calculators where the calculator routines scanned the entire system *between any keypress* because the calculator could go to sleep mode and have modules removed/added at the time. Very slow :-( jdd -- http://www.dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Monday 23 June 2014, jdd wrote:
Le 19/06/2014 03:08, Rodney Baker a écrit :
With SysV Init, one could fairly easily control the order in which services were started (at boot) and killed (at shutdown) simpy by changing the Sxx or Kxx prefix of the scripts in /etc/rc.d/init.d (or /etc/init.d, depending on your distro),
this was no more true, at least for openSUSE and for years, now. There already was requires of some sort inside the init scripts.
my understanding is that systemd was build to solve the dependency problems these init scripts are making, specially in our days where devices are hotplugged.
example, the mysql script (still present in 13.1)
### BEGIN INIT INFO # Provides: mysql # Required-Start: $network $remote_fs # Required-Stop: $network $remote_fs # Default-Start: 3 5 # (...)
This was only used for _creating_ these numbered links (insserv) but not when executing. In /etc/sysconfig/boot we had also the feature RUN_PARALLEL="no" I'm really missing it nowadays because it would avoid a lot of the systemd weiredness they same way we did it for sysvinit/startpar. Actually I have no motivation to correct any single init dependency. It's not worth to do this on my systems which have usually several months uptime. The most easy maintainable use case to have some simple sequentially executed tasks is gone ... cu, Rudi -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2014-06-19 03:08, Rodney Baker wrote:
With SysV Init, one could fairly easily control the order in which services were started (at boot) and killed (at shutdown) simpy by changing the Sxx or Kxx prefix of the scripts in /etc/rc.d/init.d (or /etc/init.d, depending on your distro), however this assumes that the entire startup process is serialised.
Just for clarification, recent year's openSUSE ignored the numbers in the link names, and this was documented. It also started scripts in parallel. Both features could be disabled, though. The order was defined at "insert" time of a service, by creating a kind of makefile (.depend.boot, .depend.halt, .depend.start, and .depend.stop), taking the decision based on certain pseudo-comments opn each script. These comments followed a documented standard, and even proprietary packages followed it (or tried to). I suppose that systemd has to properly work out the equivalent feature. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlOoIPMACgkQtTMYHG2NR9WM9gCbB7sHUP86hm3iOLVM1txhikra ZlAAniz8tBSzwwvAfyudK0wiw9Vb4HzK =AuWL -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 06/23/2014 07:43 AM, Carlos E. R. wrote:
I suppose that systemd has to properly work out the equivalent feature.
Arch has gone through this over the past 2 years and have it ironed out fairly well. Also note that many of the systemd service files are provided by upstream. They are generic. As Rodey, et. al. is finding, they require distro specific tweaks in many cases. - -- David C. Rankin, J.D.,P.E. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlOp0EEACgkQZMpuZ8Cyrciv1ACdGK2kVWQbMN8PXClYSUZ8Z6iK XSsAn0YVAQVxgJpdLoPSnI9Q55GuhAe/ =G8iV -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 06/18/2014 08:08 PM, Rodney Baker wrote:
Hi all. Slowly getting my head around Systemd and how to "control" it.
Rodney, Here is some systemd .bashrc "aspirin" than can make it less painful (keep a bottle for yourself handy). You may need to adjust the paths, these are from my arch install. systemd is a good system, but working with systemd to get comfortable with (trying all its features, journalctl quirks, etc.) it is like typing a bleeding novel... ## systemd aliases alias lsd='ls -1 /usr/lib/systemd/system/' #list systemd services alias lsde='ls -1 l1 /etc/systemd/system/multi-user.target.wants/' # list enabled alias sc='sudo systemctl' # cut down typing alias scn='sudo systemctl --no-pager' alias scdr='sudo systemctl daemon-reload' alias jc='sudo journalctl' alias jcn='sudo journalctl --no-pager' alias jcnl='sudo journalctl --no-pager --full' alias jcnlf='sudo journalctl --no-pager --full -f' (yes, I know the list systemd alias sounds like a drug ;-) -- David C. Rankin, J.D.,P.E. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 06/18/2014 08:08 PM, Rodney Baker wrote:
Hi all. Slowly getting my head around Systemd and how to "control" it.
Rodney,
Here is some systemd .bashrc "aspirin" than can make it less painful (keep a bottle for yourself handy). You may need to adjust the paths, these are from my arch install. systemd is a good system, but working with systemd to get comfortable with (trying all its features, journalctl quirks, etc.) it is
On Tue, 24 Jun 2014 14:29:09 David C. Rankin wrote: like
typing a bleeding novel... [...]
Thanks, David - very helpful. I won't comment further re LSD and systemd... :) Andrey's proposed patch had tidied up the boot order for legacy services that depend on the network being online, both for startup and shutdown. There is some debate going on as to whether it is appropriate for 13.2 but hopefully it will be accepted upstream and put the issue to bed once and for all. Rodney. -- ============================================================== Rodney Baker VK5ZTV rodney.baker@iinet.net.au ============================================================== -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
В Wed, 25 Jun 2014 20:19:34 +0930 Rodney Baker <rodney.baker@iinet.net.au> пишет:
Andrey's proposed patch had tidied up the boot order for legacy services that depend on the network being online, both for startup and shutdown. There is some debate going on as to whether it is appropriate for 13.2 but hopefully it will be accepted upstream and put the issue to bed once and for all.
This *is* upstream. Debates are what to do with SUSE-specific patches that appear to conflict with it ... -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Tuesday 24 June 2014, David C. Rankin wrote:
alias jc='sudo journalctl' alias jcn='sudo journalctl --no-pager' alias jcnl='sudo journalctl --no-pager --full' alias jcnlf='sudo journalctl --no-pager --full -f'
Don't you know about inputrc's keys "history-search-backward" and "history-search-forward" (suse default PgUp/PgDwn)? This helps to avoid long hardcoded alias lists. For example $ jo<PgUp><PgUp>... just scrolls quickly through all the 5 or 6 different journalctl commands which I have ever used. cu, Rudi -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 25/06/2014 13:10, Ruediger Meier a écrit :
Don't you know about inputrc's keys "history-search-backward" and "history-search-forward" (suse default PgUp/PgDwn)? This helps to avoid long hardcoded alias lists.
For example $ jo<PgUp><PgUp>... just scrolls quickly through all the 5 or 6 different journalctl commands which I have ever used.
looking for such command for a long time (but thinking it to be after Control R), many thnks!! jdd -- http://www.dodin.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Ruediger Meier wrote:
On Tuesday 24 June 2014, David C. Rankin wrote:
alias jc='sudo journalctl' alias jcn='sudo journalctl --no-pager' alias jcnl='sudo journalctl --no-pager --full' alias jcnlf='sudo journalctl --no-pager --full -f'
Don't you know about inputrc's keys "history-search-backward" and "history-search-forward" (suse default PgUp/PgDwn)? This helps to avoid long hardcoded alias lists.
For example $ jo<PgUp><PgUp>... just scrolls quickly through all the 5 or 6 different journalctl commands which I have ever used.
Hi Rudi thanks for that one, very helpful! (I'm always amazed at how much I don't know ...) -- Per Jessen, Zürich (18.9°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (7)
-
Andrey Borzenkov -
Carlos E. R. -
David C. Rankin -
jdd -
Per Jessen -
Rodney Baker -
Ruediger Meier