[Bug 741469] New: sendmail initialization delays boot by 5 seconds
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c0 Summary: sendmail initialization delays boot by 5 seconds Classification: openSUSE Product: openSUSE 12.1 Version: Final Platform: x86-64 OS/Version: SuSE Other Status: NEW Severity: Normal Priority: P5 - None Component: Basesystem AssignedTo: bnc-team-screening@forge.provo.novell.com ReportedBy: jdelvare@suse.com QAContact: qa@suse.de CC: werner@suse.com, fcrozat@suse.com Found By: Community User Blocker: --- On a freshly installed, up-to-date openSUSE 12.1 system: $ systemd-analyze blame | head -n 1 5209ms sendmail.service And I can see in /var/log/messages: Jan 15 11:50:07 endymion echo[1692]: Initializing SMTP port (sendmail) Jan 15 11:50:12 endymion systemd[1]: PID file /var/run/sendmail.pid not readable (yet?) after start. Jan 15 11:50:13 endymion systemd[1]: PID file /var/spool/clientmqueue/sm-client.pid not readable (yet?) after start. This looks surprisingly similar to bug #732912, although this one was closed as fixed. My take is that systemd tries to read the PID files for 5 seconds and finally gives up. And I suspect this is caused by wrong permissions on the PID files. All pid files in /var/run have permissions 644 except for sendmail.pid (and sm-notify.pid.) /var/spool/clientmqueue/sm-client.pid is also not world-readable. Another suspicious thing is that both pid files contain more than just the pid value, contrary to all other pid files. This is with: sendmail-8.14.5-72.1.3.x86_64 systemd-37-3.6.1.x86_64 Surprisingly, if I stop then start sendmail.service manually using systemctl, I get the same messages in /var/log/messages but without the 5 second delay. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c1 Dr. Werner Fink <werner@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- AssignedTo|bnc-team-screening@forge.pr |fcrozat@suse.com |ovo.novell.com | --- Comment #1 from Dr. Werner Fink <werner@suse.com> 2012-01-16 10:09:41 UTC --- Hmmm ... the ownership and permissions are
sudo ls -l /var/spool/clientmqueue/sm-client.pid -rw------- 1 mail mail 54 Sep 20 18:23 /var/spool/clientmqueue/sm-client.pid sudo ls -ld /var/spool/clientmqueue/ drwxrwx--- 2 mail mail 45 Jan 16 10:58 /var/spool/clientmqueue/
and this will not be changed for security reasons. And yes the pid files of sendmail processes also include the full command in the second line which is well known for sendmail. This is a major feature of sendmail as sendmail does rewrite its command line:
ps aux | grep sendmail root 17395 0.0 0.0 11504 772 ? Ss 2011 2:48 sendmail: accepting connections mail 17401 0.0 0.0 7588 420 ? Ss 2011 0:00 sendmail: Queue control mail 17402 0.0 0.1 39508 2584 ? S 2011 4:09 sendmail: running queue: /var/spool/clientmqueue
the question rises if systemd system uses other ownerships than root and/or can not handle a second or more lines in a pid file. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c2 --- Comment #2 from Frederic Crozat <fcrozat@suse.com> 2012-01-16 10:24:27 UTC --- pidfile used by systemd (as explained in man systemd.service) is used to detect pid of the daemon. And systemd only read the first line of pidfile. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c3 Dr. Werner Fink <werner@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nrickert@ameritech.net --- Comment #3 from Dr. Werner Fink <werner@suse.com> 2012-01-16 13:31:53 UTC --- Ok ... then it could be a permission problem ... or also possible a timing issue as the sendmail client for queue control is started if the port 25 of localhost is available that is if the sendmail server process has started and open port 25 (shouldn't this be addressed by bug #732912) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c4 --- Comment #4 from Jean Delvare <jdelvare@suse.com> 2012-01-16 16:27:47 UTC --- Werner, I don't quite buy the security reason. If it was so dangerous for pid files to be world-readable, all daemons would make their pid files 600, not just sendmail. It's not so difficult for an attacker to find that PID if they really want to, anyway. If the problem is with the second line of the pid file (the command line) then it is questionable why it is put in the same file in the first place. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c5 --- Comment #5 from Dr. Werner Fink <werner@suse.com> 2012-01-16 16:58:15 UTC --- (In reply to comment #4) Hmm ... this is how the upstream m source tar ball just do it: At least /var/spool/clientmqueue is owner mail:mail and it is not wordreadable and the pid is stored exactly there. And the client process has its specific umask for all its files below /var/spool/clientmqueue which includes the pid file. The question is why a root process shoulds not be able to read such a file? -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c6 --- Comment #6 from Neil Rickert <nrickert@ameritech.net> 2012-01-17 06:35:41 UTC ---
At least /var/spool/clientmqueue is owner mail:mail and it is not wordreadable and the pid is stored exactly there.
The readability of the pid file was never an issue in that placement. I think it was my suggestion (to sendmail.org) to put the pid file in the clientmqueue directory. The problem (as best I remember it) was that sendmail would start as root, write the file, then change to mail:mail (or whatever user is defined for that). But, if restarted by sending a SIGHUP, it was unable to write to update its pidfile. So the pidfile had to be placed somewhere that the limited privilege sendmail daemon could write to it. And the clientmqueue directory was an obvious choice. I don't know of any reason why there's a problem reading the file. There's probably some other bug involved. Looking at "lsof" output, sendmail has a write lock on the pidfile. It's an advisory lock so can be ignored. But if systemd is attempting to lock the file before reading, it might well be locked out. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c7 --- Comment #7 from Jean Delvare <jdelvare@suse.com> 2012-01-17 08:00:14 UTC --- Neil, thanks for your insight. Your point on write-locked pid files is interesting. Note that cron is doing the same though, and apparently systemd has no trouble reading /var/run/cron.pid (otherwise I guess it would complain.) As a side note, I wonder why cron and sendmail think their pid files need writing-locking when all other daemons do not. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c8 Jean Delvare <jdelvare@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|Normal |Major --- Comment #8 from Jean Delvare <jdelvare@suse.com> 2012-01-17 08:07:45 UTC --- The situation in openSUSE 12.1 is worse than I thought. I tried disabling the sendmail service, to avoid wasting 5 seconds at every boot until the problem is fixed. While systemctl, yast and chkconfig agree that the sendmail service is disabled, the sendmail daemon is _still_ started at boot: $ systemctl status sendmail.service sendmail.service - Sendmail Mail Transport Agent Loaded: loaded (/lib/systemd/system/sendmail.service; disabled) Active: active (running) since Tue, 17 Jan 2012 07:46:12 +0100; 1h 14min ago Process: 1845 ExecStart=/usr/sbin/sendmail $SENDMAIL_PORT_OPTS $SENDMAIL_ARGS (code=exited, status=0/SUCCESS) Process: 1812 ExecStartPre=/etc/mail/system/sm.pre (code=exited, status=0/SUCCESS) Process: 1780 ExecStartPre=/bin/echo Initializing SMTP port (sendmail) (code=exited, status=0/SUCCESS) Main PID: 2328 (sendmail) CGroup: name=systemd:/system/sendmail.service └ 2328 sendmail: accepting connections And I can't uninstall sendmail because the php5 package depends on it (it needs smtp_daemon.) So I just can't get rid of the 5 seconds delay. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c9 --- Comment #9 from Dr. Werner Fink <werner@suse.com> 2012-01-17 08:59:02 UTC --- Guess: you see the dependency chain for a MTA that is one service requires mail-transfer-agent.target. On the other hand there is a path dependency which may trigger the start of sendmail client process if /var/spool/clientmqueue is not empty and this will triggers the start of the main sendmail process listing on port 25. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c10 --- Comment #10 from Frederic Crozat <fcrozat@suse.com> 2012-01-17 09:19:32 UTC --- if you really want to disable a service (even if required by something else): systemctl mask sendmail.service -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c11 --- Comment #11 from Jean Delvare <jdelvare@suse.com> 2012-01-17 10:13:11 UTC --- As "systemctl show mail-transfer-agent.target" said: Before=cron.service I tried disabling cron.service too but it did not make any difference. /var/spool/clientmqueue contains a single, zero-byte file named sm-client.st. It comes with the sendmail rpm, so I think this means the queue is empty. Based on Frederic's advice in comment #10, I masked sendmail.service and then also sendmail-client.service, and with this the boot time as reported by systemd-analyze is finally down from 15 seconds to 10 seconds. I can use this as a workaround for the time being but obviously that's not a valid fix. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c12 --- Comment #12 from Jean Delvare <jdelvare@suse.com> 2012-01-17 13:00:24 UTC --- Created an attachment (id=471520) --> (http://bugzilla.novell.com/attachment.cgi?id=471520) systemd-analyze plot with sendmail enabled -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c13 --- Comment #13 from Jean Delvare <jdelvare@suse.com> 2012-01-17 13:00:58 UTC --- Created an attachment (id=471521) --> (http://bugzilla.novell.com/attachment.cgi?id=471521) systemd-analyze plot with sendmail masked -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c14 --- Comment #14 from Neil Rickert <nrickert@ameritech.net> 2012-01-17 14:44:02 UTC ---
As a side note, I wonder why cron and sendmail think their pid files need writing-locking when all other daemons do not.
I think that's for prevention of a second running daemon. On startup, if it cannot claim the lock it should log a message and go away. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c15 --- Comment #15 from Jean Delvare <jdelvare@suse.com> 2012-01-17 15:07:36 UTC --- Thanks to Frederic we found an alternative to masking sendmail.service and sendmail-client.service: one should disable sendmail.service, sendmail-client.service _and_ sendmail-client.path. So now at least I have a clean way to prevent sendmail from starting. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c16 --- Comment #16 from Dr. Werner Fink <werner@suse.com> 2012-01-17 15:13:44 UTC --- One reason for me to write sendmail-client.path was that if some mails are stored below /var/spool/clientmqueue those should be handled ;) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c17 --- Comment #17 from Jean Delvare <jdelvare@suse.com> 2012-01-17 15:43:57 UTC --- I disagree. If the service has been disabled by the system admin then it should simply not start. The admin knows better. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c18 --- Comment #18 from Dr. Werner Fink <werner@suse.com> 2012-01-17 16:17:44 UTC --- You have disabled it. Beside this it is a well known problem that a failed or missed sendmail client will cause forgotten mails. This had happen more than once and is one of the problem since sendmail has split into two distinct daemons, one for the local client mail queue and one for handling port 25. Therefore why not using one of the major feature of systemd. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c19 Frederic Crozat <fcrozat@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |NEEDINFO InfoProvider| |jdelvare@suse.com Jean Delvare <jdelvare@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |NEW Component|Basesystem |Basesystem InfoProvider|jdelvare@suse.com | Product|openSUSE 12.1 |openSUSE 12.3 OS/Version|SUSE Other |openSUSE 12.3 --- Comment #19 from Frederic Crozat <fcrozat@suse.com> 2013-03-11 15:50:47 UTC --- what is the current status of this bug ? --- Comment #20 from Jean Delvare <jdelvare@suse.com> 2013-05-27 08:40:27 UTC --- Sorry for the late reply, I somehow missed the NEEDINFO. I have upgraded the system to openSUSE 12.3 meanwhile and I'm afraid the bug is still present. I'll attach the latest output of systemd-analyze plot. That being said I'm not sure if sendmail is to blame here. If I enable ntp as as daemon, then it is ntp.service which appears to take 5 seconds to start and delay the rest of the boot process, and sendmail seems to start reasonably fast. I'll attach the output of systemd-analyze plot for that case as well. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c21 Jean Delvare <jdelvare@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #471520|0 |1 is obsolete| | --- Comment #21 from Jean Delvare <jdelvare@suse.com> 2013-05-27 08:41:35 UTC --- Created an attachment (id=541316) --> (http://bugzilla.novell.com/attachment.cgi?id=541316) systemd-analyze plot with sendmail enabled -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c22 --- Comment #22 from Jean Delvare <jdelvare@suse.com> 2013-05-27 08:42:20 UTC --- Created an attachment (id=541318) --> (http://bugzilla.novell.com/attachment.cgi?id=541318) systemd-analyze plot with ntp and sendmail enabled -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c23 --- Comment #23 from Dr. Werner Fink <werner@suse.com> 2013-05-27 08:58:04 UTC --- Hmmm ... I'am also using 12.3 with sendmail and after removing open-iscsi and masking out some not required services the system is up very fast. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c Frederic Crozat <fcrozat@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- AssignedTo|fcrozat@suse.com |systemd-maintainers@suse.de -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
https://bugzilla.novell.com/show_bug.cgi?id=741469 https://bugzilla.novell.com/show_bug.cgi?id=741469#c24 Dr. Werner Fink <werner@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |NEEDINFO InfoProvider| |jdelvare@suse.com --- Comment #24 from Dr. Werner Fink <werner@suse.com> 2014-01-13 14:49:37 UTC --- IMHO this is more a network problem. I've never seens that with my personal 12.3 (DSL) nor with any installation here around. As I'm not able to reproduce I need more informations. Something like an strace with timestamps of ntp or sendmail process (option -r and option -tt beside -f and -o /run/%N.strace) -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠novell.com