-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Monday 2006-11-13 at 22:41 +0100, Henne Vogelsang wrote:

On Monday, November 13, 2006 at 21:38:45, Carlos E. R. wrote:

...

The Monday 2006-11-13 at 09:36 -0800, Martin Mielke wrote:

...

yep... I also notice the same delays...

Some! Now it is reaching an hour and three quarters. I have notified the admin two or three times, but no comments so far :-(

Fixed now. Happy? :)

Yeap. Now is back at seven minutes delay :-) - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFFWPAQtTMYHG2NR9URAkzPAJ4sKT6/YQPCJ6MBQdExoAZDXBwzhACfU7jY gC7cjjckIwJQGQk+MooYLWc= =aLoj -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Tuesday 2006-11-14 at 15:27 +1100, Basil Chupin wrote:

...

...

Fixed now. Happy? :)

Yeap. Now is back at seven minutes delay :-)

7 minutes is far too long (not a joke). Normally I can send a message and have it returned in less than 10 seconds (if I happen to post it just as I am about to poll my ISP for new mail).

It went down to a minute very soon after that. Today it takes about a minute for the last step from lists4.suse.de to my provider. The previous steps in the received chain happen within seconds one from the next. It is just the last one that delays for some unknown reason, and some times stalls to an hour or more (twice that I could notice). - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFFWcPQtTMYHG2NR9URAl39AJ9b8IK9+F/w02+Tm04DyTLFB+ItxQCfc89M gWHwmo+SpQrkaeK6EvCGExY= =l44R -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Henne Vogelsang wrote:

Hi,

On Tuesday, November 14, 2006 at 14:25:35, Carlos E. R. wrote:

...

The Tuesday 2006-11-14 at 15:27 +1100, Basil Chupin wrote:

...

...

...

Fixed now. Happy? :) Yeap. Now is back at seven minutes delay :-) 7 minutes is far too long (not a joke). Normally I can send a message and have it returned in less than 10 seconds (if I happen to post it just as I am about to poll my ISP for new mail). It went down to a minute very soon after that. Today it takes about a minute for the last step from lists4.suse.de to my provider. The previous steps in the received chain happen within seconds one from the next. It is just the last one that delays for some unknown reason, and some times stalls to an hour or more (twice that I could notice).

The reason was too few postfix processes delivering mail, therefor the delivering queue ran full (due to some bug in postfix it seems). This is fixed now.

Usual advice is to use "relay" as transport for outgoing mail, so that incoming smtp connections do not use up all allowed smtp connections or vice versa. Sandy -- List replies only please! Please address PMs to: news-reply2 (@) japantest (.) homelinux (.) com --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Carlos E. R. wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

The Tuesday 2006-11-14 at 15:10 +0100, Sandy Drobic wrote:

...

...

The reason was too few postfix processes delivering mail, therefor the delivering queue ran full (due to some bug in postfix it seems). This is fixed now. Usual advice is to use "relay" as transport for outgoing mail, so that incoming smtp connections do not use up all allowed smtp connections or vice versa.

You mean relaying to another server? I suppose it is something else, I don't quite see it.

No, in master.cf there are two transports you can use: smtp and relay. Default is smtp, relay is simply a copy of smtp. Sandy -- List replies only please! Please address PMs to: news-reply2 (@) japantest (.) homelinux (.) com --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Joachim Schrod wrote:

Sandy Drobic wrote:

...

Henne Vogelsang wrote:

...

The reason was too few postfix processes delivering mail, therefor the delivering queue ran full (due to some bug in postfix it seems). This is fixed now.

Usual advice is to use "relay" as transport for outgoing mail, so that incoming smtp connections do not use up all allowed smtp connections or vice versa.

Please allow us the opportunity to learn more about that.

This is caused by the maxproc column in master.cf, isn't it? But I have to confess that I didn't understand it, after reading the documentation.

Sigh, I am just typing this the second time due to power outage. Well, at least I know that my ups is cleanly shutting down all my systems. Just keep in mind that postfix will use the default values if not configured otherwise. The default setting for how many processes are allowed to run simultaneously is set with $default_process_limit. The default is: `postconf -d default_process_limit` = 100.

I looked at my master.cf, and I think the relevant lines seem to be: (I hope that there won't be too much line breaks inserted.)

# service type private unpriv chroot wakeup maxproc command + args smtp inet n - n - - smtpd

That's for incoming TCP connections, AFAIU.

Yes, the command row shows smtpd, the daemon that accepts connections. If a "-" is set instead of a number, then the default is applied, in this case 100.

smtp unix - - n - - smtp relay unix - - n - - smtp

According to my syslog entries, smtp seems to get used for outgoing emails. It is the default value for default_transport which is used for all mails except to $mydestination. The default transport for incoming email is local. The default transport for relayed email is relay.

Yes, smtp is the client program of Postfix that is used to send mails via SMTP. local is a local delivery agent, it does not speak smtp.

I don't understand which service definition in master.cf causes the "use up of allowed smtp connections" for incoming and outgoing emails. Would you please do me the favor and explain that?

I was assuming a mail relay, for example a Postfix relay in front of an exchange server. In that case incoming mail for internal domains will be relayed to the internal exchange server via SMTP, outgoing mails will also be sent via SMTP to their destination servers. Another transport that normally uses the SMTP protocol is the content_filter amavisd-new. If not told otherwise, Postfix won't care, where the smtp clients are sending the mails to, be it to the internal mailserver or the external foreign destination server. A mailinglist server is a special case. If a mailing list has a lot of subscribers, then a few incoming mails are sufficient to generate many thousands of outgoing emails. So Postfix will per default use a max of 100 smtp processes. 5 emails are submitted to the mailing list manager, 10000 emails are generated, and Postfix is using all 100 smtp processes to send mails out. If you are using amavisd-new to first filter the mail, that mail might have to wait for a smtp process to be free to be delivered to amavisd-new. Now, if you are using relay as the default transport for outgoing mails, all 100 smtp processes are available for tranport to amavis or other purposes. The usual reason is, that the mailgateway is using relay as transport to send mails to the internal server and smtp for outgoing mails, so that incoming and outgoing mails do not have to fight for a common pool of available smtp processes to be delivered. They each have their own maxproc setting. As I said, a mailinglist server is a very special beast, and if you have several busy mailing lists with lots of subscribers each, you can have a huge incoming or active queue in a very short time. I haven't yet handled a server that has to send a few hundred thousand mails each day (at least). If I remember correctly Henne said that opensuse has about 1800 subscribers with probably 100-150 mails each day. So, just the opensuse list causes about 200.000 mails to be sent each day. Pack on a few more high-volume mailing lists, and you've got one really busy server. Good hardware and a fast internet connection can cope with that, but it still takes a good deal of careful configuration to deal with the spikes of mails. Sandy -- List replies only please! Please address PMs to: news-reply2 (@) japantest (.) homelinux (.) com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Carlos E. R. wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

The Wednesday 2006-11-15 at 22:08 +0100, Sandy Drobic wrote:

...

Sigh, I am just typing this the second time due to power outage. Well, at least I know that my ups is cleanly shutting down all my systems.

... ...

Nice explanation :-)

The first time all my pc were shutting down, I was at work, couldn't connect to my servers at home anymore and came back to a dark house without power. Some workers were busy digging out the earth in the neighbarhood and destroyed the main power cable for our side of the street. So the ups kept providing currency for a few minutes, then apcupsd told all server to shut down. That part worked flawlessly. After two hours the power returned, they used a temporary workaround, so people were not left in the dark. Unfortunately they needed to cut power again to swith back to the repaired normal power cable, and my ups didn't have to time to recharge sufficiently, so my workstation was shutting down almost at once again... I wasn't fast enough to save the draft I was writing on. :-/

One of the things I miss in documentations is an explanation of how things work, not the usual extensive options explanations, like in http://www.postfix.org/documentation.html. So reading your explanation is a welcome change.

It's the usual difference between a reference and an in-depth explaining book designed to teach you WHY things work the way they do. Though I have to say that Wietse takes great pain to keep the documentation accurate, up-to-date and even transparent for users of different versions (most settings available in Postfix have a remark for which versions of Postfix they have been introduced). A frequent reply from Wietse is "If it is not documented, it is not implemented".

...

A mailinglist server is a special case. If a mailing list has a lot of subscribers, then a few incoming mails are sufficient to generate many thousands of outgoing emails.

So Postfix will per default use a max of 100 smtp processes. 5 emails are submitted to the mailing list manager, 10000 emails are generated, and Postfix is using all 100 smtp processes to send mails out. If you are using amavisd-new to first filter the mail, that mail might have to wait for a smtp process to be free to be delivered to amavisd-new.

Amavis can use "lmtp" instead. But I'm too sleepy to know if it is postfix to amavis or viceversa O:-)

Bothe amavis and Postfix can use lmtp to transfer mails. The recommended way though is to define additional transports with the settings you need for that transport. One example is "relay", another frequently used is "amavis" or "amavis-smtp". That transport is used to restrict the number of concurrent processes at a limit the server can handle. master.cf: smtp inet n - n - 100 smtpd -o content_filter=smtp-amavis:[127.0.0.1]:10024 smtp-amavis unix - - n - 2 smtp -o content_filter= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o mynetworks=127.0.0.0/8 That way there are 100 smtpd processes that accept mails but only two smtp processes are allowed to transfer mails to the expensive content_filter. If necessary, Mails will pile up in front of the content_filter, but the server will eventually process them (provided mails don't come in faster than Amavis can scan them in a reasonable time). Sandy -- List replies only please! Please address PMs to: news-reply2 (@) japantest (.) homelinux (.) com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

On 2006-11-13 15:41, Henne Vogelsang wrote:

Hi,

On Monday, November 13, 2006 at 21:38:45, Carlos E. R. wrote:

...

The Monday 2006-11-13 at 09:36 -0800, Martin Mielke wrote:

...

yep... I also notice the same delays...

Some! Now it is reaching an hour and three quarters. I have notified the admin two or three times, but no comments so far :-(

Fixed now. Happy? :)

Henne

We're always happy to see you, Henne -- don't worry about Carlos too much, it's just a slow day everywhere :-) I'm just glad I'm not seeing anymore of those dumb "bounce" messages from isotruck.com (don't know if you did that, or if Ma Bell did it, but I sure don't care either :-) ) -- --------------------------------------------------------------------- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org