[opensuse] 10.2 Postfix question
Since we are spending so much time and messages on the deal, instead of
making 10.2 a very good release, I wanted to get back more on topic.
Running beta2 here, but noticed this in beta1. Before, including 10.1,
putting my local user in the Masquerading option in Yast's, MTA module
(which filled in sender_canonical, with my real address, allowed scripts
run as my user and needing to send mail outside my system to be replaced
by my real address and everything worked fine. in 10.2, my mail was
rejected because of the postfix from=my user@my local domain. Searching
through the documentation and Google, I found it necessary to add this
mapping to generic, and it appeared from the documentation this was the
correct place for this mapping. My question is, did postfix change in
10.2, is the Yast module now not working, or what happened?
An example of the error is:
The Postfix program
Wed, 15 Nov 2006, by Joe_Morris@ntm.org:
Since we are spending so much time and messages on the deal, instead of making 10.2 a very good release, I wanted to get back more on topic. Running beta2 here, but noticed this in beta1. Before, including 10.1, putting my local user in the Masquerading option in Yast's, MTA module (which filled in sender_canonical, with my real address, allowed scripts run as my user and needing to send mail outside my system to be replaced by my real address and everything worked fine. in 10.2, my mail was rejected because of the postfix from=my user@my local domain. Searching through the documentation and Google, I found it necessary to add this mapping to generic, and it appeared from the documentation this was the correct place for this mapping. My question is, did postfix change in 10.2, is the Yast module now not working, or what happened?
An example of the error is:
The Postfix program
: host smtp.postoffice.net[165.212.11.125] said: 553 Invalid sender domain (in reply to MAIL FROM command) ------------------------------------------------------------------------
Reporting-MTA: dns; jmorris.home X-Postfix-Queue-ID: 639CC26F0DF X-Postfix-Sender: rfc822; joe@jmorris.home Arrival-Date: Tue, 7 Nov 2006 19:30:01 +0800 (PHT)
Final-Recipient: rfc822; report@dshield.org Action: failed Status: 5.0.0 Remote-MTA: dns; smtp.postoffice.net Diagnostic-Code: smtp; 553 Invalid sender domain
Thanks for any help or insights you may have.
The mailserver at postoffice.net checked for the existence of "jmorris.home", and couldn't find it, which is not surprising. ; <<>> DiG 9.3.1 <<>> +noshort jmorris.home ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 56293 ^^^^^ Use an existing TLD (like "ntm.org") to send mail with and recipients shall rejoyce.. You can put "myorigin = ntm.org" in main.cf Theo -- Theo v. Werkhoven Registered Linux user# 99872 http://counter.li.org ICBM 52 13 26N , 4 29 47E. + ICQ: 277217131 SUSE 9.2 + Jabber: muadib@jabber.xs4all.nl Kernel 2.6.8 + See headers for PGP/GPG info. Claimer: any email I receive will become my property. Disclaimers do not apply. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Theo v. Werkhoven wrote:
Reporting-MTA: dns; jmorris.home X-Postfix-Queue-ID: 639CC26F0DF X-Postfix-Sender: rfc822; joe@jmorris.home Arrival-Date: Tue, 7 Nov 2006 19:30:01 +0800 (PHT)
Final-Recipient: rfc822; report@dshield.org Action: failed Status: 5.0.0 Remote-MTA: dns; smtp.postoffice.net Diagnostic-Code: smtp; 553 Invalid sender domain
Thanks for any help or insights you may have.
The mailserver at postoffice.net checked for the existence of "jmorris.home", and couldn't find it, which is not surprising.
I think he already solved the problem, but asked why yast did not set up generic as required, when he configured postfix in yast.
Use an existing TLD (like "ntm.org") to send mail with and recipients shall rejoyce.. You can put "myorigin = ntm.org" in main.cf
$myorigin will only be used to complete non-FQDN addresses, but will not rewrite an existing sender address user@morris.home. I can't really comment on the yast module for the postfix configuration, since I never used it (I always configure all files manually). Sandy -- List replies only please! Please address PMs to: news-reply2 (@) japantest (.) homelinux (.) com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Sandy Drobic wrote:
Theo v. Werkhoven wrote:
Reporting-MTA: dns; jmorris.home X-Postfix-Queue-ID: 639CC26F0DF X-Postfix-Sender: rfc822; joe@jmorris.home
Remote-MTA: dns; smtp.postoffice.net Diagnostic-Code: smtp; 553 Invalid sender domain
The mailserver at postoffice.net checked for the existence of "jmorris.home", and couldn't find it, which is not surprising.
I think he already solved the problem, but asked why yast did not set up generic as required, when he configured postfix in yast. Almost correct. I did find the problem. The Yast MTA module has a Masquerading button, which effectively edits sender_canonical and creates the hash. This has worked up until 10.2. This quit working with 10.2, so I did some digging. I believe it still changed the from address, but somehow there was a header line, X-Postfix-Sender (see above), with the original sender. Research seemed to indicate what I needed to use was generic and not sender_canonical. So, I added via POSTFIX_ADD_GENERIC the hash type and path to /etc/sysconfig/postfix, as well as added generic to the postfix maps SuSEconfig creates. I then added the same mapping as in sender_canonical to generic, ran SuSEconfig, and it fixed the problem. So my question I guess, in the interest of making 10.2 a great release, is has postfix changed in this regard, has generic vs sender_canonical changed, which may need a bug report for the Yast module, or have I made some flawed steps in my understanding or troubleshooting? I can't really comment on the yast module for the postfix configuration, since I never used it (I always configure all files manually).
Sandy I figured as much. I have appreciated your depth of postfix knowledge, Sandy, but for me SuSEconfig's postfix script and the Yast module have done a very good job for the most part, and improving it would help more users in the long run. Thanks again for your thoughts.
-- Joe Morris Registered Linux user 231871 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Joe Morris (NTM) wrote:
Sandy Drobic wrote:
Theo v. Werkhoven wrote:
Reporting-MTA: dns; jmorris.home X-Postfix-Queue-ID: 639CC26F0DF X-Postfix-Sender: rfc822; joe@jmorris.home
Remote-MTA: dns; smtp.postoffice.net Diagnostic-Code: smtp; 553 Invalid sender domain The mailserver at postoffice.net checked for the existence of "jmorris.home", and couldn't find it, which is not surprising. I think he already solved the problem, but asked why yast did not set up generic as required, when he configured postfix in yast.
Almost correct. I did find the problem. The Yast MTA module has a Masquerading button, which effectively edits sender_canonical and creates the hash. This has worked up until 10.2. This quit working with 10.2, so I did some digging. I believe it still changed the from address, but somehow there was a header line, X-Postfix-Sender (see above), with the original sender. Research seemed to indicate what I needed to use was generic and not sender_canonical. So, I added via
What I find strange about this is why this header was added at all. Headers with the format X-xxxxxx are non-standard headers, and a closer look in this case reveals that it isn't a header at all, it is part of the body of the bounce message.
POSTFIX_ADD_GENERIC the hash type and path to /etc/sysconfig/postfix, as well as added generic to the postfix maps SuSEconfig creates. I then added the same mapping as in sender_canonical to generic, ran SuSEconfig, and it fixed the problem. So my question I guess, in the interest of making 10.2 a great release, is has postfix changed in this regard, has generic vs sender_canonical changed, which may need a bug report for the Yast module, or have I made some flawed steps in my understanding or troubleshooting?
Can't say that without more details. It should work with canonical as well, but it might be that additional settings are necessary. Here's what the documentation say: By default the canonical(5) mapping affects both message header addresses (i.e. addresses that appear inside messages) and message envelope addresses (for example, the addresses that are used in SMTP protocol commands). This is controlled with the canonical_classes parameter. NOTE: Postfix versions 2.2 and later rewrite message headers from remote SMTP clients only if the client matches the local_header_rewrite_clients parameter, or if the remote_header_rewrite_domain configuration parameter speci- fies a non-empty value. To get the behavior before Postfix 2.2, specify "local_header_rewrite_clients = static:all". It might be worth to investigate this. Could you check your configuration in for these settings with output of postconf: - "postconf local_header_rewrite_clients" - "postconf remote_header_rewrite_domain" - "postconf canonical_classes"
I can't really comment on the yast module for the postfix configuration, since I never used it (I always configure all files manually).
Sandy I figured as much. I have appreciated your depth of postfix knowledge, Sandy, but for me SuSEconfig's postfix script and the Yast module have done a very good job for the most part, and improving it would help more users in the long run. Thanks again for your thoughts.
The yast modules is helping to set up a local server for a small company or a home network. Most of these users don't need to work out the finer details of antispam configuration or tune the server for effective use of available resources. A few weeks ago I set up a temporary replacement server for our mailgateway (Suse 9.2 needed to be upgraded) and tried to set up the basic config with the postfix yast module (now on Suse 10.0). Half way during the configuration I didn't understand what settings yast offered to me and gave up. My trouble appearantly was that I couldn't understand Yast anymore since it didn't speak to me in Postfix terms. (^-^) Sandy -- List replies only please! Please address PMs to: news-reply2 (@) japantest (.) homelinux (.) com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Sandy Drobic wrote:
NOTE: Postfix versions 2.2 and later rewrite message headers from remote SMTP clients only if the client matches the local_header_rewrite_clients parameter, or if the remote_header_rewrite_domain configuration parameter speci- fies a non-empty value. To get the behavior before Postfix 2.2, specify "local_header_rewrite_clients = static:all".
It might be worth to investigate this. Could you check your configuration in for these settings with output of postconf:
- "postconf local_header_rewrite_clients"
local_header_rewrite_clients = permit_inet_interfaces
- "postconf remote_header_rewrite_domain" remote_header_rewrite_domain =
- "postconf canonical_classes"
canonical_classes = envelope_sender, envelope_recipient, header_sender, header_recipient I just checked a 10.1 and a 9.3, and those were the same exactly. I don't understand it. It appears sender_canonical does some but not enough rewriting the address, but generic worked immediately. and the man page said this: DESCRIPTION The optional generic(5) table specifies an address mapping that applies when mail is delivered. This is the opposite of canonical(5) mapping, which applies when mail is received. Typically, one would use the generic(5) table on a system that does not have a valid Internet domain name and that uses something like localdo‐ main.local instead. The generic(5) table is then used by the smtp(8) client to transform local mail addresses into valid Internet mail addresses when mail has to be sent across the Internet. See the EXAM‐ PLE section at the end of this document. The generic(5) mapping affects both message header addresses (i.e. addresses that appear inside messages) and message envelope addresses (for example, the addresses that are used in SMTP protocol commands). Which seems exactly what I needed, and indeed did work (but in the process confused me how it ever worked before). Is the above saying (as it seems to me) that canonical mapping are only for incoming mail, while generic is for outgoing mail? Is sender_canonical changing the sender of incoming mail? That is how it appears, but made me doubt how it seemed to work before. Thanks again for your help in my trying to get an understanding of postfix. -- Joe Morris Registered Linux user 231871 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Joe Morris (NTM) wrote:
Sandy Drobic wrote:
NOTE: Postfix versions 2.2 and later rewrite message headers from remote SMTP clients only if the client matches the local_header_rewrite_clients parameter, or if the remote_header_rewrite_domain configuration parameter speci- fies a non-empty value. To get the behavior before Postfix 2.2, specify "local_header_rewrite_clients = static:all".
It might be worth to investigate this. Could you check your configuration in for these settings with output of postconf:
- "postconf local_header_rewrite_clients"
local_header_rewrite_clients = permit_inet_interfaces
Did you send the mail directly from the pc where Postfix is running or was it sent from a pc within your network? What does "postconf inet_interfaces" say?
- "postconf remote_header_rewrite_domain" remote_header_rewrite_domain =
- "postconf canonical_classes"
canonical_classes = envelope_sender, envelope_recipient, header_sender, header_recipient
Those settings should work.
I just checked a 10.1 and a 9.3, and those were the same exactly. I don't understand it. It appears sender_canonical does some but not enough rewriting the address, but generic worked immediately. and the man page said this:
I know that Suse 9.2 used Postfix 2.1.5 where generic wasn't implemented yet, unfortunately I don't have a Suse 9.3 available, and on my 10.0 systems I already installed a recent version of Postfix.
Which seems exactly what I needed, and indeed did work (but in the process confused me how it ever worked before). Is the above saying (as it seems to me) that canonical mapping are only for incoming mail, while generic is for outgoing mail? Is sender_canonical changing the sender of incoming mail? That is how it appears, but made me doubt how it
Postfix isn't built with a single I-do-everything binary, instead it uses several programs to handle specific tasks. Generic is applied by the smtp client program, so it can only be used for mails which are handed to the smtp client, and the job of the smtp client is usually to send a mail out. canonical on the other hand is used by the cleanup daemon which checks a mail prior to queueing it to make sure that all required headers are present and if necessary insert it. Cleanup is also the daemon that applies header/body checks, by the way. So these checks and rewriting take place for incoming mails, before they are queued. Some headers may not be present at the time cleanup is checking the mail, while generic will see all headers since it sees the mails at the time it leaves the system.
seemed to work before. Thanks again for your help in my trying to get an understanding of postfix.
Once you start to understand Postfix it is a lot of fun. (^-°) Sandy -- List replies only please! Please address PMs to: news-reply2 (@) japantest (.) homelinux (.) com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Sandy Drobic wrote:
Joe Morris (NTM) wrote:
Sandy Drobic wrote:
- "postconf local_header_rewrite_clients" local_header_rewrite_clients = permit_inet_interfaces
Did you send the mail directly from the pc where Postfix is running or was it sent from a pc within your network? Directly from the computer running postfix.
What does "postconf inet_interfaces" say? jmorris:/home/joe # postconf inet_interfaces inet_interfaces = 127.0.0.1 ::1
I just checked a 10.1 and a 9.3, and those were the same exactly. I forgot, I have upgraded the original postfix on the 9.3 box. On the 9.3 box; joe@server:~> rpm -q postfix postfix-2.3_20051106-0.1 On the 10.1 box; joe@Sempron:~> rpm -q postfix postfix-2.2.9-10 Postfix isn't built with a single I-do-everything binary, instead it uses several programs to handle specific tasks. Generic is applied by the smtp client program, so it can only be used for mails which are handed to the smtp client, and the job of the smtp client is usually to send a mail out. So generic IS for outgoing mail.
canonical on the other hand is used by the cleanup daemon which checks a mail prior to queueing it to make sure that all required headers are present and if necessary insert it. Cleanup is also the daemon that applies header/body checks, by the way.
So these checks and rewriting take place for incoming mails, before they are queued. So the different canonical databases are for incoming mail, or with mail being scanned by amavisd-new, does all mail become incoming with regard to the queue? Some headers may not be present at the time cleanup is checking the mail, while generic will see all headers since it sees the mails at the time it leaves the system. So it could be a header added later than sender_canonical but caught by generic? If that is so, then it seems to be a new thing and may result in a bug for the Yast Postfix module (i.e. MTA).
-- Joe Morris Registered Linux user 231871 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Joe Morris (NTM) wrote:
Sandy Drobic wrote:
Joe Morris (NTM) wrote:
Sandy Drobic wrote:
- "postconf local_header_rewrite_clients" local_header_rewrite_clients = permit_inet_interfaces
Did you send the mail directly from the pc where Postfix is running or was it sent from a pc within your network?
Directly from the computer running postfix.
What does "postconf inet_interfaces" say?
jmorris:/home/joe # postconf inet_interfaces inet_interfaces = 127.0.0.1 ::1
Okay, now the question is, what IP address of the server was used to submit the mail. If only localhost is enabled for Postfix, then it's clear that the mail could only be sent from the server itself. Or was the mail submitted with the sendmail binary via command line? It shows in your log with "postfix/pickup" as the first entry of the mail.
I just checked a 10.1 and a 9.3, and those were the same exactly. I forgot, I have upgraded the original postfix on the 9.3 box. On the 9.3 box; joe@server:~> rpm -q postfix postfix-2.3_20051106-0.1
Ah, that's a snapshot version from last year. Did you compile from source or did you use a rpm?
On the 10.1 box; joe@Sempron:~> rpm -q postfix postfix-2.2.9-10
The big difference in the configuration was that the default for local_header_rewrite_clients was changed. I would have expected Suse to change the default to "static:all" to get the previous behavior of Postfix 2.1.
Postfix isn't built with a single I-do-everything binary, instead it uses several programs to handle specific tasks. Generic is applied by the smtp client program, so it can only be used for mails which are handed to the smtp client, and the job of the smtp client is usually to send a mail out. So generic IS for outgoing mail.
At least it is used for Mails that leave the current server. If that server is used as a Mailgateway, then the definition of outgoing and incoming is only defined by the internal/external destination ip addresses, when Postfix sends the mail.
canonical on the other hand is used by the cleanup daemon which checks a mail prior to queueing it to make sure that all required headers are present and if necessary insert it. Cleanup is also the daemon that applies header/body checks, by the way.
So these checks and rewriting take place for incoming mails, before they are queued. So the different canonical databases are for incoming mail, or with mail being scanned by amavisd-new, does all mail become incoming with regard to the queue?
If the server is the final destination for a mail, then the difference is quite big. The smtp client might never be called for the incoming mail. Even with a content_filter like amavisd-new, the transport could happen via a pipe, not smtp. In that case, generic would never be used. canonical on the other hand is used when the system receives a mail. Once the mail is accepted, cleanup will examine the mail, correkt broken headers and add missing neccessary headers.
Some headers may not be present at the time cleanup is checking the mail, while generic will see all headers since it sees the mails at the time it leaves the system.
So it could be a header added later than sender_canonical but caught by generic? If that is so, then it seems to be a new thing and may result in a bug for the Yast Postfix module (i.e. MTA).
It is possible, though I don't think that should happen. The headers you showed were from a bounce message, and they were part of the body of the mail, not within the header of the mails itself. If you have a content_filter like amavisd-new, every mail will be seen by cleanup twice. Once before the content_filter, and after the content_filter sends the mail back to Postfix. So even headers added by the content_filter should be rewritten, when the mails is resubmitted from the content_filter. Sandy -- List replies only please! Please address PMs to: news-reply2 (@) japantest (.) homelinux (.) com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Sandy Drobic wrote:
Joe Morris (NTM) wrote:
jmorris:/home/joe # postconf inet_interfaces inet_interfaces = 127.0.0.1 ::1
Okay, now the question is, what IP address of the server was used to submit the mail. If only localhost is enabled for Postfix, then it's clear that the mail could only be sent from the server itself.
Or was the mail submitted with the sendmail binary via command line? It shows in your log with "postfix/pickup" as the first entry of the mail.
postfix-2.3_20051106-0.1
Ah, that's a snapshot version from last year. Did you compile from source or did you use a rpm? I used an rpm from people directory, but that is a different machine
The headers you showed were from a bounce message, and they were part of the body of the mail, not within the header of the mails itself. The log for the mail is above. Since it was never actually sent, but rejected by my relayhost, I cannot tell where the header was.
If you have a content_filter like amavisd-new, every mail will be seen by cleanup twice. Once before the content_filter, and after the content_filter sends the mail back to Postfix. So even headers added by the content_filter should be rewritten, when the mails is resubmitted from the content_filter. Just to summarize a bit, am I correct that sender_canonical did NOT work because I had misconfigured my local inet addresses for postfix
I think you may be on to something here. In my /etc/hosts, my local
domain, i.e jmorris.home is defined as 192.168.10.1. The mailing
program (dshield iptables script) uses /usr/sbin/sendmail -oi -t to send
its mail. The logs showed:
Nov 7 19:30:02 jmorris postfix/pickup[28444]: 639CC26F0DF: uid=1000
from=<joe>
Nov 7 19:30:02 jmorris postfix/cleanup[30908]: 639CC26F0DF:
message-id=<20061107113002.639CC26F0DF@jmorris.home>
Nov 7 19:30:02 jmorris postfix/qmgr[28445]: 639CC26F0DF:
from=
participants (3)
-
Joe Morris (NTM)
-
Sandy Drobic
-
Theo v. Werkhoven