[opensuse] Postfix question relaying through Exchange
I just ran into a problem, for which I have found a workaround, but am wondering if anyone (you know who you are Sandy :-) ) can shed some additional light to actually fix the problem. The scenario is I am now relaying through the Exchange server at our HQ (their choice). There are some domains that are being rejected because of Body Type not supported by Remote Host. The problem seems to be because Exchange advertises 8bitmime (the fix I implemented was to discard the 8bitmime ehlo keyword from their smtp server). Evidently, the domains we are having problems with do not support 8bitmime. My understanding is when Postfix sees the Exchange server does support it, it converts the message, reported as 7 bit in the email client, to 8 bit based on the 8bitmime response code of Exchange. When Exchange relays the message, and finds the destination smtp server does not support 8bitmime, it bounces it back. Since the message was 7 bit to start with, and telling Postfix to discard the 8bitmime ehlo response allows it to work, which is the problem? Should Postfix convert the message to 8bit? Is that the proper response to the 8bitmime ehlo response? Is the problem Exchanges non conversion to 7bit when the destination server does not support 8bitmime? IOW, they could configure Exchange to not advertise 8bitmime and it would work. Telling Postfix to discard the response from their server also works, but it is in the smtp compatibility section of the Postfix docs, which lead me to believe it was added to allow Postfix to work in the presence of a broken smtp server. My fix is a workaround, which if the problem is in Exchange, will only mask the real problem. As a matter of interest, one of the domains we are having problems with, apparently because of a lack of 8bitmime support, is aol.com. TIA for any light you can shed to actually resolve this problem. -- Joe Morris Registered Linux user 231871 running openSUSE 10.2 x86_64 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Joe Morris (NTM) wrote:
I just ran into a problem, for which I have found a workaround, but am wondering if anyone (you know who you are Sandy :-) ) can shed some additional light to actually fix the problem. The scenario is I am now relaying through the Exchange server at our HQ (their choice). There are some domains that are being rejected because of Body Type not supported by Remote Host. The problem seems to be because Exchange advertises 8bitmime (the fix I implemented was to discard the 8bitmime ehlo keyword from their smtp server). Evidently, the domains we are having problems with do not support 8bitmime. My understanding is when Postfix sees the Exchange server does support it, it converts the message, reported as 7 bit in the email client, to 8 bit based on the 8bitmime response code of Exchange. When Exchange relays the message, and finds the destination smtp server does not support 8bitmime, it bounces it back.
The correct solution would be for Exchange to convert the message from 8bit to 7bit, if the destination system does not announce support for 8bit. In any case, your solution is the only possible one for your situation, when you can neither reconfigure the Exchange server nor the destination server.
Since the message was 7 bit to start with, and telling Postfix to discard the 8bitmime ehlo response allows it to work, which is the problem? Should Postfix convert the message to 8bit? Is that the proper response to the 8bitmime ehlo response? Is the problem Exchanges non conversion to 7bit when the destination server does not support 8bitmime? IOW, they could configure Exchange to not advertise 8bitmime
Postfix does not convert any 7bitmime to 8bitmime, it merely converts 8bitmime to 7bitmime, if the destination server does not support 8bitmime. Though there was a bug recently in a perl module that amavisd-new uses (Net::Cmd 2.2.7), which caused ALL mail to be converted to UTF8 regardless of the encoding of the headers. It caused a lot of screams here in Germany because the problem immediately became apparent with the German "Umlaute" äüö etc.
and it would work. Telling Postfix to discard the response from their server also works, but it is in the smtp compatibility section of the Postfix docs, which lead me to believe it was added to allow Postfix to work in the presence of a broken smtp server. My fix is a workaround, which if the problem is in Exchange, will only mask the real problem. As a matter of interest, one of the domains we are having problems with, apparently because of a lack of 8bitmime support, is aol.com.
Your problem is, that you can only influence message handling on your outgoing server, so you have to choose the most compatible setting directly at the beginning of the transport chain. The exchange server doesn't seem to have problems with 8bit, but apparently can't convert the message when the remote server does not announce 8bit. So the mesage is bounced. Not very nice, but you do not lose the mail, at least. I had a similar problem last weekend when I set up my server to use amavisd-new as a smtp-proxy filter. For whatever reason the cyrus-users listserver is adding a "auth=<>" to the "mail from" command, which is no problem for Postfix but causes amavisd-new to (wrongly) reject the message. Postfix announces support for auth, and hands the connection over to amavisd-new, which does not seem to support auth correctly (smtp auth connections work, but not the auth extension in the "mail from" command. The only solution I found was to suppress the auth announcement in the ehlo response when the cyrus-users listserver connects. My post on the amavis list didn't get any replies, so for the moment it is the only solution. -- Sandy List replies only please! Please address PMs to: news-reply2 (@) japantest (.) homelinux (.) com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Sandy Drobic wrote:
Joe Morris (NTM) wrote:
Since the message was 7 bit to start with, and telling Postfix to discard the 8bitmime ehlo response allows it to work, which is the problem? Should Postfix convert the message to 8bit? Is that the proper response to the 8bitmime ehlo response? Is the problem Exchanges non conversion to 7bit when the destination server does not support 8bitmime? IOW, they could configure Exchange to not advertise 8bitmime
Postfix does not convert any 7bitmime to 8bitmime, it merely converts 8bitmime to 7bitmime, if the destination server does not support 8bitmime.
But didn't the message have to be converted to 8 bit to be rejected by those domains we had trouble sending to , and whose servers do not advertise 8bitmime? I guess I was assuming they had to be since they were bounced. And they now work with 8bitmime keyword discarded. So am I wrong that, since the client says the message was 7 bit (Thunderbird), i.e. Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit and it now goes through with only the change to what Postfix sees in the response codes from Exchange, that they had to be being converted to 8 bit somewhere. I do also run amavisd-new with virus and spam checking, so all the messages do also go through there. If Postfix doesn't convert 7 bit to 8 bit, how do they now work? I checked amavisd-new, and it also advertises 8bitmime. If it started off as a 7 bit encoded message, and it went through Postfix 2x and amavisd-new once, and discarding 8bitmime AFAIU should make Postfix convert the message to 7 bit, where could it have become an 8 bit message?
Though there was a bug recently in a perl module that amavisd-new uses (Net::Cmd 2.2.7), which caused ALL mail to be converted to UTF8 regardless of the encoding of the headers. It caused a lot of screams here in Germany because the problem immediately became apparent with the German "Umlaute" äüö etc.
I checked and I don't have that module installed. pin doesn't even list that module. But that got me to wondering if amavisd-new could be converting it. I had totally forgot about that one.
I had a similar problem last weekend when I set up my server to use amavisd-new as a smtp-proxy filter. For whatever reason the cyrus-users listserver is adding a "auth=<>" to the "mail from" command, which is no problem for Postfix but causes amavisd-new to (wrongly) reject the message. Postfix announces support for auth, and hands the connection over to amavisd-new, which does not seem to support auth correctly (smtp auth connections work, but not the auth extension in the "mail from" command.
The only solution I found was to suppress the auth announcement in the ehlo response when the cyrus-users listserver connects. My post on the amavis list didn't get any replies, so for the moment it is the only solution.
Well, I am encouraged by the fact I actually understood most of that, and understand exactly what you did. I am learning. Your help as always has been much appreciated. -- Joe Morris Registered Linux user 231871 running openSUSE 10.2 x86_64 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Joe Morris (NTM) wrote:
Sandy Drobic wrote:
Joe Morris (NTM) wrote:
Since the message was 7 bit to start with, and telling Postfix to discard the 8bitmime ehlo response allows it to work, which is the problem? Should Postfix convert the message to 8bit? Is that the proper response to the 8bitmime ehlo response? Is the problem Exchanges non conversion to 7bit when the destination server does not support 8bitmime? IOW, they could configure Exchange to not advertise 8bitmime
Postfix does not convert any 7bitmime to 8bitmime, it merely converts 8bitmime to 7bitmime, if the destination server does not support 8bitmime.
But didn't the message have to be converted to 8 bit to be rejected by those domains we had trouble sending to , and whose servers do not advertise 8bitmime? I guess I was assuming they had to be since they were bounced. And they now work with 8bitmime keyword discarded. So am I wrong that, since the client says the message was 7 bit (Thunderbird), i.e.
Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit
The only way to be sure is to inspect the message after they went through a hop and compare them with the originally sent message. Use the "hold" action to capture the mail and have a look at it with "postcat -q queue-id". That way you can compare the mail before and after it went through Amavisd-new. I am not an expert on mime encapsulation but AFAIK every mime entity can have a different encoding. If you are using AlterMIME to add Footers to outgoing mails or something similar, that could also introduce 8bitmime.
and it now goes through with only the change to what Postfix sees in the response codes from Exchange, that they had to be being converted to 8 bit somewhere. I do also run amavisd-new with virus and spam checking, so all the messages do also go through there. If Postfix doesn't convert 7 bit to 8 bit, how do they now work? I checked amavisd-new, and it also advertises 8bitmime. If it started off as a 7 bit encoded message, and it went through Postfix 2x and amavisd-new once, and discarding 8bitmime AFAIU should make Postfix convert the message to 7 bit, where could it have become an 8 bit message?
Any hop on the way could have done it. But better take a look at the exact message content after each hop (yes, Amavisd-new also counts as hop).
Though there was a bug recently in a perl module that amavisd-new uses (Net::Cmd 2.2.7), which caused ALL mail to be converted to UTF8 regardless of the encoding of the headers. It caused a lot of screams here in Germany because the problem immediately became apparent with the German "Umlaute" äüö etc.
I checked and I don't have that module installed. pin doesn't even list that module. But that got me to wondering if amavisd-new could be converting it. I had totally forgot about that one.
Easy to forget. I had an eye on Amavisd-new because I needed to implement DSN, so I kept in mind that Amavisd-new also acts as an smtp server. Net::Cmd is part of libnet 1.20. I was bitten by that bug when I updated my Perl installation through the cpan shell. -- Sandy List replies only please! Please address PMs to: news-reply2 (@) japantest (.) homelinux (.) com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Sandy Drobic wrote:
The only way to be sure is to inspect the message after they went through
a hop and compare them with the originally sent message. Use the "hold" action to capture the mail and have a look at it with "postcat -q queue-id". That way you can compare the mail before and after it went through Amavisd-new.
I wasn't sure how to do that, but I thought of an easy test. I tried it without discarding the 8bitmime, which caused the message to bounce again. I then disabled the antivirus checking. Tried the message again, and it went through, so it was amavis changing the message to 8 bit. I then downloaded 2.4.5 and rebuilt it and installed it. Turned the antivirus scanning back on, and the message went through again. I saw this in the release notes - prefer '7bit' as Content-Transfer-Encoding when attaching original message or its headers (message/rfc822 or text/rfc822-headers) to DSN or to a defanged mail, and only specify '8bit' when necessary; So I don't know if it was a known problem or not, but I do know it is now working without discarding the 8bitmime.
I am not an expert on mime encapsulation but AFAIK every mime entity can have a different encoding. If you are using AlterMIME to add Footers to outgoing mails or something similar, that could also introduce 8bitmime.
I saw something about that as well in the release notes, but I don't use that. So thanks once again. Your help was invaluable in tracking this down at least to the source of the conversion, which was the point causing me the most confusion. I couldn't figure out that while the reports I found on the bounce messages pointed to an 8bitmime problem, I was sending a 7 bit message and shouldn't have been affected. I will file a bug report on amavisd-new, but realize since Postfix works correctly when connecting to a server that does not advertise 8bitmime, this would only show up in a situation like mine. Thanks again for all your help. -- Joe Morris Registered Linux user 231871 running openSUSE 10.2 x86_64 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Sandy Drobic wrote:
Postfix does not convert any 7bitmime to 8bitmime, it merely converts 8bitmime to 7bitmime, if the destination server does not support 8bitmime.
Though there was a bug recently in a perl module that amavisd-new uses (Net::Cmd 2.2.7), which caused ALL mail to be converted to UTF8 regardless of the encoding of the headers. It caused a lot of screams here in Germany because the problem immediately became apparent with the German "Umlaute" äüö etc.
What version of amavisd-new was that? It sure seemed like to me that amavisd-new 2.4.3 was converting every email to 8 bit mime, and I don't have Net::Cmd 2.2.7 installed. Amavisd-new version 2.4.5, the current stable version, appears to work correctly in that regard. I was just wondering which was really the culprit, the perl module or amavisd-new. I would assume I can rule out all perl modules used by amavisd-new since I only updated amavisd-new, and everything seems to be working as expected. -- Joe Morris Registered Linux user 231871 running openSUSE 10.2 x86_64 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Joe Morris (NTM) wrote:
Sandy Drobic wrote:
Postfix does not convert any 7bitmime to 8bitmime, it merely converts 8bitmime to 7bitmime, if the destination server does not support 8bitmime.
Though there was a bug recently in a perl module that amavisd-new uses (Net::Cmd 2.2.7), which caused ALL mail to be converted to UTF8 regardless of the encoding of the headers. It caused a lot of screams here in Germany because the problem immediately became apparent with the German "Umlaute" äüö etc.
What version of amavisd-new was that? It sure seemed like to me that amavisd-new 2.4.3 was converting every email to 8 bit mime, and I don't have Net::Cmd 2.2.7 installed. Amavisd-new version 2.4.5, the current stable version, appears to work correctly in that regard. I was just wondering which was really the culprit, the perl module or amavisd-new. I would assume I can rule out all perl modules used by amavisd-new since I only updated amavisd-new, and everything seems to be working as expected.
Net::Cmd is a prerequisite of amavisd-new and part of libnet. If you use CPAN to stay up-to-date you have probably updated that module. During the start of Amavisd-new it logs all perl module versions to the maillog: Mar 4 11:21:29 katgar amavis[12186]: Module Net::Cmd 2.26 The trouble with Net::Cmd 2.27 was, that everything was converted to UTF8. Again, this is NOT part of Amavisd-new, it only depends on it. Currently I have Amavisd-new 2.4.4 installed with no problems. -- Sandy List replies only please! Please address PMs to: news-reply2 (@) japantest (.) homelinux (.) com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Sandy Drobic wrote:
Net::Cmd is a prerequisite of amavisd-new and part of libnet. If you use CPAN to stay up-to-date you have probably updated that module. During the start of Amavisd-new it logs all perl module versions to the maillog:
Mar 4 11:21:29 katgar amavis[12186]: Module Net::Cmd 2.26
You are correct again. It is a part of SUSE's perl-5.8.8 package, and mine IS the same version , 2.26, so that wasn't causing my problem in this case. It also helps me to understand that this had to be a bug in the 2.4.3 version, as all that changed is it is now running 2.4.5, i.e. Mar 4 11:30:23 jmorris amavis[4048]: starting. /usr/sbin/amavisd at jmorris.ho me amavisd-new-2.4.5 (20070130), Unicode aware, LC_ALL="POSIX", and now, even without the 8bitmime discard, emails are still going through to the trouble domains through Exchange. I decided to leave the 8bitmime discard on at the office, where I have less control over what is sent, but have removed it here at home, which I use as my test bed.
The trouble with Net::Cmd 2.27 was, that everything was converted to UTF8. Again, this is NOT part of Amavisd-new, it only depends on it.
Understood. Since I don't keep that up to date on the different perl modules, this one did not affect me, though apparently amavisd-new 2.4.3 DOES convert every message to 8 bit somehow, and that is the default version with 10.2. It was an interesting problem to track down and I learned a lot.
Currently I have Amavisd-new 2.4.4 installed with no problems.
Perhaps that was when this bug was fixed. I already filed a bug report, so we'll see what is done. I used to run 9.3 and I believe amavisd-new 2.3.3 in the office until a few weeks ago, which did not have that problem. So thanks much for your insights, they were very helpful in tracking this to resolution. -- Joe Morris Registered Linux user 231871 running openSUSE 10.2 x86_64 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
participants (2)
-
Joe Morris (NTM) -
Sandy Drobic