On Sun, Oct 17, 2004 at 04:48:04PM +0200, Mathias Homann wrote:

Am Sonntag, 17. Oktober 2004 16:38 schrieb Marcus Meissner:

...

On Sun, Oct 17, 2004 at 04:13:51PM +0200, Mathias Homann wrote:

...

one post to the list, three automated replies from bots or MTAs. great. IMO its time that suse starts some serious thinking about how subscriptions to lists could be checked.

bling.

I tried to find the address of one bot on the subscriber list, but failed. Let me see if I can get the other 3.

Ciao, Marcus

/me hands marcus assorted cookies and several pieces of cake for reacting to something like this that fast on a sunday afternoon. want coffee with that? ;)

The info@jam-software.com reply should be gone, but I did not find the custreply.com bot. (seems to come from paradise.net(.nz) or similar, but the subscribers from there do not autoreply if i mail them by hand). I did not get a third bot. Ciao, Marcus

The Monday 2004-10-18 at 09:04 +1300, Volker Kuhlmann wrote:

I suggest you simply change your From: to something useless, and all your problems evaporate. SuSE's lists are some of the very few who let you do this, kudos to SuSE for having a clue.

I have heard of this many times, and it involves setting the "envelope from" diferently of the "from". The only way I know is using Mutt, so the rest of us can not do the trick. Else... does somebody know how to do it using Pine and postfix? -- Cheers, Carlos Robinson

but some nonsense address (please really use an invalid toplevel domain, not just a made-up address, which actually may exists!)

I've seen one mail server which dumps all mails with a non-existant From: domain to /dev/null without notification, and that's none too silly. You'll find out if your posts are blackholed. Just make sure the address you use doesn't exist. Volker -- Volker Kuhlmann is possibly list0570 with the domain in header http://volker.dnsalias.net/ Please do not CC list postings to me.

/ 2004-10-21 15:04:20 +0200 \ Carlos E. R.:

The Thursday 2004-10-21 at 17:49 +1300, Volker Kuhlmann wrote:

...

...

but some nonsense address (please really use an invalid toplevel domain, not just a made-up address, which actually may exists!)

I've seen one mail server which dumps all mails with a non-existant From: domain to /dev/null without notification, and that's none too silly. You'll find out if your posts are blackholed. Just make sure the address you use doesn't exist.

The normal thing is to simply reject any email with non-existant "From" domain address. Time ago, SuSE distros configured sendmail that way; I haven't checked about postfix.

um. we still talk about the difference of Envelope and Header from? of course, non-existent _Envelope_ From should be rejected. lge

On Thursday, 21 October 2004 17.03, Philippe Vogel wrote:

O.K. nice discussion about that, but how we do get rid of this bots? A simple reauthentification, that you are homan, like on postfix-users list would help a lot. If you are a bot you can't answer the sign-up request.

removing the Reply-To from the subscription mails would probably also do something to help. That would at least make it more difficult to subscribe the support addresses that reply automatically with some "your mail has been received" mail

The Thursday 2004-10-21 at 17:03 +0200, Philippe Vogel wrote:

O.K. nice discussion about that, but how we do get rid of this bots?

Well, the thing is that if we, users, get a method to post to the list with a valid "envelope from" and invalid "header from" the robots can never reach us, simple users. Mmm, I had to look up "bot" in the jargon dictionary. I prefer the older "robot" O:-) That trick also helps to keep our mail box free of spam. Unfortunately, as I said, the only MUA that I know can do it is mutt. It is not possible with Pine, mozilla, kmail, balsa... etc, etc.

A simple reauthentification, that you are homan, like on postfix-users list would help a lot. If you are a bot you can't answer the sign-up request.

True. But ezmlm triggers on the "To" address alone, the contents of the email is ignored.

I inserted a reject-line in my mta but this does not help the others.

Mmmm... Perhaps it is easier to simply delete them as they come. They change address anyway, don't they?

I know it's hard to track back those bots. A worse solution would be to kill all signed up accounts not writing anything (which is no real solution).

I guess not, because there must be many readers out there doing simply that, reading. -- Cheers, Carlos Robinson

The Saturday 2004-10-23 at 19:24 +0300, Hugo wrote:

...

...

O.K. nice discussion about that, but how we do get rid of this bots?

Some of the suggestions I didn't understand.

I'll try again :-) When we post an email, it goes out with a "From" header with our own address. There is also a hidden header, added automatically later, called "envelope from" - the reason doesn't matter, and I don't know with precision. There is a mail howto that explains it, I think. The SuSE email list server daemon program (ezmlm), only looks at the "envelope from", not at the "from" address inside - like the mail man looks at the envelope, and not at the inside of the letter. This allows a trick. The poster (ie, you, me) can post using a false "From" address, but a real "envelope from" address. It works. But anybody seeing the list will never see the "envelope header", it is not saved: thus any direct replies will fail. The problem is how to set that "envelope from" in this way: only mutt allows it. And of course, only works for those posters capable of setting it, not for everybody. For example, not for me.

So, I'll just give and example of working system from a list that has been running for 10 years and doesn't have problems with spam or bots.

- Subscription done so that it can not be confirmed by just hitting reply

Yes, I agree with that.

- Replies to the list (not to the sender like here)

Arguable. It also has other problems. Don't think will ever change. You know that the welcome email to some of SuSE email lists actually recommends always replying to the original poster, not to the list, and when finally the problem is solved, the original poster should post to the list a second time, explaining the solution. This is on the Spanish list (R5.4). It was not before.

- Limit posting to subscribers (others bounce)

Yes, that is done here as well.

- Check the posts for list admin footer, if it is included bounce. This gets rid of sloppy quoting also as if the footer is included, then somebody replied to the email, wrote their answer on top and included everything else without deleting unneeded stuff. Not nice, so bounce. Bots automatically answering always include everything and thus they are bounced.

Ah, interesting trick... it should bounce with an explanation, or will cause endless complaints.

- If some bot is actually subscribed, kick them out (some subscriber might set up a holiday notification...)

Manually? That is done here also, of course - when found. Some are difficult to track, it appears.

There are other filters in place also (html, attachments...), but these I think are the main filters. No bots and no spam.

I think some of SuSE lists do have such filters, some not. It depends. Attachments can be helpful to send logs, for example - in the other hand, sending a large log instead of an excerpt seems excessive some times. -- Cheers, Carlos Robinson

The Sunday 2004-10-24 at 11:29 +0300, Hugo wrote:

OK, I got it now... I think I just got confused as I didn't find header named envelope-from... I once wrote a simple program for sending email. I was quite surprised how easy it was to fake everything. That was years ago, but perhaps it's still the same.

I think so.

...

This is on the Spanish list (R5.4). It was not before.

The version I got just said that "It's better this way. Trust us." Which is why I'm answering to the list (as usually with the lists)... comments like this somehow do not convince me :-) And in this case, I think the conversation is important. But sorry for breaking the rules. I'll try to remember that better in the future.

No, no, please! You missunderstood me. I said that _some_ of suse lists say that in their rules: in particular, the suse spanish list (suse-linux-s) does (pasting the rule would be OT here, I think, and of no use to those not speaking the language). That rule was not there when we subscribed, and we certainly do not follow it. I don't know which, if any, other suse list have that rule. Perhaps none, I haven't checked recently. I find that absurd and unpractical.

...

...

- If some bot is actually subscribed, kick them out (some subscriber might set up a holiday notification...)

Manually? That is done here also, of course - when found. Some are difficult to track, it appears.

Yes, manually. If only list members are allowed to subscribe, shouldn't it then be easy to see who "is a bot"? Ah, yeah it should. But here as the replies come to the sender and not to the list, only the senders are seeing those bots and not the list admin or others. And as they do not see the problem... there is no problem. Or?

The mechanism is that you are supposed to forward those bounces (a sample) complete with headers to the mail admin (-owner). He takes care of unsubscribing him. The problem is that some times (like now) the "from" address of the bot is not subscribed... making it quite difficult to find out which address is really receiving the list and forwarding to the bot or so. -- Cheers, Carlos Robinson

On Sun, Oct 24, 2004 at 04:18:25PM +0200, Mathias Homann wrote:

Am Sonntag, 24. Oktober 2004 13:26 schrieb Carlos E. R.:

...

The problem is that some times (like now) the "from" address of the bot is not subscribed... making it quite difficult to find out which address is really receiving the list and forwarding to the bot or so.

best idea would be:

1. change the way the list is managed, so that a simple reply to a mail does NOT confirm a subscription. Generate some dynamical link instead that the user has to follow or something like that. That way, nobody can just subscribe some weird autoreplying bots to the list via the web frontend.

2. send a message to all the subscribers explaining the reasons for step 3.

3. empty the subscription list.

that way, it would be close to impossible to subscribe a bot, and the people who actually care about getting the list will most probably also like the idea of getting rid of bots&bounces more than they dislike to have to subscribe again.

This is really not the case for this problem, since the autoresponder was most likely subscribed by a system administrator knowingly. Ciao, Marcus

The Sunday 2004-10-24 at 19:01 +0200, Marcus Meissner wrote:

...

that way, it would be close to impossible to subscribe a bot, and the people who actually care about getting the list will most probably also like the idea of getting rid of bots&bounces more than they dislike to have to subscribe again.

This is really not the case for this problem, since the autoresponder was most likely subscribed by a system administrator knowingly.

Mmmm... :-? Are you sure? Then he would know what would happen, if he is an admin. I would understand if he is a malicious admin... Notice that it is easy for anyone to subscribe any autoresponder: because the confirmation email will be answered by the autoresponder, and it will get subscribed. Thus, the confirmation email method is not valid any longer on its own: it has to be suplemented with a method to force human intervention by the subscriber. For example, some services I have to use force the user to answer writing between to precisely marked lines (no top posting, no bottom posting, but "midle posting"): anything outside is ignored. Thus, an autoanswer would be ignored. How the bots are getting in could be checked by saving both the subscription email, and the confirmation email, for later checking if necesary. -- Cheers, Carlos Robinson

The Monday 2004-10-25 at 18:16 +0200, Christian Boltz wrote:

Am Montag, 25. Oktober 2004 15:57 schrieb Carlos E. R.: [several theories how to avoid autoresponder subscriptions]

...

Notice that it is easy for anyone to subscribe any autoresponder: because the confirmation email will be answered by the autoresponder, and it will get subscribed. Thus, the confirmation email method is not valid any longer on its own: it has to be suplemented with a method to force human intervention by the subscriber.

OK, this could be avoided by removing the Reply-To header.

But there is no Reply-To header [...] hold on, the confirmation emails do have a Reply-To header, one created on the fly so that the responder can be identified.

But I wonder wyh the autoresponder problem only exists on this list (from those I read, including suse-linux with 100-200 mails daily which would be a more interesting target for autoresponders ;-)

Me too. :-o Not only that, but out of office replies are common here; right now I'm getting one from ROtt@nordit.de - in german, which I do not understand.

So maybe it's not this easy for autoresponders ;-)

...

How the bots are getting in could be checked by saving both the subscription email, and the confirmation email, for later checking if necesary.

You really believe that autoresponders subscribe theirself?

They might, a pissed employee, for example.

I think, if someone wants to subscribe an autoresponder intentionally, he will subscribe with a "normal" adress and, after subscribing, forward all mails from this adress to the autoresponder adress. And if the autoresponder in on another domain, it's very hard to track :-(

But we should avoid writing more mails in this thread than the autoresponders ;-)

X-) Well, if it gives the administrators ideas on how to clear this, I think we are welcome ;-) Yes, there is a way for anybody, including virus, to get any autoresponder address subscribed. I told it on this thread, but I'll repeat for clarification. Notice that I can send an email with a from address claiming to be you, for example (its terribly easy! That's why gpg signatures are interesting). If such a person, or virus, happens to email to listaddress-subscribe, with a from address set to paradise@custhelp.com, the suse list server will dutifully send a confirmation email, not to the person that really sent the email, but to paradise@custhelp.com, who being an autoresponder will autorespond: and hey presto! they are subscribed. But I wonder what on earth are the people at paradise@custhelp.com (are you reading?) think they are doing. Because their system must be registering all the email from the list as requests for help! By the way, notice the way their autorespond is set up: | |[===> Please enter your reply below this line <===] | |[===> Please enter your reply above this line <===] | You have to answer them putting your text inside those lines. Such a simple device, which is used by several such customer help email systems, serves the purpose of automatically cleaning spam and virus responses. -- Cheers, Carlos Robinson

The Sunday 2004-10-24 at 19:28 -0000, suse@karsites.net wrote:

If the mailing list is going to be altered, why can't we have a randomly generated *graphical* password displayed on the subscription page, that needs to be read by a real person, and then typed into a password box, to confirm subscription to the list.

I think the confirmation has be done by email, for two reasons: one is to check that the subscription email really works, and two, because it is not proper to force using a browser to subscribe to a mail list. -- Cheers, Carlos Robinson

The Thursday 2004-10-21 at 15:20 +0200, Lars Ellenberg wrote:

/ 2004-10-21 15:04:20 +0200 \ Carlos E. R.:

...

The normal thing is to simply reject any email with non-existant "From" domain address. Time ago, SuSE distros configured sendmail that way; I haven't checked about postfix.

um. we still talk about the difference of Envelope and Header from? of course, non-existent _Envelope_ From should be rejected.

I think both inexistent envelope and header from should be rejected. Unless there is a valid reason not to :-? My reasoning is that you can not answer to such emails, therefore they are not valid, useless. -- Cheers, Carlos Robinson