On Sun, 28 Aug 2011 19:36:55 +0530, James Knott wrote:

Anton Aylward wrote:

...

So I'm wondering if there is a way to get postfix out of the loop for INCOMING mail.

On my home network, I've been running UoW imap and fetchmail for years. AFAIK, I'm not running Postfix, which I thought was a replacement for Sendmail.

really? if i remember correctly, it's included by default. what happens if you run (from a root console) "rcpostfix status" ? -- phani. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

James Knott said the following on 08/28/2011 10:06 AM:

Anton Aylward wrote:

...

So I'm wondering if there is a way to get postfix out of the loop for INCOMING mail.

On my home network, I've been running UoW imap and fetchmail for years. AFAIK, I'm not running Postfix, which I thought was a replacement for Sendmail.

Indeed. My point. If you are the only user there is no need for Postfix to process incoming mail. Outgoing, that's another matter: queueing, determining which SMTP server, error handling. Yes I suppose I could get by with Thunderbird if and only if this was the only station on the LAN generating outgoing mail and I didn't care about error handling, and more. At the moment, for incoming mail, I have fetchmail at the front end feeding in to the MTA mda '/usr/bin/procmail -d %T' You'll see that in many HowTos. Nothing remarkable there. But I'm also running Dovecot. It has its own LDA. If you use that it indexes the mail as it arrives. But my procmail delivers to many folders. I can't see how I can use Dovecot's LDA with that. -- Any man who afflicts the human race with ideas must be prepared to see them misunderstood. --Henry Louis Mencken -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

James Knott wrote:

Anton Aylward wrote:

...

If you are the only user there is no need for Postfix to process incoming mail.

Outgoing, that's another matter: queueing, determining which SMTP server, error handling.

At work, I was planning on using the existing SMTP server and just using fetchmail to get POP mail from them for the IMAP server, just like I do at home.

That's exactly what i do w/my ISP -- cept they run IMAP as well. I'm still using 'sendmail', but I won't try to convince anyone how great it is! -- it does work, though, usually. ;-) So haven't desired to 'fix it' by changing over to postfix.. But I don't think it would be good to try to run w/o a mail daemon. When cron jobs, or at scripts, or any other processes run and send mail, do you really them to send status or fault email that's probably addressed to 'postmaster' or root at 'locahost', some 'ISP' and hope that they will know that since the email came from your machine, then local host would mean send it to your queue? I doubt many (if any) ISP's would try to implement such... There have been times (very recently) when my username wouldn't resolve (I've been using samba/winbind for account resolution, and I haven't had samba working correctly since I upgraded to 11.4 in April (a led to b, b led to c, c led to d, etc... )... This happened for about ... maybe 16 hours before I caught the problem. How did I catch it? No incoming email. But fetchmail was running... sendmail was just 'queuing it' (had been told not to bounce, but continue trying for some number of days...). Once I fixed the user-account lookup, the email started pouring in from queue -- maybe 2-4 hundred (get a couple to several hundred/day depending on whether or not my 'lkml' sub it active at the time... ;-) (lkml=linux kernel mailing list -- high volume)... Nice thing about having email in standard formats, you can write own own little stat progs:

ll daily_stats hourly_stats -rwxr-x--- 1 40 Jun 7 2005 daily_stats* -rwxr-x--- 1 834 Sep 27 2005 hourly_stats* # emails/day by date daily_stats|tail 188 20110819 279 20110820 354 20110821 496 20110822 520 20110823 503 20110824 401 20110825 449 20110826 293 20110827 312 20110828

emails received by hour: as a graph for past 48 hours:

hourly_stats |tail -48 20110826:19 ****************** 20110826:20 ********** 20110826:21 ***** 20110826:22 ********** 20110826:23 *************** 20110827:00 ********** 20110827:01 ************ 20110827:02 ************* 20110827:03 ****** 20110827:04 ****** 20110827:05 ********************** 20110827:06 ****************** 20110827:07 ******************** 20110827:08 ************************* 20110827:09 ****************************** 20110827:10 **************** 20110827:11 ************* 20110827:12 ****************** 20110827:13 ************ 20110827:14 ****** 20110827:15 ************ 20110827:16 ************* 20110827:17 ************ 20110827:18 ********* 20110827:19 ** 20110827:20 ***** 20110827:21 **** 20110827:22 * 20110827:23 ******** 20110828:00 ************** 20110828:01 ***************** 20110828:02 ************* 20110828:03 ************* 20110828:04 *************** 20110828:05 ******************** 20110828:06 ************* 20110828:07 ************************************* 20110828:08 ************************************ 20110828:09 ***************** 20110828:10 ************* 20110828:11 ************ 20110828:12 ************************ 20110828:13 ******************** 20110828:14 ******* 20110828:15 *** 20110828:16 ******* 20110828:17 *************

Still working after 6 years... (when my email sorting prog stores a message, it. Those scripts are trivial shell scripts that process the log file writes a 1 line log message that looks like:

tail -3 mail/.log 20110828:184935 (Sun) SPAM tagged mail , Supposedly to , thru . mail (***SPAM*** Fat Burning Furnace) received ("fbf" ) (2.11sec processing) 20110828:185035 (Sun) OpenSuSE mail (Re: [opensuse] installing openSUSE on an older pc)[CLEANED] received (larry stotler ) (2.12sec processing) 20110828:185935 (Sun) OSuSE Proj & Factory mail (Re: [opensuse-factory] Re: gdk-pixbuf/gdk-pixbuf.h: No such file or directory)[CLEANED] received (ilya chernykh ) (2.34sec processing)

The one's that say 'CLEANED' have the "[listname]" from the subject, It's annoying to open a folder 'OpenSuse', and see [opensuse] at the start of every subject -- doesn't really add much to a subject. So... by running a mail server, you can get some flexibility and reliability that you might might find difficult to do otherwise, but keeping it all running can be 'educational' at times... -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Per Jessen said the following on 08/29/2011 01:55 AM:

I think(!) you can use fetchmail+dovecot without either knowing about the other. fetchmail does deliveries to your maildir, dovecot will pick up the changes, although delayed.

That's what I'm doing. Fetchmail uses procmail to run through spamassassin and procmail plonks it into the mailbox. I just wondered of procmail should - or could - use dovecot's lda. Perhaps this is really a question for the dovecot list .. -- A nation or civilization that continues to produce soft-minded men purchases its own spiritual death on an instalment plan. --Martin Luther King, Jr. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Per Jessen said the following on 08/29/2011 08:35 AM:

Anton Aylward wrote:

...

Per Jessen said the following on 08/29/2011 01:55 AM:

...

I think(!) you can use fetchmail+dovecot without either knowing about the other. fetchmail does deliveries to your maildir, dovecot will pick up the changes, although delayed.

That's what I'm doing. Fetchmail uses procmail to run through spamassassin and procmail plonks it into the mailbox.

Ah, I missed that.

...

I just wondered of procmail should - or could - use dovecot's lda.

It would no doubt improve your overall setup, and you'd be able to use e.g. quotas and sieve. I'm guessing fetchmail call any arbitrary executable and then provide the mail on stdin? If that works, I think you can just call "/usr/lib/dovecot/deliver".

... And that's what worries me. With procmail I can plonk the mail down into the folder I want. Where is dovecot going to 'deliver' to? I ask this because I have a non-standard arrangement. Having accidentally wiped out /var/spool/mail in past upgrades I now have my inbox under ~/Mail and my folder tree in subdirectories thereof. It all gets backed up nicely. My backup strategy isn't very aggressive about things that are 'system', things that I can get back from the installation DVD or repositories. It is aggressive about /home. -- What is wanted is not the will to believe, but the will to find out, which is the exact opposite. --Bertrand Russell, _Skeptical Essays_ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

` Anton Aylward wrote:

Per Jessen said the following on 08/29/2011 01:55 AM:

...

I think(!) you can use fetchmail+dovecot without either knowing about the other. fetchmail does deliveries to your maildir, dovecot will pick up the changes, although delayed.

That's what I'm doing. Fetchmail uses procmail to run through spamassassin and procmail plonks it into the mailbox.

I just wondered of procmail should - or could - use dovecot's lda. Perhaps this is really a question for the dovecot list ..

Why would you use another lda when what you have works? (I have something similar... a 'pre-procmail-era script, that does similar, written in perl..., it calls spamassassin on the email 'inline' and sorts spam into a spam folder. -- else it sorts it according to various conditions (i.e. perlregex's on From/To/Subject)....) I tried procmail when it first came out to see if I could replace my script... at the time, procmail wasn't quite as reliable. So stuck w/script .. anytime it's had a prob, it's easier to fix than trying to get a another project to fix something... (course when things break, I can't blame others either...oh well...;-) ). Dovecot pics up the changes to mail folders anytime a client asks for an update. It may use 'inotify' to get notice as soon as mail folder is written to... but TB isn't setup to receive a 'push' from an IMAP server, so it is set to check once a minute. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

On Tuesday 30 Aug 2011 14:19:33 Anton Aylward wrote:

Linda, there's not need to cc me if you're sending to the lost

I think that sums us up nicely ;) Bob -- Registered Linux User #463880 FSFE Member #1300 GPG-FP: A6C1 457C 6DBA B13E 5524 F703 D12A FB79 926B 994E openSUSE 11.4 64-bit, Kernel 2.6.37.6-0.5-desktop, KDE 4.6.5 Intel Core2 Quad Q9400 2.66GHz, 8GB DDR RAM, nVidia GeForce 9600GT -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Anton Aylward wrote:

I'm told that the dovecot lda does the indexing on delivery. So its already indexed when I visit with TB. So there isn't the delay.

That is correct. -- Per Jessen, Zürich (21.7°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Per Jessen said the following on 08/30/2011 10:16 AM:

Anton Aylward wrote:

...

As you say, when I visit a folder, dovecot (re-)indexes it. That's very useful if I want to do a search on the server.

Wait - are you talking about a full text index? Normally dovecot keeps an index of a maildir to avoid having to reread the entire directory, but a fulltext index is something else.

No I'm not talking about a full-text index. Nor am I talking about a Maildir And yes, I realise Thunderbird builds its own indexes ... and I wish it wouldn't -- "The capacity to learn is a gift; The ability to learn is a skill; The willingness to learn is a choice." - Brain Herbert, -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Per Jessen said the following on 08/30/2011 12:55 PM:

Regardless, dovecot will maintain an index and if you use dovecots LDA, it update the index at time of delivery.

So we're back to where we started, yes that was what I wanted. No, how can I use it with fetchmail and ... I was saying procmail but... The "BUT" is the big one. People are recommending 'sieve'. OK, but HOW? The stuff I see on the net assimes mail is coming in via SMTP into Postfix whihc has - or maybe not - a like to spamd and then loops abck to the dovecot LDA and somehow .... somehow ... somehow ... to the sieve. But the sieve examples I see aren't as flexible as Procmail. Now I can see 'deliver' being used in a pipeline as a LDA with procmail instead of plunking it in the INBOX. That's fine for the INBOX, even if it is causing another process creations just to do that. But what about all the other boxes that my procmail delivers to? Ah, right, 'sieve'. But wasn't that replacing procmail? So now I have fetchmail .... something something into Spamassassin somehow and then into sieve. Only that's not what I want. My current procmail does whitelisting and delivery (for example of this list), blacklist, handling of known spam - BEFORE passing to SpamAssassin. Then if its over the threshold for spam it goes into various buckets. The algorithm isn't perfect, some marginal stuff shouldn't be there, and sometimes a reasonable post gets a large score for whatever reason. Procmail handles all this gracefully and without the odd syntax that sieve has. I'm willing to put up with the slightly-removed-from-sendmail geek-happy syntax of sieve, but I can't see how to do what procmail is doing so easily. Having to run everything through spamassassin will be a performance pig! -- "We hear very little and we understand even less" - An anonymous Englishman, The opening scene of Cassablanca -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Per Jessen said the following on 08/30/2011 03:03 PM:

[snips...]

...

The "BUT" is the big one. People are recommending 'sieve'. OK, but HOW?

sieve scripts are located in your users home directory - for dovecot, the default script is ".dovecot.sieve".

I got that from the docco ...:-)

There is some fairly good reference/example stuff for sieve "out there".

Yes, I know, I've googled and followed links for the best part of day on this already and ...

...

The stuff I see on the net assumes mail is coming in via SMTP into Postfix whihc has - or maybe not - a like to spamd and then loops abck to the dovecot LDA and somehow .... somehow ... somehow ... to the sieve. But the sieve examples I see aren't as flexible as Procmail.

Please read that again. Please also go back though this thread and look at other exchanges about K.I.S.S and not making use of Postfix when running fetchmail

sieve is invoked by dovecot as part of delivery. For instance, here's a bit of my dovecot.conf:

protocol lmtp { mail_plugins = sieve quota }

And how is that relevant ? I see what it does but how is it relevant to what I'm faced with? Quotas aren't a problem for me. I can throw another 1T at my mail store if I want. Its MY mail store.

I don't know procmail very well, so regarding how it compares to sieve, well, you tell us when you're done converting :-)

HAHAHAHA!

...

Ah, right, 'sieve'. But wasn't that replacing procmail?

Yup.

...

My current procmail does whitelisting and delivery (for example of this list), blacklist, handling of known spam - BEFORE passing to SpamAssassin.

I'm not sure you'll be able to run spamassassin as part of a sieve script.

Then what use is it to me?

I would recommend running that under the control of postfix,

So I now do fetchmail from @antonaylward.com which is off site and push that into Postfix which promptly delivers it back to my off site address.. WTF! My home site is distinct from the domains I own (and those are implemented at various ISP). No doubt other people are in this situation. My cable provider will not let me run a 'server', be it SMTP or HTTP. and they run scans. So all my mail is delivered off site. That includes the mailboxes at my cable provider's address. Which is why I use fetchmail. And to be able to test, I need to be able to send from home to those addresses, so Postfix doesn't see them as local delivery. The whole point of this is to NOT involve Postfix! Mail does not come in. The is no "port 25 incoming". Mail is fetched. Postfix is not currently in the loop for that. Postfix on the mail hub is there for convenience when I'm not using TB. TB can send mail without Postfix.

maybe with amavisd. A while ago, I wrote an article about a postfix-fetchmail-spamassassin config:

http://jessen.ch/articles/spamassassin-and-postfix/

Yes I saw that when googling. A good example of what I'm trying to avoid.

...

Then if its over the threshold for spam it goes into various buckets. The algorithm isn't perfect, some marginal stuff shouldn't be there, and sometimes a reasonable post gets a large score for whatever reason.

Procmail handles all this gracefully and without the odd syntax that sieve has.

procmail just uses it's own odd syntax :-) Sure, sieve is different, but you'll get used to it.

All syntax is odd to someone; try learning Chinese as an English speaker. Or vice versa.

Leaving out running spamassassin, what else do you do with procmail?

I thought I'd covered that more than once on this thread. Whitelisted, blacklisted, dealt with incorrectly formed headers, archaic addresses, lists that won't unsubscribe and delivery to list mailboxes. You'd be amazed how many list use the "[listname]" format on the subject line, which with a pattern match makes delivery to the right box easy. :-)

Your whitelisting probably isn't necessary, so leave that out too.

I disagree.

...

Having to run everything through spamassassin will be a performance pig!

Not at all. You won't notice it on any halfway modern machine. I have a test system running SA on 400MHz PII - a bit slow, but works fine.

My mail hub is about the same and I've experimented. Running spamassassin on lists that I want to accept and file is a waste of time and, yes, I've measured it, it does slow delivery down. More to the point, some of the things I whitelist are whitelisted because spamassassin marked them as spam in the past. The contents of this list included! Whitelisting and blacklisting make a lot of sense for me. YMMV. -- As security or firewall administrators, we've got basically the same concerns [as plumbers]: the size of the pipe, the contents of the pipe, making sure the correct traffic is in the correct pipes, and keeping the pipes from splitting and leaking all over the place. Of course, like plumbers, when the pipes do leak, we're the ones responsible for cleaning up the mess, and we're the ones who come up smelling awful... -- Marcus J. Ranum -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Patrick Shanahan said the following on 08/30/2011 06:15 PM:

...

WTF!

yeah, wt...

I run my own mail server, but also use gmail, rr, ...

from my ~/.fetchmailrc for rr poll pop-server.indy.rr.com tracepolls with proto POP3 timeout 60 user 'pshanahan' there with password 'xxxxxxxxxxxx' is 'paka' here options fetchall stripcr mda '/usr/lib/sendmail -i -oem -f %F %T'

of course, /usr/lib/sendmail is actually postfix which hands off to procmail which invokes spamd after delivering msgs which do not need spam filtering.

That seems long winded to me I run fetchmail Example from ~/.fetchmailrc poll mail.antonaylward.com with proto POP3 user 'xxxx@antonaylward.com' there with password 'XXXXX' is 'anton' here options fetchall smtphost HOME mda '/usr/bin/procmail -d %T' Note that last line. There is no need to run though Postfix to get to procmail And as you say, procmail can deliver messages that don't need spam filtering. INCLUDERC=$LIB/whitelist.rc INCLUDERC=$LIB/blacklist.rc INCLUDERC=$LIB/mailinglists.rc # Another kind of whitelist INCLUDERC=$LIB/attachment.rc INCLUDERC=$LIB/likelyspam.rc INCLUDERC=$LIB/fonts.rc Fonts? Well if its in russian or chinese then I treat it as spam Likely Spam? Call it 'known spam' Sometimes certain patterns crop up, usually in subject lines, so predictably procmail can handle them faster and easier than spamassassin. The whitelist, blacklist and mailing list is a straight forward fgrep. Again, faster than spamassassin.

ps. have you tried running a mail server from a non-standard port? Maybe not checked by your isp....

Yes. Now I only have to get the rest world to stop using port 25 for SMTP. -- All national institutions of churches, whether Jewish, Christian, or Turkish, appear to me no other than human inventions, set up to terrify and enslave mankind, and monopolize power and profit. --Thomas Paine -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Per Jessen said the following on 08/31/2011 02:11 AM:

Anton Aylward wrote:

...

...

...

[...] But the sieve examples I see aren't as flexible as Procmail.

Please read that again.

Please also go back though this thread and look at other exchanges about K.I.S.S and not making use of Postfix when running fetchmail

Yes, I remember all of that, but if you want to use dovecot local delivery, I don't think there is a way around it.

*sigh* Oh for more Perl-people ... TAANWTDI

I took your "but HOW?" to mean you were having difficulties getting sieve to work in/with dovecot.

No, the HOW was 'HOW to get spamassassin integrated without using Postfix?'

...

...

I'm not sure you'll be able to run spamassassin as part of a sieve script.

Then what use is it to me?

Sorry, I didn't realise that running spamassassin from sieve was a hard requirement. That's your show-stopper.

*sigh* Anyone? I think I need to bump this to the [dovecot] list then.

...

which promptly delivers it back to my off site address..

WTF!

Bad postfix configuration. If postfix attempts to relay mail sent to @antonaylward.com instead of delivering it locally, antonaylward.com has probably not been configured as a virtual domain?

Correct. How else can I send to @antonaylward.com from my HOME system if my HOME system has antonaylward.com set up to deliver locally?

BTW, regarding my comments on SA performance, I neglected to emphasize that there is a big performance difference between running a spamc/spamd setup and running SA via 'spamassassin'.

Yes, I'm aware of that. -- Objects in calendar are closer than they appear. -- Jim Duncan -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

` Per Jessen wrote:

Anton Aylward wrote:

...

...

...

which promptly delivers it back to my off site address..

WTF!

Bad postfix configuration. If postfix attempts to relay mail sent to @antonaylward.com instead of delivering it locally, antonaylward.com has probably not been configured as a virtual domain?

Correct. How else can I send to @antonaylward.com from my HOME system if my HOME system has antonaylward.com set up to deliver locally?

Unless your home system is 'antonaylward.com', then you don't do that. My home system has a linux-computer handling the computers inside the house as a subnet. So you just send email to a correctly configured host inside the subnet, or the subnetname, itself, if you've aliased that to the local mailserver host. i.e. myhouse.mydomain.net =PTR> house-serve.myhouse.mydomain.net So email to any number of aliases will have it NOT go out, but have the server recognize it as local. Worst case, which is actually a good connectivity case is <user>@mydomain.net, -- goes out to my ISP and then gets fetched back down by the server... You are stressing about moving to a new filter system and integrating spamassassin, yet you haven't even _tried_ just plopping in Dovecot and seeing how the performance is... It's like salting your food before tasting it...what if it already salted by the chef? But perhaps you like the challenge.. still , I prefer stepwise upgrades...like getting dovecot working first seems like a valid first step that you'll have to do no matter what .. you may decide you'll need to make some minor config file change later on, if you don't like the performance, but at least you'll know your readers can read it. and it is already setup to handle your local dirs... But, it sounds like you'd prefer to setup everything and then change to a new IMAP server all at once. Me, I'm sure I will have something that won't work right, so getting 1 piece to work at a time is *safest* for me (not that I always take the safest route! (*sigh*))... -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Sorry for the double post, but there's a HOWTO on how to dovecot '_on how to re-filter your mail automatically as it comes in using a util called 'incron... it uses linux imap to get notifed as soon as someting changes the file... See http://wiki2.dovecot.org/HowTo/RefilterMail -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org