Hi all, This is just out of curiosity, but fixing the problem would lessen some irritations too. I have fetchmail --> sendmail --> procmail <spamassassin> (if I understand the order of events correctly. Whenever I download lots of e-mail at a time, I notice my system gets slower and less responsive, sometimes so bad that apps like evolution and mozilla seem to freeze until the mail is delivered. I never noticed the system monitor showning increased cpu activity, and top didn't turn up anything either. Today I realised that it's the hard drive that's going bezerk when I'm downloading mail. Why? It's not a slow drive, at least not slow enough that the colletive size of the mail should tax it. Seagate 40gb ATA-100 5400rpm disc (reiserfs) on a ATA-100 controller. Yast2 reports DMA to work in ATA-100 mode. Any clues? Thanks Hans
--=.nt:9b_FlrjEk+Y
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
On Tue, 26 Aug 2003 23:59:23 +0200
H du Plooy
Whenever I download lots of e-mail at a time, I notice my system gets slower and less responsive, sometimes so bad that apps like evolution and mozilla seem to freeze until the mail is delivered.
Yes, spamassassin scans are very disk and resource intensive. However
you can improve you system response by:
(1) Using spamd/spamc combo instead of spamassassin.
(2) Use a lock file to make sure only one message is process at a time.
(3) Limit the size of the mail scanned since spam messages are small.
For example:
# The lock file ensures that only 1 spamassassin invocation happens
# at 1 time, to keep the load down.
#
:0fw: spamassassin.lock
#
# Limit size to 250 KB
* < 256000
#
# Use spamc
| spamc -d 127.0.0.1 -f -u
Today I realised that it's the hard drive that's going bezerk when I'm downloading mail.
You can improve this by using hdparm -i <device> to find out the highest transfer mode your hard drive support and then use hdparm to set it to the highest mode both your system and hard drive will allow. You might want to enable 32 bit file transfer, etc. For example: /sbin/hdparm -d1 -m16 -X udma2 -u1 -W1 -A1 -c3 -k1 /dev/hda Please refer to the hdparm manpage.
It's not a slow drive, at least not slow enough that the colletive size of the mail should tax it.
A faster drive does make difference. I had the same problem as you do with my old drive. Tuning helped a little bit, but problem remains when that is a lot of mail. However, the problem immediately went away after I installed my new 80 GB "WD Caviar Special Edition" with 8Mb cache. Charles -- Your job is being a professor and researcher: That's one hell of a good excuse for some of the brain-damages of minix. (Linus Torvalds to Andrew Tanenbaum) --=.nt:9b_FlrjEk+Y Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.2 (GNU/Linux) iD8DBQE/S+Iq3epPyyKbwPYRAll3AJ45Hyu9sgdlviHYV+2dt40PuhF9SgCcCUGI /kazQs93LvdVsgyK8Z0Tx/E= =Resg -----END PGP SIGNATURE----- --=.nt:9b_FlrjEk+Y--
On Wed, 2003-08-27 at 00:41, Charles Philip Chan wrote:
(1) Using spamd/spamc combo instead of spamassassin. That's what I've been using. Sorry, I thought: spamd/spamc = spamassassin.
(2) Use a lock file to make sure only one message is process at a time. Thanks, this helped a great deal.
(3) Limit the size of the mail scanned since spam messages are small. This wouldn't make much of a difference, as most of the mail I get are quite small.
For example:
# The lock file ensures that only 1 spamassassin invocation happens # at 1 time, to keep the load down. # :0fw: spamassassin.lock # # Limit size to 250 KB * < 256000 # # Use spamc | spamc -d 127.0.0.1 -f -u
Just for the sake of interest, if I understand the manpage correct, can I run spamd on another machine in my local network and have spamc on my machine connect to the other one and have it do the hard work? The "other machine" happens to be a PII-233 with two ancient first generation IDE drives, so I doubt if it will save me any time, but I'm curious.
that is a lot of mail. However, the problem immediately went away after I installed my new 80 GB "WD Caviar Special Edition" with 8Mb cache. Much as I would like to have one of those, that's going to have to wait until next year - I've spent too much money on the machine already this year...
Thanks for the help Hans
On Wednesday 27 August 2003 2:00 pm, H du Plooy wrote:
On Wed, 2003-08-27 at 00:41, Charles Philip Chan wrote:
(1) Using spamd/spamc combo instead of spamassassin.
That's what I've been using. Sorry, I thought: spamd/spamc = spamassassin.
(2) Use a lock file to make sure only one message is process at a time.
Thanks, this helped a great deal.
I lost your original post so I have to reply to this one, but do you have procmail sort the posts for sle out before spamassassin? This has helped a great deal on my machine. I also wanted to thank charles. The lock file has helped cpu usage, it used to go to 80-90% now it stays at about 20% without taking significantly longer. -- Franklin Maurer Using SuSE 8.2 Pro
On Wed, 27 Aug 2003 20:00:33 +0200
H du Plooy
That's what I've been using. Sorry, I thought: spamd/spamc = spamassassin.
spamd is a daemonized version of spamassassin. One can either just used spamassassin directly or the use spamd/spamc combo.
Just for the sake of interest, if I understand the manpage correct, can I run spamd on another machine in my local network and have spamc on my machine connect to the other one and have it do the hard work?
You should be able to do that, but I haven't tried it myself.
The"other machine" happens to be a PII-233 with two ancient first generation IDE drives, so I doubt if it will save me any time, but I'm curious.
If the other machine is lying idle, why not use it? It is better than tying up resources on your machine.
Much as I would like to have one of those, that's going to have to wait until next year
Did you also tune your disk with hdparm like I suggested?
Thanks for the help
No problem. Charles -- "The IETF motto is 'rough consensus and running code'" -- Scott Bradner (Open Sources, 1999 O'Reilly and Associates)
On Wed, 2003-08-27 at 23:52, Charles Philip Chan wrote:
If the other machine is lying idle, why not use it? It is better than tying up resources on your machine. It's running seti. If it had a fast hard disc, I would have done it, but here we pay for local calls, and I don't want to waist time downloading e-mail. One day, when I replace this hard disc, I'll stick it in the old PC and try it out.
Did you also tune your disk with hdparm like I suggested? I have tried it before, as per suggestion from the list (on another thread), but it didn't make a noticeable difference on my system over the default "dma = 1"
Thanks Hans
On Tuesday 26 August 2003 23:41, Charles Philip Chan wrote: <SNIP>
(2) Use a lock file to make sure only one message is process at a time.
How do I do this?
(3) Limit the size of the mail scanned since spam messages are small.
For example:
# The lock file ensures that only 1 spamassassin invocation happens # at 1 time, to keep the load down. #
:0fw: spamassassin.lock
# # Limit size to 250 KB * < 256000 # # Use spamc
| spamc -d 127.0.0.1 -f -u
Where do I find that? I've looked through the files in /usr/share/spamassassin and /etc/mail but can't find those lines anywhere? Cheers Dylan -- Sweet moderation Heart of this nation Desert us not We are between the wars - Billy Bragg
On Wed, 2003-08-27 at 20:51, Dylan wrote:
# The lock file ensures that only 1 spamassassin invocation happens # at 1 time, to keep the load down. #
:0fw: spamassassin.lock
# # Limit size to 250 KB * < 256000 # # Use spamc
| spamc -d 127.0.0.1 -f -u
Where do I find that? I've looked through the files in /usr/share/spamassassin and /etc/mail but can't find those lines anywhere?
This is all supposed to be in your ~/.procmailrc Hans
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wednesday 27 August 2003 01:52 pm, H du Plooy wrote:
On Wed, 2003-08-27 at 20:51, Dylan wrote:
# The lock file ensures that only 1 spamassassin invocation happens # at 1 time, to keep the load down. #
:0fw: spamassassin.lock
# # Limit size to 250 KB * < 256000 # # Use spamc
| spamc -d 127.0.0.1 -f -u
Where do I find that? I've looked through the files in /usr/share/spamassassin and /etc/mail but can't find those lines anywhere?
This is all supposed to be in your ~/.procmailrc
I've been following this thread and a question has come to mind... I don't use fetchmail, procmail etc, just lowly kmail to download and read my mail. (don't have time to learn about fetchmail, procmail etc, right now). I set up a filter in kmail to use spamc to sort out spam mails and its working well - except for the horrible slowdown. I've read the spamassasin docs but I don't see any info about how to setup a "lock" file, especially in conjunction w/ kmail. Can this be done with kmail? If so how. Thanks for your help. ps, If someone believes I'm hijacking the original thread I apologize, I just thought this question belonged here. Thanks again - -- dh Don't shop at GoogleGear.com! -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (GNU/Linux) iD8DBQE/TZlfBwgxlylUsJARAmijAJ4ujQO6jdJEcTq23FLzq+2O80Fz3wCgjz7A b4cB27QBCClr9RPmW6V8dtI= =OC+O -----END PGP SIGNATURE-----
On Tuesday 26 August 2003 13:59, H du Plooy wrote:
Hi all,
This is just out of curiosity, but fixing the problem would lessen some irritations too.
I have fetchmail --> sendmail --> procmail <spamassassin> (if I understand the order of events correctly.
Whenever I download lots of e-mail at a time, I notice my system gets slower and less responsive, sometimes so bad that apps like evolution and mozilla seem to freeze until the mail is delivered. I never noticed the system monitor showning increased cpu activity, and top didn't turn up anything either.
Today I realised that it's the hard drive that's going bezerk when I'm downloading mail. Why?
You also want to considder if virus scanning is necessary if this machine only fetches its own mail (not serving as a mail server for windows machines). Virus scanning for linus is (at this point in time) largely a waste of time and resources because there are no known viruses that infect linux and no virus scanner databases detect these (non-existant) viruses. (Amavis is simply a front end processor for the actual virus checker. It rips an email appart, writes the parts to a directory, scans that directory and then re-builds the email. - All very usefull if your machine serves mail to windows machines, but otherwise a waste of resources). -- _____________________________________ John Andersen
On Wed, 2003-08-27 at 11:18, John Andersen wrote:
You also want to considder if virus scanning is necessary if this machine only fetches its own mail (not serving as a mail server for windows machines). Virus scanning for linus
Thanks for the suggestion. I don't use virus scanning, since it's only mail for this machine. I can certainly understand the use one runs a mail server that passes mail on to windows clients though.
(Amavis is simply a front end processor for the actual virus checker. It rips an email appart, writes the parts to a directory, scans that directory and then re-builds the email. I understand spamd/spamc does the same. Any idea where it puts the temp file/directory? Putting that on a seperate disc (if I had an extra) would certainly also help!
Thanks Hans
participants (6)
-
Charles Philip Chan -
David Herman -
Dylan -
Franklin Maurer -
H du Plooy -
John Andersen