Hi guys, I've been following this mailing list since SUSE 8.0 came out, and have kept every single mail. I know that might sound excessive, but having my own copy of at least a piece of "the archives" has helped more than a couple of times when connectivity isn't available. Currently I have all my mail stashed in ~/Maildir - courier-imap gives me access to it. Nice setup as I can chop&change between mail clients any time I like. However, the sheer volume of mail is becoming a problem. It's getting slower and slower, searches through mail folders take for ever and max out my CPU for minutes at a time. I was wondering if there's a better way to store my mail? Thanks -- Kind regards Hans du Plooy Newington Consulting Services hansdp at newingtoncs dot co dot za
* Hans du Plooy <hansdp@newingtoncs.co.za> [01-31-05 14:54]:
I've been following this mailing list since SUSE 8.0 came out, and have kept every single mail. I know that might sound excessive, but having my own copy of at least a piece of "the archives" has helped more than a couple of times when connectivity isn't available.
Currently I have all my mail stashed in ~/Maildir - courier-imap gives me access to it. Nice setup as I can chop&change between mail clients any time I like.
However, the sheer volume of mail is becoming a problem. It's getting slower and slower, searches through mail folders take for ever and max out my CPU for minutes at a time.
I was wondering if there's a better way to store my mail?
I can and do part of the time, handle my own mail, but have taken to directing *all* mailing list to my gmail address. I then fetchmail to local for reading, but label and retain copies of all mail passing thru gamil. Searching for a particular email/subject/thread is much faster on gmail's site than on my own computer (aged 733) and a gig of space is available. If the space is not enough, get more gmail accounts and disburse the delivery of mail lists thru the different accounts. A plus, gmail's spam filtering is better than spamassassin on my local computer. Just a thought, -- Patrick Shanahan Registered Linux User #207535 http://wahoo.no-ip.org @ http://counter.li.org HOG # US1244711 Photo Album: http://wahoo.no-ip.org/photos
as the OP mentioned, it is important for them to have when not connected. I would think you could simply create sub-folders for different months/years, or database them with mysql or something..... I've been saving, slowing down myself.... I was going to purge based on date. As fast as linux grows, I wouldn't think something from a year ago would be the same. B-) On Monday 31 January 2005 01:00 pm, Patrick Shanahan wrote:
* Hans du Plooy <hansdp@newingtoncs.co.za> [01-31-05 14:54]:
I've been following this mailing list since SUSE 8.0 came out, and have kept every single mail. I know that might sound excessive, but having my own copy of at least a piece of "the archives" has helped more than a couple of times when connectivity isn't available.
Currently I have all my mail stashed in ~/Maildir - courier-imap gives me access to it. Nice setup as I can chop&change between mail clients any time I like.
However, the sheer volume of mail is becoming a problem. It's getting slower and slower, searches through mail folders take for ever and max out my CPU for minutes at a time.
I was wondering if there's a better way to store my mail?
I can and do part of the time, handle my own mail, but have taken to directing *all* mailing list to my gmail address. I then fetchmail to local for reading, but label and retain copies of all mail passing thru gamil.
Searching for a particular email/subject/thread is much faster on gmail's site than on my own computer (aged 733) and a gig of space is available. If the space is not enough, get more gmail accounts and disburse the delivery of mail lists thru the different accounts.
A plus, gmail's spam filtering is better than spamassassin on my local computer.
Just a thought, -- Patrick Shanahan Registered Linux User #207535 http://wahoo.no-ip.org @ http://counter.li.org HOG # US1244711 Photo Album: http://wahoo.no-ip.org/photos
Top posted and full quoted. Extra quoting removed. * Brad Bourn <brad@summitrd.com> [01-31-05 15:10]:
as the OP mentioned, it is important for them to have when not connected.
I see that on second read. Just ignore my post, thankyou. -- Patrick Shanahan Registered Linux User #207535 http://wahoo.no-ip.org @ http://counter.li.org HOG # US1244711 Photo Album: http://wahoo.no-ip.org/photos
as the OP mentioned, it is important for them to have when not connected.
I would think you could simply create sub-folders for different months/years, or database them with mysql or something..... He he, I've already done that. It becomes a drag to search through them
On Monday 31 January 2005 22:07, Brad Bourn wrote: though, especially if the mail client doesn't support searching through subfolders. The mysql thing might be an idea. Do you know of any apps/web-frontends that will do this? Thanks -- Kind regards Hans du Plooy Newington Consulting Services hansdp at newingtoncs dot co dot za
I don't. A couple things come to mind though, To get spamassasin to work, I had to use a program called safecat. It's been a while, but it takes stdin (i think) and converts it to maildir format. So, if you stored EVERTHING as a large text blob or something searchable with SQL, you could then write a script / app that would take a search parameter, get all messages that meet the search, and 'export' them out of sql into a maildir folder (using safecat) to then be searched by your mail client. If you run php/mysql, this shouldn't be too hard. B-) On Monday 31 January 2005 02:00 pm, Hans du Plooy wrote:
On Monday 31 January 2005 22:07, Brad Bourn wrote:
as the OP mentioned, it is important for them to have when not connected.
I would think you could simply create sub-folders for different months/years, or database them with mysql or something.....
He he, I've already done that. It becomes a drag to search through them though, especially if the mail client doesn't support searching through subfolders.
The mysql thing might be an idea. Do you know of any apps/web-frontends that will do this?
Thanks -- Kind regards Hans du Plooy Newington Consulting Services hansdp at newingtoncs dot co dot za
On Monday 31 January 2005 23:00, Hans du Plooy wrote:
The mysql thing might be an idea. Do you know of any apps/web-frontends that will do this? Ooh, bad idea: http://archives.neohapsis.com/archives/postfix/2003-02/1613.html
It seems I already have the most efficient solution - Maildir. The only way to improve it, from what I've read so far, is to move the Maildir over to a XFS filesystem. Ahh well... -- Kind regards Hans du Plooy Newington Consulting Services hansdp at newingtoncs dot co dot za
NO, NO, NO! This article talks about using Access as a back-end. This is NOT true. Access IS NOT A BACKEND, OR EVEN A DATABASE. Access is only a FRONT END application. The database is JET / SQL. Access uses JET by default, but can use SQL. JET is very slow for network lookups. It was only designed for single-station use. Besides, we already know anything M$ does is going to be bloated. hehehe B-) On Monday 31 January 2005 02:09 pm, Hans du Plooy wrote:
On Monday 31 January 2005 23:00, Hans du Plooy wrote:
The mysql thing might be an idea. Do you know of any apps/web-frontends that will do this?
Ooh, bad idea: http://archives.neohapsis.com/archives/postfix/2003-02/1613.html
It seems I already have the most efficient solution - Maildir. The only way to improve it, from what I've read so far, is to move the Maildir over to a XFS filesystem.
Ahh well...
-- Kind regards Hans du Plooy Newington Consulting Services hansdp at newingtoncs dot co dot za
The obvious way to handle mail archiving would be if the mailer program had the ability to open a mail file in an arbitrary location -- including, say, a CD (in read-only mode, of course). In fact, little or nothing would be lost if archives were always made available only in read-only mode. Unfortunately Kmail doesn't provide for that, though it's been requested. I don't know about other Linux mailers. (I don't think Windows mailers do any better.) I don't know why Kmail doesn't have this obvious enhancement. Maybe it's possible to respecify the location of the mail folder (though I haven't unearthed that setting), but even if it is, that's surely the long way 'round the barn. Paul
* Paul W. Abrahams <abrahams@acm.org> [01-31-05 17:06]:
The obvious way to handle mail archiving would be if the mailer program had the ability to open a mail file in an arbitrary location -- including, say, a CD (in read-only mode, of course). In fact, little or nothing would be lost if archives were always made available only in read-only mode.
mutt, THE linux email client *can* *do* *this*, but the read-only mode from the mail client is unnecessary as you cannot write via the mail client direct to a cd anyway.
Unfortunately Kmail doesn't provide for that, though it's been requested. I don't know about other Linux mailers.
mutt, and perhaps pine
I don't know why Kmail doesn't have this obvious enhancement.
??
Maybe it's possible to respecify the location of the mail folder (though I haven't unearthed that setting), but even if it is, that's surely the long way 'round the barn.
?? -- Patrick Shanahan Registered Linux User #207535 http://wahoo.no-ip.org @ http://counter.li.org HOG # US1244711 Photo Album: http://wahoo.no-ip.org/photos
On Monday 31 January 2005 5:24 pm, Patrick Shanahan wrote:
[I recommend] mutt, and perhaps pine
Can anyone point me to a comparison of Kmail, mutt, and pine? How easy is it to switch from Kmail to mutt (and back again, if I don't like the results)? Paul
* Paul W. Abrahams <abrahams@acm.org> [01-31-05 17:53]:
On Monday 31 January 2005 5:24 pm, Patrick Shanahan wrote:
[I recommend] mutt, and perhaps pine
^^^^^^^^^^^^^^ No, this is NOT what I said. I USE mutt and like it. I didn't tell you to use it. I said that it was capable of reading mail files from a cd.
Can anyone point me to a comparison of Kmail, mutt, and pine? How easy is it to switch from Kmail to mutt (and back again, if I don't like the results)?
mutt can read kmail's files, just tell it to. I haven't used pine in 10 years. I liked elm better and elm -> mutt. The *only* way to find out if you like one better than the other is to *use* them and make your own comparison. -- Patrick Shanahan Registered Linux User #207535 http://wahoo.no-ip.org @ http://counter.li.org HOG # US1244711 Photo Album: http://wahoo.no-ip.org/photos
On Monday 31 January 2005 5:59 pm, Patrick Shanahan wrote:
mutt can read kmail's files, just tell it to.
Is the reverse also true? That would be essential for me or anyone else trying out mutt -- I wouldn't want to be stuck with it and unable to convert back if I don't like the results. Kmail doesn't have an obvious import filter for mutt. But then again, it doesn't have an import filter for any other Linux mailer either as far as I can tell. Paul
* Paul W. Abrahams <abrahams@acm.org> [01-31-05 18:40]:
Is the reverse also true? That would be essential for me or anyone else trying out mutt -- I wouldn't want to be stuck with it and unable to convert back if I don't like the results. Kmail doesn't have an obvious import filter for mutt. But then again, it doesn't have an import filter for any other Linux mailer either as far as I can tell.
yes, mutt *can* read/share kamil's files as long as you set it up to do so. Mutt can co-exist and share kmail's files as long as you set both to use the same mail storage format. Mutt can transfer files from one storage format to another and back, but no necessary if you use the same format with both clients. -- Patrick Shanahan Registered Linux User #207535 http://wahoo.no-ip.org @ http://counter.li.org HOG # US1244711 Photo Album: http://wahoo.no-ip.org/photos
The Monday 2005-01-31 at 17:24 -0500, Patrick Shanahan wrote:
Unfortunately Kmail doesn't provide for that, though it's been requested. I don't know about other Linux mailers.
mutt, and perhaps pine
No, Pine can not read email from a CD, it complains that it can not write to it - apparently it wants to change the "read" status, "answered", and such. Unless there is a trick I haven't discovered.
I don't know why Kmail doesn't have this obvious enhancement.
??
Mmm. Perhaps. You (well, the OP) could try separating the mail folders (mbox format) on a directory on the CD, and the indexes files locally on the HD. I can think this out another day, I'm interested myself. -- Cheers, Carlos Robinson
On Tuesday 01 February 2005 00:03, Paul W. Abrahams wrote:
Maybe it's possible to respecify the location of the mail folder (though I haven't unearthed that setting), but even if it is, that's surely the long way 'round the barn.
ln -s /extra/disc ~/mail by any chance? -- Kind regards Hans du Plooy Newington Consulting Services hansdp at newingtoncs dot co dot za
Hans, On Tuesday 01 February 2005 00:06, Hans du Plooy wrote:
On Tuesday 01 February 2005 00:03, Paul W. Abrahams wrote:
Maybe it's possible to respecify the location of the mail folder (though I haven't unearthed that setting), but even if it is, that's surely the long way 'round the barn.
ln -s /extra/disc ~/mail by any chance?
If someone decides to try that or some variation on or elaboration of it, the symlink should be placed in ~/Mail, not ~/mail.
Hans du Plooy
Randall Schulz
On Tuesday 01 February 2005 11:13 am, Randall R Schulz wrote:
Hans,
On Tuesday 01 February 2005 00:06, Hans du Plooy wrote:
On Tuesday 01 February 2005 00:03, Paul W. Abrahams wrote:
Maybe it's possible to respecify the location of the mail folder (though I haven't unearthed that setting), but even if it is, that's surely the long way 'round the barn.
ln -s /extra/disc ~/Mail by any chance?
If I were to do symlinking at all, it would probably be from a subfolder. But I'm still not satisfied with that solution, for several reasons. For instance, would Kmail get a bad case of indigestion if the symlink is to a read-only directory such as a CD? More important, I'm looking for a way to *temporarily* connect to an archive folder and choose that folder dynamically. Sure, I could create a symlink and later remove it, but as I said earlier, that's the long way 'round the barn. It requires going outside of the mailer to do the necessary twiddles. And it offends my sense of programming aesthetics. Paul
Paul, On Tuesday 01 February 2005 10:33, Paul W. Abrahams wrote:
On Tuesday 01 February 2005 11:13 am, Randall R Schulz wrote:
Hans,
On Tuesday 01 February 2005 00:06, Hans du Plooy wrote:
On Tuesday 01 February 2005 00:03, Paul W. Abrahams wrote:
Maybe it's possible to respecify the location of the mail folder (though I haven't unearthed that setting), but even if it is, that's surely the long way 'round the barn.
ln -s /extra/disc ~/Mail by any chance?
I didn't write that. I only wrote to correct the name of "~/Mail" (in the post you're actually quoting, it was spelled "~/mail").
If I were to do symlinking at all, it would probably be from a subfolder. But I'm still not satisfied with that solution, for several reasons. For instance, would Kmail get a bad case of indigestion if the symlink is to a read-only directory such as a CD? More important, I'm looking for a way to *temporarily* connect to an archive folder and choose that folder dynamically. Sure, I could create a symlink and later remove it, but as I said earlier, that's the long way 'round the barn. It requires going outside of the mailer to do the necessary twiddles. And it offends my sense of programming aesthetics.
Well, depending on what you usage patterns and aesthetic criteria, you do not necessarily need to go outside KMail, if you're willing to be a little creative. KMail filters can invoke external filters, which are a hook to executing arbitrary programs. KMail filters can be selectively placed into the Message -> Apply Filter menu and / or assigned to toolbar buttons. So if the symlink trick itself works, you can probably come up with a way to get at it from within KMail.
Paul
Randall Schulz
The Tuesday 2005-02-01 at 13:33 -0500, Paul W. Abrahams wrote:
If I were to do symlinking at all, it would probably be from a subfolder. But I'm still not satisfied with that solution, for several reasons. For instance, would Kmail get a bad case of indigestion if the symlink is to a read-only directory such as a CD?
I would like to try the following. In a new subfolder, create symlinks to mailfolders in the CD, but not to the index files, which will reside in the new subfolder in the HD. I know I'm fuzzy, but perhaps you get the idea? I haven't tried it myself, but I intend to, sometime.
More important, I'm looking for a way to *temporarily* connect to an archive folder and choose that folder dynamically. Sure, I could create a symlink and later remove it, but as I
You probably do not need to remove the symlink. -- Cheers, Carlos Robinson
I can and do part of the time, handle my own mail, but have taken to directing *all* mailing list to my gmail address. I then fetchmail to local for reading, but label and retain copies of all mail passing thru gamil.
if you need to get an gmail account i have a few invites from two different account to give away so email me if you want one. jack
On Tue, 2005-02-01 at 01:41, Jack Malone wrote:
I can and do part of the time, handle my own mail, but have taken to directing *all* mailing list to my gmail address. I then fetchmail to local for reading, but label and retain copies of all mail passing thru gamil.
if you need to get an gmail account i have a few invites from two different account to give away so email me if you want one.
jack
Dear Jack, I was following this thread and I would like a gmail invite if you have one. I have not posted to this thread, but I also store all of my suse list messages and face the same problem. I switched from kmail to evolution as a solution to the problem, and it helped but evolution's search leaves a lot to be desired. My wife suggested I use gmail for all list's mail and on exploring this I think she is right. Patricks's suggestion reinforced the idea. Can you spare an ivitation? Thanks, Gustav Degreef.
On Mon, 31 Jan 2005, Hans du Plooy wrote:
Hi guys,
I've been following this mailing list since SUSE 8.0 came out, and have kept every single mail. I know that might sound excessive, but having my own copy of at least a piece of "the archives" has helped more than a couple of times when connectivity isn't available.
Currently I have all my mail stashed in ~/Maildir - courier-imap gives me access to it. Nice setup as I can chop&change between mail clients any time I like.
However, the sheer volume of mail is becoming a problem. It's getting slower and slower, searches through mail folders take for ever and max out my CPU for minutes at a time.
I was wondering if there's a better way to store my mail?
OK. For 95% of people's purposes, courier-imap ROCKS. It's fast[1], stable, fairly well supported, and "It Just Works". However, my note, [1] is that it's fast until you get large mailboxes. Sam Varshavchik (sp?) is the author of courier-imap, and he feels it's the filesystem's problem if your very large mailbox is slow. courier-imap does NO CACHING whatsoever. So, so scan a mailbox with 10,000 entries for anything (any kind of searching or sorting at all), every single email must be opened and read at least partially. Enter dovecot. "Mostly" compatible with courier-imap, it uses little databases of information "about" individual emails (aka, meta data), like subject, date, author, etc... Give dovecot a try. THere is a wiki entry for doing just that on the dovecot wiki. Either that, or a) throw hardware at it b) split the mailbox up -- Carpe diem - Seize the day. Carp in denim - There's a fish in my pants! Jon Nelson <jnelson-suse@jamponi.net>
Hans du Plooy wrote:
Hi guys, [snipped] However, the sheer volume of mail is becoming a problem. It's getting slower and slower, searches through mail folders take for ever and max out my CPU for minutes at a time.
I was wondering if there's a better way to store my mail?
Would hypermail work for you? I haven't tried it but a recent Linux mag had an article on hypermail and it sounded interesting.
Thanks
No probs -- GPG fingerprint = 3D45 5509 D380 26A4 523E A9D8 A66A 5F38 CA43 BB0E
participants (10)
-
Brad Bourn -
Carlos E. R. -
Hans du Plooy -
Jack Malone -
jalal -
Jon Nelson -
Patrick Shanahan -
Paul W. Abrahams -
rada and gus -
Randall R Schulz