[opensuse] maildirs and filesystems
Carlos E. R. wrote:
On 2017-04-14 16:31, Per Jessen wrote:
Anton Aylward wrote:
On 14/04/17 08:41 AM, Per Jessen wrote:
Carlos E. R. wrote:
Funny. I remember the complains with Exchange having a 2 GiB limit per mailbox, and here we go with a 50 MB default limit.
It's probably just an old default Postfix setting. My guess is it's not used much anymore - people set it to 0, and then leave the mailbox/-dir to be governed by quotas in dovecot.
And it has to load the entire mbox in RAM, to just append a new mail at the end?
I wouldn't think so, but we use maildirs.
And that's why I hate the ext[234] family of file systems.
Not sure I get the connection with Carlo's mailbox sizes?
No, the connection is with maildir. The only filesystem I know that is really happy in that usage is reiserfs.
Like I said, JFS works very well too. I would also expect XFS to do a good job, I just don't have any reason to experiment. For hosted email, I think we have some 500Gb for mail storage, each user has a quota of up to 5Gb. We use maildirs on JFS in other contexts too - email quarantine and archiving, both with a lot more storage. The only issue I can recall was about a year ago, when we had to do a complete fsck on the quarantine on one system, that took a loooooong time.
There are other mailbox formats, though, that stress less the filesystem, hybrids between maildir and mbox: storing several mails on the same file, but up to a file size limit, perhaps 1 MB, when it switches to a new file.
Does dovecot and postfix work with those? Might be interesting to investigate. I also maintain a large database of various graphics. Some 5-6million files, stored as individual files in multiple directories, but quite a lot per directory. I thought it would be interesting to load them into mysql instead (as blobs), but that totally failed. It basically slowed down everything on that database machine :-) -- Per Jessen, Zürich (12.9°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 15/04/2017 à 09:40, Per Jessen a écrit :
email, I think we have some 500Gb for mail storage, each user has a quota of up to 5Gb.
not much. One of my daughter uses already more than 10Go (pro account) :-). But now space is nearly unlimited
quarantine on one system, that took a loooooong time.
I have old qmail archives, simply *copying* then when I need to change server is *very* long
There are other mailbox formats, though, that stress less the filesystem, hybrids between maildir and mbox: storing several mails on the same file, but up to a file size limit, perhaps 1 MB, when it switches to a new file.
could be interesting also think that the "sent" folder is often the one nobody think to clean. Having it split by year, for example could be interesting jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
jdd wrote:
Le 15/04/2017 à 09:40, Per Jessen a écrit :
email, I think we have some 500Gb for mail storage, each user has a quota of up to 5Gb.
not much. One of my daughter uses already more than 10Go (pro account) :-). But now space is nearly unlimited
The 5Gb is more than plenty for all but one user, but this is in a business context, not really many huge emails. Space is virtually unlimited, but what else do you use to differentiate the price? -- Per Jessen, Zürich (13.4°C) http://www.cloudsuisse.com/ - your owncloud, hosted in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 15/04/2017 à 10:13, Per Jessen a écrit :
unlimited, but what else do you use to differentiate the price?
you don't anymore http://www.o2switch.fr/ and for the price (not even advertised) a full mail system only thing not done is cloud, but then I have cloud subscription (but not unlimited) https://owndrive.com/ I discovered this recently and I'm in the way to move all my work to such hosts, so I won't any more have to deal with servers (I'm 71, now and have to lower my computer involvement) and most if not all of this is on ssd... hard time for traditional host providers :-( all these hosts uses linux, but probably debian ones (for what I can guess, systems are of course concealed) jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 15/04/2017 à 09:40, Per Jessen a écrit : also think that the "sent" folder is often the one nobody think to clean. Having it split by year, for example could be interesting thunderbird seems to make this for archives right now (since some time) jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 15/04/17 09:06, jdd wrote:
Le 15/04/2017 à 09:40, Per Jessen a écrit :
also think that the "sent" folder is often the one nobody think to clean. Having it split by year, for example could be interesting
thunderbird seems to make this for archives right now (since some time)
Thunderbird does have an archive system, true, but iirc it is a manual thing. You have to do it yourself. Myself, I've got TB configured to use the inbox as the sent folder, so all my mail is threaded together. Oh - and as for usage - how many people get oodles of spam? And DON'T clean out their email? (That said, if your normal UI is a mobile phone, cleaning out an inbox isn't easy I don't hink...) Whenever I look at my daughter's inbox, it's maybe 5% HAM, and the junk goes back to last decade! Cheers, Wol -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 15/04/2017 à 11:01, Wols Lists a écrit :
Thunderbird does have an archive system, true, but iirc it is a manual thing. You have to do it yourself.
not really, there is an advanced menu in copies and folder lessage archive where one can setup yearly or monthly subfolders (just noticed this today :-) jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2017-04-15 09:40, Per Jessen wrote:
Carlos E. R. wrote:
No, the connection is with maildir. The only filesystem I know that is really happy in that usage is reiserfs.
Like I said, JFS works very well too. I would also expect XFS to do a good job, I just don't have any reason to experiment. For hosted email, I think we have some 500Gb for mail storage, each user has a quota of up to 5Gb.
XFS is the second best, of those provided by openSUSE. It creates the equivalent of inodes dynamically. However, reiserfs can cope with a really huge number of small files at full speed; XFS eventually gets full (but much later than ext4). I did a comparison of some years ago of all the available filesystems with millions of small files. Btrfs crashed, but I think that was solved. However, it became very slow as the number of files grew. JFS I haven't tested.
We use maildirs on JFS in other contexts too - email quarantine and archiving, both with a lot more storage. The only issue I can recall was about a year ago, when we had to do a complete fsck on the quarantine on one system, that took a loooooong time.
There are other mailbox formats, though, that stress less the filesystem, hybrids between maildir and mbox: storing several mails on the same file, but up to a file size limit, perhaps 1 MB, when it switches to a new file.
Does dovecot and postfix work with those? Might be interesting to investigate.
The problem is that those new formats are not standard, and only supported by one application, in this case, dovecot. You have to do the delivery calling the dovecot tool for that. I wanted to try them, but the idea of support by only one tool deters me. The documentation for dovecot talks about those formats available. Alpine also supports its own idea. I haven't tried. Its original late developer was creating an improvement to his uw imap implementation with such a format, but that is now stalled, obviously.
I also maintain a large database of various graphics. Some 5-6million files, stored as individual files in multiple directories, but quite a lot per directory. I thought it would be interesting to load them into mysql instead (as blobs), but that totally failed. It basically slowed down everything on that database machine :-)
The reiserfs project wanted to create databases using the filesystem as engine, instead of creating their own structure. Version 4 was going that road. Thus, each record could be a directory, and each field would be a file. -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)
Carlos E. R. wrote:
We use maildirs on JFS in other contexts too - email quarantine and archiving, both with a lot more storage. The only issue I can recall was about a year ago, when we had to do a complete fsck on the quarantine on one system, that took a loooooong time.
There are other mailbox formats, though, that stress less the filesystem, hybrids between maildir and mbox: storing several mails on the same file, but up to a file size limit, perhaps 1 MB, when it switches to a new file.
Does dovecot and postfix work with those? Might be interesting to investigate.
The problem is that those new formats are not standard, and only supported by one application, in this case, dovecot. You have to do the delivery calling the dovecot tool for that.
Ah, I know - dbox. We deliver mails via dove-lmtp anyway, but I don't know if dbox was ever interesting to us. -- Per Jessen, Zürich (14.2°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (4)
-
Carlos E. R. -
jdd -
Per Jessen -
Wols Lists