[opensuse] Does this error look familiar (dovecot)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I get this error in the warn log: <2.3> 2017-04-14 02:44:30 Telcontar dovecot - - - lda(cer): Error: write() failed with mbox file /home/cer/Mail/_Lists/y_gml_os-unifd: File too large (process was started with ulimit -f limit) (file size is currently 225122K) I think it is related to this criptic error in procmail.log procmail: Program failure (75) of "/usr/lib/dovecot/dovecot-lda" The rule in procmail is like this: DELIVER = "/usr/lib/dovecot/dovecot-lda" FORMAIL = "/usr/bin/formail" ... :0f * ^X-Mailinglist: opensuse-es | $FORMAIL -bfi 'Reply-To: "oS-es" <opensuse-es@opensuse.org>' :0 aw: $HOME/Mail/.D-locks/_Lists/y_gml_os-unifd.lock | $DELIVER -m _Lists/y_gml_os-unifd Now the question is: Where is that "ulimit -f" set up? In dovecot code? In procmail code? In some config of either one? - -- Cheers Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAljwLB8ACgkQtTMYHG2NR9UYEgCfR5T8xOcAiTrPTH6aAercUf8F 3wAAn1ojgAJCaCC4kNr0Oat4N8fb42sG =Avs2 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 04/13/2017 06:55 PM, Carlos E. R. wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I get this error in the warn log:
<2.3> 2017-04-14 02:44:30 Telcontar dovecot - - - lda(cer): Error: write() failed with mbox file /home/cer/Mail/_Lists/y_gml_os-unifd: File too large (process was started with ulimit -f limit)
(file size is currently 225122K)
I think it is related to this criptic error in procmail.log
procmail: Program failure (75) of "/usr/lib/dovecot/dovecot-lda"
The rule in procmail is like this:
DELIVER = "/usr/lib/dovecot/dovecot-lda" FORMAIL = "/usr/bin/formail"
...
:0f * ^X-Mailinglist: opensuse-es | $FORMAIL -bfi 'Reply-To: "oS-es" <opensuse-es@opensuse.org>' :0 aw: $HOME/Mail/.D-locks/_Lists/y_gml_os-unifd.lock | $DELIVER -m _Lists/y_gml_os-unifd
Now the question is: Where is that "ulimit -f" set up?
In dovecot code? In procmail code? In some config of either one?
- -- Cheers
Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2
iEYEARECAAYFAljwLB8ACgkQtTMYHG2NR9UYEgCfR5T8xOcAiTrPTH6aAercUf8F 3wAAn1ojgAJCaCC4kNr0Oat4N8fb42sG =Avs2 -----END PGP SIGNATURE-----
Carlos, have a look at /etc/security/limits.conf -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2017-04-14 05:34, Bruce Ferrell wrote:
Carlos,
have a look at /etc/security/limits.conf
Hardly would limit at 250 MB. Everything commented out, except: Telcontar:~ # cat /etc/security/limits.conf | egrep -v "^[[:space:]]*$|^#" * hard nproc 1700 * soft nproc 1200 root hard nproc 3000 root soft nproc 1850 Telcontar:~ # -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)
Carlos E. R. wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I get this error in the warn log:
<2.3> 2017-04-14 02:44:30 Telcontar dovecot - - - lda(cer): Error: write() failed with mbox file /home/cer/Mail/_Lists/y_gml_os-unifd: File too large (process was started with ulimit -f limit)
(file size is currently 225122K)
What is your mailbox size limit ? -- Per Jessen, Zürich (10.4°C) http://www.cloudsuisse.com/ - your owncloud, hosted in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2017-04-14 09:06, Per Jessen wrote:
Carlos E. R. wrote:
I get this error in the warn log:
<2.3> 2017-04-14 02:44:30 Telcontar dovecot - - - lda(cer): Error: write() failed with mbox file /home/cer/Mail/_Lists/y_gml_os-unifd: File too large (process was started with ulimit -f limit)
(file size is currently 225122K)
What is your mailbox size limit ?
I wasn't aware of having one. :-} :-?
https://wiki.dovecot.org/LDA/Postfix
Postfix's mailbox_size_limit setting applies to all files that are written via dovecot-lda. The default is 50 MB, so dovecot-lda can't write any files larger than that, including mbox files or log files.
(that's why I asked about your mailbox_size_limit).
cer@Telcontar:~> grep mailbox_size_limit /etc/postfix/main.cf #mailbox_size_limit = 51200000 # migth cause procmail error. http://www.xray.mpe.mpg.de/mailing-lists/procmail/2006-10/msg00042.html mailbox_size_limit = 200000000 cer@Telcontar:~> That seems close. And it seems I hit that problem previously. Doubling the value. Funny. I remember the complains with Exchange having a 2 GiB limit per mailbox, and here we go with a 50 MB default limit. And it has to load the entire mbox in RAM, to just append a new mail at the end? -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Friday, 2017-04-14 at 12:21 +0200, Carlos E. R. wrote:
On 2017-04-14 09:06, Per Jessen wrote:
What is your mailbox size limit ?
I wasn't aware of having one. :-} :-?
Well, that was it. Verified. Thanks! :-) - -- Cheers, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAljwpQMACgkQtTMYHG2NR9WyfwCbB9ZtPi7v7fuWgHSmUmnmy2TL 5eAAmwbRhzN8dD3e2skwhXJPfqHgiRHd =UXZp -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Carlos E. R. wrote:
Funny. I remember the complains with Exchange having a 2 GiB limit per mailbox, and here we go with a 50 MB default limit.
It's probably just an old default Postfix setting. My guess is it's not used much anymore - people set it to 0, and then leave the mailbox/-dir to be governed by quotas in dovecot.
And it has to load the entire mbox in RAM, to just append a new mail at the end?
I wouldn't think so, but we use maildirs. -- Per Jessen, Zürich (14.7°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2017-04-14 14:41, Per Jessen wrote:
Carlos E. R. wrote:
Funny. I remember the complains with Exchange having a 2 GiB limit per mailbox, and here we go with a 50 MB default limit.
It's probably just an old default Postfix setting. My guess is it's not used much anymore - people set it to 0, and then leave the mailbox/-dir to be governed by quotas in dovecot.
And it has to load the entire mbox in RAM, to just append a new mail at the end?
I wouldn't think so, but we use maildirs.
Ah, I misinterpreted. ULIMIT(1P) −f Set (or report, if no blocks operand is present), the file size limit in blocks. The −f option shall also be the default case. -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)
Per Jessen wrote:
Carlos E. R. wrote:
Funny. I remember the complains with Exchange having a 2 GiB limit per mailbox, and here we go with a 50 MB default limit.
It's probably just an old default Postfix setting. My guess is it's not used much anymore - people set it to 0, and then leave the mailbox/-dir to be governed by quotas in dovecot.
Is this an old postfix installation, Carlos? Recent ones have default mailbox_size_limit = 0. -- Per Jessen, Zürich (14.7°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2017-04-14 15:35, Per Jessen wrote:
Per Jessen wrote:
Carlos E. R. wrote:
Funny. I remember the complains with Exchange having a 2 GiB limit per mailbox, and here we go with a 50 MB default limit.
It's probably just an old default Postfix setting. My guess is it's not used much anymore - people set it to 0, and then leave the mailbox/-dir to be governed by quotas in dovecot.
Is this an old postfix installation, Carlos? Recent ones have default mailbox_size_limit = 0.
Yes, it is. This system was first installed with SuSE 6.2, then upgraded through the years (software and hardware). I will then set it to zero as well. -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)
On 14/04/17 08:41 AM, Per Jessen wrote:
Carlos E. R. wrote:
Funny. I remember the complains with Exchange having a 2 GiB limit per mailbox, and here we go with a 50 MB default limit.
It's probably just an old default Postfix setting. My guess is it's not used much anymore - people set it to 0, and then leave the mailbox/-dir to be governed by quotas in dovecot.
And it has to load the entire mbox in RAM, to just append a new mail at the end?
I wouldn't think so, but we use maildirs.
And that's why I hate the ext[234] family of file systems. You either run out of file blocks while there's still space devoted to i-nodes or you run out of i-nodes while there's still space available. The whole idea of pre-provisioing i-nodes, the idea of devoting a fixed amount space to i-nodes and a fixed amut of space to data blocks is right out of the of the original UNIX of the 1970s. The real B-tree file systems, notably ReiserFS, XFS, BtrFS, don't have this limitation. They allocate space as needed. The irony here is that ext4 is a brilliant, otherwise superbly designed and crafted file system, really superb. Really drool-worthy in many aspects. But it has this idiotic 1970s approach to pre-provisioning, setting the fixed amounts of space at MKFS time. Yes, it's more flexible about that than its extFS predecessors, but whenever I've used it for email or enews I've been bitten by this. Maybe Richard is smart enough to work out the pre-provisioning so that doesn't happen to him. Yes, he's smarter than me about FS matters, but I really don't want to have to spend time with that sort of calculation. Its things like that make me realise that perhaps BtrFS is a good choice. Just keep purging the backups you don't want :-) -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Anton Aylward wrote:
On 14/04/17 08:41 AM, Per Jessen wrote:
Carlos E. R. wrote:
Funny. I remember the complains with Exchange having a 2 GiB limit per mailbox, and here we go with a 50 MB default limit.
It's probably just an old default Postfix setting. My guess is it's not used much anymore - people set it to 0, and then leave the mailbox/-dir to be governed by quotas in dovecot.
And it has to load the entire mbox in RAM, to just append a new mail at the end?
I wouldn't think so, but we use maildirs.
And that's why I hate the ext[234] family of file systems.
Not sure I get the connection with Carlo's mailbox sizes?
You either run out of file blocks while there's still space devoted to i-nodes or you run out of i-nodes while there's still space available.
The whole idea of pre-provisioing i-nodes, the idea of devoting a fixed amount space to i-nodes and a fixed amut of space to data blocks is right out of the of the original UNIX of the 1970s.
The real B-tree file systems, notably ReiserFS, XFS, BtrFS, don't have this limitation. They allocate space as needed.
We've been using JFS for 14-15 years, since around 2003 I think. Not that I have any issue with ext4, I use for root file systems too occasionally. -- Per Jessen, Zürich (14.8°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 14/04/17 10:31 AM, Per Jessen wrote:
We've been using JFS for 14-15 years, since around 2003 I think. Not that I have any issue with ext4, I use for root file systems too occasionally.
Like the song about the English country garden, "I'll tell you now of some that I know / Those I miss you'll surely pardon". I too use ext4 for my RootFS for a simple reason. It can be reasonably accurately pre-provisioned, or at least that is the case for me where the indeterminate are under /home and /tmp. And its not as if there is going to be much churn on the RootFS except when I run 'zypper' to update. The !BIG! files are under ~/MyMovies and ~/MyMusic. Oh, and log files! Given the outstanding size of video and audio files, I've tried creating specialized, carefully pre-provisioned ext4FS for those. Lets face it, there isn't going to be much churn; once a music or video file is downloaded that's it! The files themselves don't grow. The number of them do, perhaps. The directory arrangements might change, but the 'mv' command doesn't copy data if it can just move a directory entry. Now ~/MyPhotographs is quite another matter. I use Darktable which makes a point of never altering the 'original'. So I upload the RAW or JPG files files from my camera, and those are BIG. Even my little 16M pocket sized P&S start at 25M. Compare that to 750M to 1300M for a movie and think that I deal with between 20 and 200 photos a month vis perhaps one or two or month. Darktable is smart, it uses 'sidebar' files to record the edit, but the moment I export a high-res JPG there goes another few meg, usually just around 3M but perhaps as many as 10M or larger (don't ask, it happens sometimes). It adds up, but more to the point its the contrast between all those multi-meg files and the equally large number of sidebar files that are only a few K in size. I'm sure this would not strain ext4, properly provisioned, but its simpler to use ReiserFS or XFS in these situations. **************** <========= What I would like is a (preferable text-mode) file system scanner that tells me the distribution of files, parameters allowing the 'resolution' and granularity. The reality is that past years of the photographs are _almost_ archival, they don't need to flexibility of XFS or ReiserFS. I _could_ manage with a fixed provisioning with a small amount of slack. **************** <========= In a more perfect world if static storage devices like DVD were really, really fast, we could make use of the "Overlay File System". The superfast-DVD being the base, the slower 'rotating rust' running the overlayFS with updates. Then, periodically, burn a new baseline superfast-DVD. It would work wonders for the ~/MyMusic, ~/MyMovies and ~/MyPhotographs. I'm sure for other people other baselines would apply, web services, documents, man pages. It just has to be cheap and fast. Well, dream on Anton. Don't you recall the day when you were glad to 100M drive for $100? Oh wow! A dollar a megabyte! Now I can pick up a 3T drive for that, can that's 100 Kanukistani pesos on the high street. Track density (aka reduced head movement, read rates) mean that the OS you were using in those 100M drive days will fit on one revolution of the disk, so what are you complaining about? I'll make a note to look into using JFS when I set up my NFS server again. -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2017-04-14 16:31, Per Jessen wrote:
Anton Aylward wrote:
On 14/04/17 08:41 AM, Per Jessen wrote:
Carlos E. R. wrote:
Funny. I remember the complains with Exchange having a 2 GiB limit per mailbox, and here we go with a 50 MB default limit.
It's probably just an old default Postfix setting. My guess is it's not used much anymore - people set it to 0, and then leave the mailbox/-dir to be governed by quotas in dovecot.
And it has to load the entire mbox in RAM, to just append a new mail at the end?
I wouldn't think so, but we use maildirs.
And that's why I hate the ext[234] family of file systems.
Not sure I get the connection with Carlo's mailbox sizes?
No, the connection is with maildir. The only filesystem I know that is really happy in that usage is reiserfs. There are other mailbox formats, though, that stress less the filesystem, hybrids between maildir and mbox: storing several mails on the same file, but up to a file size limit, perhaps 1 MB, when it switches to a new file. -- Cheers / Saludos, Carlos E. R. (from 42.2 x86_64 "Malachite" at Telcontar)
Carlos E. R. wrote:
I get this error in the warn log:
<2.3> 2017-04-14 02:44:30 Telcontar dovecot - - - lda(cer): Error: write() failed with mbox file /home/cer/Mail/_Lists/y_gml_os-unifd: File too large (process was started with ulimit -f limit)
https://wiki.dovecot.org/LDA/Postfix Postfix's mailbox_size_limit setting applies to all files that are written via dovecot-lda. The default is 50 MB, so dovecot-lda can't write any files larger than that, including mbox files or log files. (that's why I asked about your mailbox_size_limit). -- Per Jessen, Zürich (10.6°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (4)
-
Anton Aylward -
Bruce Ferrell -
Carlos E. R. -
Per Jessen