[opensuse] procmail, dovecot-lda, and lockfiles: problems.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I had this type of rule in .procmailrc: :0f * ^X-Mailinglist: opensuse | /usr/bin/formail -bfi 'Reply-To: "OS-en" <opensuse@opensuse.org>' :0 a: $HOME/Mail/_Lists/os-en which, following talks here, I change to this type of rule: DELIVER="/usr/lib/dovecot/dovecot-lda" ... :0f * ^X-Mailinglist: opensuse | /usr/bin/formail -bfi 'Reply-To: "OS-en" <opensuse@opensuse.org>' :0 aw: | $DELIVER -m _Lists/D-os-en which produces these entries in the procmail log: procmail: Couldn't determine implicit lockfile from "/usr/lib/dovecot/dovecot-lda" procmail: Locking ".lock" procmail: Executing "/usr/lib/dovecot/dovecot-lda,-m,lists/D-os-en" procmail: Assigning "LASTFOLDER=/usr/lib/dovecot/dovecot-lda -m lists/D-os-en" procmail: Unlocking ".lock" procmail: Notified comsat: "cer@:/usr/lib/dovecot/dovecot-lda -m lists/D-os-en" The reason is explained in this section of the manual: Local lockfile If you put a second (trailing) ':' on the first recipe line, then procmail will use a locallockfile (for this recipe only). You can optionally specify the locallockfile to use; if you don't however, procmail will use the destination filename (or the filename following the first '>>') and will append $LOCKEXT to it. It also says this about that error message: Couldn't determine implicit lockfile from "x" There were no `>>' redirectors to be found, using simply `$LOCKEXT' as locallockfile. The manual also discourages against using a global lock: LOCKFILE Global semaphore file. If this file already exists, procmail will wait until it has gone before proceeding, and will create it itself (cleaning it up when ready, of course). If more than one lockfile are specified, then the previous one will be removed before trying to create the new one. The use of a global lockfile is discouraged, whenever possible use locallock- files (on a per recipe basis) instead. The working is thus. In the first type of recipe: :0 a: $HOME/Mail/_Lists/os-en procmail easily determines that it has to use "$HOME/Mail/_Lists/os-en.lock" as local lock file. On the second type, however: :0 aw: | $DELIVER -m _Lists/D-os-en It knows that there is a pipe, so it looks for the redirect to file at the end of the line to find out what is the destination "folder" of the recipe to create a lock with that name, and not finding one, it simply creates one with the default name of ".lock", which ends being a global lock and breaks parallelization of procmail. The solution, it would seem, is for me to specify a lock file myself: :0f * ^X-Mailinglist: opensuse | /usr/bin/formail -bfi 'Reply-To: "OS-en" <opensuse@opensuse.org>' :0 aw: HOME/Mail/_Lists/os-en.lock | $DELIVER -m _Lists/os-en But this creates a new problem: procmail: [20651] Thu Aug 21 12:02:34 2014 procmail: Locking "/home/cer/Mail/_Lists/os-en.lock" procmail: Unlocking "/home/cer/Mail/_Lists/yp-os-en.lock" procmail: Locking "/home/cer/Mail/_Lists/os-en.lock" procmail: [20702] Thu Aug 21 12:02:34 2014 procmail: Locking "/home/cer/Mail/_Lists/os-en.lock" procmail: [20815] Thu Aug 21 12:02:35 2014 procmail: Locking "/home/cer/Mail/_Lists/os-en.lock" procmail: [20773] Thu Aug 21 12:02:35 2014 procmail: Locking "/home/cer/Mail/_Lists/os-en.lock" procmail: [20652] Thu Aug 21 12:02:42 2014 procmail: Locking "/home/cer/Mail/_Lists/os-en.lock" ... procmail: Locking "/home/cer/Mail/_Lists/os-en.lock" procmail: [20652] Thu Aug 21 12:04:27 2014 procmail: Locking "/home/cer/Mail/_Lists/os-en.lock" procmail: Executing "/usr/lib/dovecot/dovecot-lda,-m,_Lists/os-en" 2 minutes timeout. procmail: [20702] Thu Aug 21 12:05:31 2014 procmail: [20794] Thu Aug 21 12:05:31 2014 procmail: Locking "/home/cer/Mail/_Lists/os-en.lock" procmail: Locking "/home/cer/Mail/_Lists/os-en.lock" ... procmail: Locking "/home/cer/Mail/_Lists/os-en.lock" procmail: [20773] Thu Aug 21 12:06:03 2014 procmail: Locking "/home/cer/Mail/_Lists/os-en.lock" procmail: [20651] Thu Aug 21 12:06:08 2014 procmail: Assigning "LASTFOLDER=/usr/lib/dovecot/dovecot-lda -m _Lists/os-en" procmail: Unlocking "/home/cer/Mail/_Lists/os-en.lock" procmail: Couldn't unlock "/home/cer/Mail/_Lists/os-en.lock" procmail: Notified comsat: "cer@:/usr/lib/dovecot/dovecot-lda -m _Lists/os-en" From opensuse+bounces-162890-......... Thu Aug 21 12:02:10 2014 Subject: [opensuse] Re: on why maildir isn't standard... Folder: /usr/lib/dovecot/dovecot-lda -m _Lists/os-en 9722 The problem is that dovecot-lda also checks and creates a lock file, using that same lock file, of course (/home/cer/Mail/_Lists/os-en.lock). As the lock already exists, it waits... for ever, or till I manually delete the lock file. What should I do? I think that I should tell procmail to abstain from creating a lock file at that point, and let "dovecot-lda" handle it itself. But I have not found a setting for that in procmail. Perhaps removing the last ":" from the recipe :-? But that would instead create a global lock file instead, which breaks parallellization. The manual says: Recipes A line starting with ':' marks the beginning of a recipe. It has the following format: :0 [flags] [ : [locallockfile] ] <zero or more conditions (one per line)> <exactly one action line> ... Local lockfile If you put a second (trailing) ':' on the first recipe line, then procmail will use a locallockfile (for this recipe only). You can optionally specify the locallockfile to use; if you don't however, procmail will use the destination filename (or the filename following the first '>>') and will append $LOCKEXT to it. So I think I should specify a dummy lock file somewhere, perhaps under "/run/user/[UID]/" for speed. But I don't know how to automatically create a folder hierarchy there for my use, on every boot. And link to it by name, not UID. I may concoct something. I will try that. Reading material: The documentation on the dovecot site doesn't mention the issue at all: <http://wiki2.dovecot.org/procmail> And on the procmail site: <http://www.procmail.org/>, many of the links are dead. * a searchable archive of the procmail mailing list <http://www.rosat.mpe-garching.mpg.de/mailing-lists/procmail/> --> Error 404 * Era Eriksson's Procmail Mini-FAQ <http://www.iki.fi/era/procmail/mini-faq.html> redirects to <http://partmaps.org/>, which is in German, I think. Google translate translates: +++······························· Simple shower enclosure or shower cubicle luxury Posted by Maxi on August 14, 2014 . No comments . Besides shower enclosures, which are available in different shapes, you can buy individual shower elements, such as a shower door or a shower wall or a shower enclosure for the tub. Especially handy are the foldable shower enclosures, which are simply placed on the bathtubs and there it ·······························++- I guess that does not help me. :-} * Peter Hartzler's SmartList FAQ <http://www.hartzler.net/smartlist/SmartList-FAQ.html> - --> The requested URL /smartlist/SmartList-FAQ.html was not found on this server. * Jari Aalto's Procmail resources <http://pm-doc.sourceforge.net/> There is nothing here, just a stub from sourceforge. But it links to "Procmail Documentation main page" at <http://pm-doc.sourceforge.net/doc/>, which finally contains real text. I'll read that. There is an interesting section: «4.11 Local lockfile usage» [...] Nope, no way to disable the lock. «5.4 Flag w, lock file and recipe with pipe(|) » It says that if you do not use 'w', the lock file is inmediately removed after handling over to the pipe. BUT, if the pipe fails, procmail would not learn about it, and the email would be lost (if it fails and knows about it, the email is sent instead to the default folder). * Nancy McGough's Procmail Quick Start page at it's primary server or from a mirror. - --> <http://www.ii.com/internet/robots/procmail/qs/>, and the site is up (PROCMAIL QUICK START - An introduction to email filtering with a focus on procmail). Last modified: 2007-11-27T14:37:28 CET There is an interesting info here: it says to run "procmail -v" to find out the lock strategy used by procmail (<http://www.ii.com/internet/robots/procmail/qs/#lock>). It turns out to be: Locking strategies: dotlocking, fcntl() The procmail people aparently decided, long ago, not to use kernel locks - see <http://pm-doc.sourceforge.net/doc/#compiling_procmail_and_choosing_locking_scheme> +++······························· General advice: Everything except dot locking is usually broken. [stephen, <199607292139.XAA12433@hera.cuci.nl>]. Remove fcntl() and lockf(), only allow flock() (or omit it completely) Kernel locks don't work. But that's all some programs use. Across a networked filesystem, lockf() doesn't work, fcntl() and flock() should, but they don't either because the lockd is buggy. Mailtool uses fcntl() but does it wrong, so that's another problem. The only thing that works on all platforms, all networks, all the time are .lock files. ·······························++- The above mentioned link on <http://www.ii.com/internet/robots/procmail/qs/#lock> has very interesting info on locking, but some of the links are dead. +++······························· Mailbox Locking If you use a mail or news client to change a mailbox that is receiving messages — for example, to delete a message — make sure that both procmail and your client use the same locking scheme. To determine what locking strategies are used by your installation of procmail, and the order in which these locking strategies are used, run: procmail -v and look at the value of “Locking strategies.” To learn about file locking, see: 11.4 How does folder locking work? and questions 11.5, 11.6, & 11.16 in the UW Pine Information Center's Info for systems administrators, developers, etc. This is useful to read even if you do not use Pine. (working link). File locking in the Era's Procmail FAQ (redirects to page about showers) 4.11 Local lockfile usage and 4.12 Global lockfile in Jari's Procmail Tips (broken link: 404) The thread "Lock files - confused" in the Procmail mailing list archives (working link) Configuring Netscape Mail on Unix by Jamie Zawinski — this is useful to read even if Netscape is not your mail client. (redirects to silly netscape.aol.com page) [!] Make sure that you are not delivering to mailboxes in Unix spool (mbox) format on an NFS or any other network file system since locking does not help on those systems. For a possible workaround to this, see the comp.mail.imap message Re: UoW IMAP, lock files and multiple servers by Mark Crispin. The Netscape mail client does no locking of its local mailboxes, meaning that there is no safe way to deliver directly to them. Kmail seems to be flaky about locking local mailboxes. For details, search the KDE Documentation for procmail. ·······························++- * Timo Salmi's procmail tips and recipes <http://www.uwasa.fi/~ts/info/proctips.html> redirects to <http://www.netikka.net/tsneti/info/proctips.php>, with the correct info. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iEYEARECAAYFAlP2VLgACgkQtTMYHG2NR9XWwgCfagPCZRpg31oX5LcNzlXz1Kvz i8wAn3D59PZBDvqsas8QEvNc9uUycGfb =A9yO -----END PGP SIGNATURE-----
Carlos E. R. wrote:
procmail: Couldn't determine implicit lockfile from "/usr/lib/dovecot/dovecot-lda"
It also says this about that error message:
Couldn't determine implicit lockfile from "x" There were no `>>' redirectors to be found, using simply `$LOCKEXT' as locallockfile. .... :0 aw: | $DELIVER -m _Lists/D-os-en
It knows that there is a pipe , so it looks for the redirect to file at the end of the line to find out what is the destination "folder" of the recipe to create a lock with that name, and not finding one, it simply creates one with the default name of ".lock", which ends being a global lock and breaks parallelization of procmail....
---- This is why I never converted to procmail...ARG!... Is that for a maildir format? I.e. if you are delivering messages to a directory, doesn't sound like mbox, but if it's a dir, do you need locking? (app would use open+excl to get uniq filename lock)... If you are relying on specific filenames in the dir, why aren't they in the recipe? (in case not obvious, I know nothing about procmail but thought I try some general Q's....) There are times I think about trying to get rid of my email script (1st written in perl ~ 89?). I certainly wouldn't design it the same way today... ;-) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2014-08-22 07:52, Linda Walsh wrote:
Carlos E. R. wrote:
procmail: Couldn't determine implicit lockfile from "/usr/lib/dovecot/dovecot-lda"
It also says this about that error message:
Couldn't determine implicit lockfile from "x" There were no `>>' redirectors to be found, using simply `$LOCKEXT' as locallockfile. .... :0 aw: | $DELIVER -m _Lists/D-os-en
It knows that there is a pipe , so it looks for the redirect to file at the end of the line to find out what is the destination "folder" of the recipe to create a lock with that name, and not finding one, it simply creates one with the default name of ".lock", which ends being a global lock and breaks parallelization of procmail....
---- This is why I never converted to procmail...ARG!...
Is that for a maildir format?
Nope.
I.e. if you are delivering messages to a directory, doesn't sound like mbox, but if it's a dir, do you need locking?
It is an mbox file: "_Lists/D-os-en". There is no ending "/", so it is mbox, not maildir.
(app would use open+excl to get uniq filename lock)...
If you are relying on specific filenames in the dir, why aren't they in the recipe?
But they are. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
participants (3)
-
Carlos E. R.
-
Carlos E. R.
-
Linda Walsh