On 2006-01-08 19:47 Carl Hartung wrote:
Hi Scott, Hi Anders...
OK, if we're going the Google research route, I also did some searching out of curiosity and what I read indicated the same thing... the kernel starts shutting down processes to preserve itself when it is running out of memory. There were no runaway incremental PIDs being generated, so it seems reasonable to suspect the problem is related to a specific application and not the kernel or overall system health.
Also, one of the original errors that you posted, Anders, pertained to "fast flush". AIUI, the mail server maintains a cache of addresses with pointers to mails that fail to deliver and are stored so sending can be attempted again repeatedly until either a "give it up" point is reached or the mail successfully sends. It looked like I was digging up situations with that error message where the cache and/or stored mails were growing too large and/or too quickly and/or failing to flush correctly... possibly DOS attack and/or spam related.
Could be worthless fodder or clues, I don't know. I'm just hoping to be helpful by tossing it into your salad of things to sample...
Good luck!
- Carl
PS: And Happy New Year!
I don't know if this means anything, but looking in todays 'messages', I see this: Jan 9 05:04:19 iris master[3467]: about to exec /usr/lib/cyrus/bin/pop3d Jan 9 05:04:19 iris master[1030]: process 2174 exited, signaled to death by 9 Jan 9 05:04:19 iris master[1030]: process 2170 exited, signaled to death by 9 Jan 9 05:04:43 iris master[1030]: process 2165 exited, signaled to death by 9 Jan 9 05:04:51 iris kernel: __alloc_pages: 0-order allocation failed (gfp=0x1d2/0) Jan 9 05:04:52 iris kernel: VM: killing process imapd Jan 9 05:04:52 iris kernel: klogd 1.4.1, ---------- state change ---------- Jan 9 05:07:31 iris master[1030]: process 2223 exited, signaled to death by 9 Jan 9 05:08:08 iris master[1030]: process 3467 exited, signaled to death by 9 Jan 9 05:09:29 iris imapd[2232]: seen_db: user nb0002 opened /var/lib/imap/user/n/nb0002.seen Jan 9 05:09:49 iris kernel: Inspecting /boot/System.map-2.4.21-297-default Jan 9 05:13:17 iris master[3493]: about to exec /usr/lib/cyrus/bin/imapd Jan 9 05:13:17 iris master[1030]: process 2063 exited, signaled to death by 9 Jan 9 05:13:17 iris imap[3493]: executed Jan 9 05:13:17 iris master[3494]: about to exec /usr/lib/cyrus/bin/imapd Jan 9 05:13:17 iris imapd[3493]: accepted connection Jan 9 05:13:17 iris imap[3494]: executed Jan 9 05:13:17 iris imapd[3494]: accepted connection Jan 9 05:13:17 iris master[3495]: about to exec /usr/lib/cyrus/bin/imapd Jan 9 05:13:17 iris imap[3495]: executed Jan 9 05:13:17 iris master[3496]: about to exec /usr/lib/cyrus/bin/imapd Jan 9 05:13:17 iris imap[3496]: executed Jan 9 05:13:17 iris imapd[3495]: accepted connection Jan 9 05:13:17 iris imapd[3496]: accepted connection Then looking in the processlist that was dumped by 'ps' half an hour earlier, all the processes that got "signaled to death" are imapd processes, that would be my Cyrus-IMAP... I don't know if it's relevant, but it somehow "feels" like it could be a Cyrus problem... 1890 ? Ss 0:04 /usr/sbin/httpd2-prefork -f /etc/apache2/httpd.conf -DSSL 2041 ? S 0:00 imapd 2049 ? S 0:00 imapd 2051 ? S 0:00 imapd 2061 ? S 0:00 imapd 2063 ? S 0:00 imapd 2155 ? S 0:00 imapd 2156 ? S 0:00 imapd 2157 ? S 0:00 imapd 2158 ? S 0:00 imapd 2159 ? S 0:00 imapd 2160 ? S 0:00 imapd 2161 ? S 0:00 imapd 2162 ? S 0:00 imapd 2163 ? S 0:00 imapd 2164 ? S 0:00 imapd 2165 ? S 0:00 imapd 2166 ? S 0:00 imapd 2167 ? S 0:00 imapd 2168 ? S 0:00 imapd 2169 ? S 0:00 imapd 2170 ? S 0:00 imapd 2171 ? S 0:00 imapd 2172 ? S 0:00 imapd 2173 ? S 0:00 imapd 2174 ? S 0:00 imapd 2175 ? S 0:00 imapd 2176 ? S 0:00 imapd 2177 ? S 0:00 imapd 2178 ? S 0:00 imapd 2179 ? S 0:00 imapd 2222 ? S 0:00 imapd 2223 ? S 0:00 imapd 2224 ? S 0:00 imapd 2225 ? S 0:00 imapd 2232 ? S 0:00 imapd 2857 ? S 0:01 qmgr -l -t fifo -u 3346 ? S 0:00 imapd 3635 ? S 0:00 imapd 3938 ? S 0:00 /usr/lib/samba/classic/smbd -D -s /etc/samba/smb.conf -- Anders Norrbring Norrbring Consulting