Hello, After installing the latest kernel update (from 6.Aug) and rebooting, I'm now getting hunderds of mysqld zombies (defunct) one some servers. I don't really see why yet... the only thing which changed was the kernel... The mysqld is a self-compiled one. The services are still working fine, but that is still a bit disturbing... Anybody noticed the same thing? regards, Olivier -- _______________________________________________________ Olivier Müller - PGP key ID: 0x0E84D2EA - Switzerland E-Mail: http://omx.ch/mail/ - AIM/iChat: swix3k
On Sat, 7 Aug 2004, Olivier Mueller wrote:
Date: Sat, 07 Aug 2004 14:54:01 +0200 From: Olivier Mueller <om-lists-suse-security@omx.ch> To: suse-security@suse.com Subject: [suse-security] many mysqld zombies after latest kernel update
Hello,
After installing the latest kernel update (from 6.Aug) and rebooting, I'm now getting hunderds of mysqld zombies (defunct) one some servers. I don't really see why yet... the only thing which changed was the kernel... The mysqld is a self-compiled one.
The services are still working fine, but that is still a bit disturbing... Anybody noticed the same thing?
Had sam problem but so far I have tracked it back to a corrupted table from a crash last week - I am currently restoring the table from a previous dump. Do a mysqlcheck -A to check the tables to make sure all your tables are ok. Regards Hubba
On Sat, 2004-08-07 at 14:07 +0100, Hubertus Haniel wrote:
Had sam problem but so far I have tracked it back to a corrupted table from a crash last week - I am currently restoring the table from a previous dump. Do a mysqlcheck -A to check the tables to make sure all your tables are ok.
thanks for the idea! But according to mysqlcheck, all tables were ok. So there are still many many little zombies (even after another reboot): 7310 pts/8 Z 0:00 \_ [mysqld] <defunct> 7316 pts/8 Z 0:00 \_ [mysqld] <defunct> 7317 pts/8 Z 0:00 \_ [mysqld] <defunct> 7320 pts/8 Z 0:00 \_ [mysqld] <defunct> 7321 pts/8 Z 0:00 \_ [mysqld] <defunct> [...] regards, Olivier PS: it's a suse 9.0-based system, mysql 4.0.18. currently installing 4.0.20 to see if it may help
On Sat, 2004-08-07 at 15:24 +0200, Olivier Mueller wrote:
PS: it's a suse 9.0-based system, mysql 4.0.18. currently installing 4.0.20 to see if it may help
result: no improvment :-( if you have any idea how I could try to debug that, that would be nice, thanks! (by private mail maybe: it's not really on topic, while still security-update related). I found nothing in the logs (mysql,server) and dmesg. Olivier
On Sat, 7 Aug 2004, Olivier Mueller wrote:
Date: Sat, 07 Aug 2004 15:33:32 +0200 From: Olivier Mueller <om-lists-suse-security@omx.ch> To: suse-security@suse.com Subject: Re: [suse-security] many mysqld zombies after latest kernel update
On Sat, 2004-08-07 at 15:24 +0200, Olivier Mueller wrote:
PS: it's a suse 9.0-based system, mysql 4.0.18. currently installing 4.0.20 to see if it may help
result: no improvment :-(
if you have any idea how I could try to debug that, that would be nice, thanks! (by private mail maybe: it's not really on topic, while still security-update related). I found nothing in the logs (mysql,server) and dmesg.
Olivier
I am SuSE 8.2 with the mysql from the distro - still restoring the corrupted database but others are fine and no more zombies since I have dropped the corrupted database. I am however getting amavis-milter zombies for every incoming and outging mail. This has also started since the new kernel. Regards Hubba
On Sat, 2004-08-07 at 14:54 +0200, Olivier Mueller wrote:
After installing the latest kernel update (from 6.Aug) and rebooting, I'm now getting hunderds of mysqld zombies (defunct) one some servers. I don't really see why yet... the only thing which changed was the kernel... The mysqld is a self-compiled one.
FYI, after reverting to the old kernel, everything is running fine again. En résumé: With: k_smp4G-2.4.21-238.rpm -> zombie problems (kernel-51547 patch) With: k_smp4G-2.4.21-231.rpm -> fine. If you need more details, please feel free to ask. regards, Olivier
On Sat, Aug 07, 2004 at 04:04:57PM +0200, Olivier Mueller wrote:
On Sat, 2004-08-07 at 14:54 +0200, Olivier Mueller wrote:
After installing the latest kernel update (from 6.Aug) and rebooting, I'm now getting hunderds of mysqld zombies (defunct) one some servers. I don't really see why yet... the only thing which changed was the kernel... The mysqld is a self-compiled one.
FYI, after reverting to the old kernel, everything is running fine again.
En résumé:
With: k_smp4G-2.4.21-238.rpm -> zombie problems (kernel-51547 patch) With: k_smp4G-2.4.21-231.rpm -> fine.
If you need more details, please feel free to ask.
Can you run strace -f on the mysqld one and look for /proc/ files it opens ? Ciao, Marcus
On Sat, 7 Aug 2004, Marcus Meissner wrote:
Date: Sat, 7 Aug 2004 16:09:20 +0200 From: Marcus Meissner <meissner@suse.de> To: Olivier Mueller <om-lists-suse-security@omx.ch> Cc: suse-security@suse.com, security@suse.de Subject: Re: [suse-security] many mysqld zombies after latest kernel update
On Sat, Aug 07, 2004 at 04:04:57PM +0200, Olivier Mueller wrote:
On Sat, 2004-08-07 at 14:54 +0200, Olivier Mueller wrote:
After installing the latest kernel update (from 6.Aug) and rebooting, I'm now getting hunderds of mysqld zombies (defunct) one some servers. I don't really see why yet... the only thing which changed was the kernel... The mysqld is a self-compiled one.
FYI, after reverting to the old kernel, everything is running fine again.
En résumé:
With: k_smp4G-2.4.21-238.rpm -> zombie problems (kernel-51547 patch) With: k_smp4G-2.4.21-231.rpm -> fine.
If you need more details, please feel free to ask.
Can you run strace -f on the mysqld one and look for /proc/ files it opens ?
After fixing my database corruption mysql seems to run fine again - I am still having problems with amavis-milter under SuSE 8.2 pro though which have appeared since the new kernel update. Can not see that it is opening any proc files though: haniel:~ # ps -ef | grep amavis vscan 15734 15733 0 17:26 ? 00:00:00 /usr/sbin/amavis-milter -D -p local:/var/run/amavis/amavis-milter.sock vscan 15735 15734 0 17:26 ? 00:00:00 /usr/sbin/amavis-milter -D -p local:/var/run/amavis/amavis-milter.sock vscan 15809 1 0 17:26 ? 00:00:00 /usr/bin/perl -T /usr/sbin/amavisd vscan 15825 15734 0 17:26 ? 00:00:00 [amavis-milter] <defunct> vscan 15909 15734 0 17:27 ? 00:00:00 [amavis-milter] <defunct> vscan 17586 15734 0 17:37 ? 00:00:00 [amavis-milter] <defunct> vscan 17709 15734 0 17:38 ? 00:00:00 [amavis-milter] <defunct> vscan 15733 1 0 17:26 ? 00:00:00 /usr/sbin/amavis-milter -D -p local:/var/run/amavis/amavis-milter.sock root 17785 13746 0 17:38 pts/1 00:00:00 grep amavis haniel:~ # strace -fp 15733 select(6, [5], NULL, [5], {3, 520000}) = 0 (Timeout) select(6, [5], NULL, [5], {5, 0}) = 1 (in [5], left {1, 790000}) accept(5, {sa_family=AF_UNIX, path=@}, [2]) = 6 setsockopt(6, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0 rt_sigprocmask(SIG_SETMASK, NULL, [HUP INT TERM RTMIN], 8) = 0 write(4, "`\0\7@\0\0\0\0\0\0\0\0P\273\4\10x\375\5\10\3@\0\200\0\0"..., 148) = 148 rt_sigprocmask(SIG_SETMASK, NULL, [HUP INT TERM RTMIN], 8) = 0 rt_sigsuspend([HUP INT TERM] <unfinished ...> --- SIGRTMIN (Unknown signal 32) @ 0 (0) --- <... rt_sigsuspend resumed> ) = -1 EINTR (Interrupted system call) sigreturn() = ? (mask now [HUP INT TERM RTMIN]) select(6, [5], NULL, [5], {5, 0}) = 0 (Timeout) select(6, [5], NULL, [5], {5, 0}) = 0 (Timeout) select(6, [5], NULL, [5], {5, 0} regards Hubba
On Sat, 7 Aug 2004, Hubertus Haniel wrote:
Date: Sat, 7 Aug 2004 17:41:09 +0100 (BST) From: Hubertus Haniel <hubba@unixcook.com> To: Marcus Meissner <meissner@suse.de> Cc: Olivier Mueller <om-lists-suse-security@omx.ch>, suse-security@suse.com, security@suse.de Subject: Re: [suse-security] many mysqld zombies after latest kernel update
On Sat, 7 Aug 2004, Marcus Meissner wrote:
Date: Sat, 7 Aug 2004 16:09:20 +0200 From: Marcus Meissner <meissner@suse.de> To: Olivier Mueller <om-lists-suse-security@omx.ch> Cc: suse-security@suse.com, security@suse.de Subject: Re: [suse-security] many mysqld zombies after latest kernel update
On Sat, Aug 07, 2004 at 04:04:57PM +0200, Olivier Mueller wrote:
On Sat, 2004-08-07 at 14:54 +0200, Olivier Mueller wrote:
After installing the latest kernel update (from 6.Aug) and rebooting, I'm now getting hunderds of mysqld zombies (defunct) one some servers. I don't really see why yet... the only thing which changed was the kernel... The mysqld is a self-compiled one.
FYI, after reverting to the old kernel, everything is running fine again.
En résumé:
With: k_smp4G-2.4.21-238.rpm -> zombie problems (kernel-51547 patch) With: k_smp4G-2.4.21-231.rpm -> fine.
If you need more details, please feel free to ask.
Can you run strace -f on the mysqld one and look for /proc/ files it opens ?
After fixing my database corruption mysql seems to run fine again - I am still having problems with amavis-milter under SuSE 8.2 pro though which have appeared since the new kernel update. Can not see that it is opening any proc files though:
[SNIP] Correction on that - I just rebooted again and the mysql zombies are back so I am downgrading back to 2.4.20-115 as 2.4.20-118 creates too many problems. For some reason the mysql problem only comes up straight after the reboot, if you stop mysql after booting with rcmysql stop leaving it down for a couple of minutes and then starting it up again with rcmysql start does not let you reproduce the problem under 8.2. Best regards Hubba
Olivier Mueller schrieb:
Hello,
After installing the latest kernel update (from 6.Aug) and rebooting, I'm now getting hunderds of mysqld zombies (defunct) one some servers. I don't really see why yet... the only thing which changed was the kernel... The mysqld is a self-compiled one.
The services are still working fine, but that is still a bit disturbing... Anybody noticed the same thing?
regards, Olivier
Same problem here, after kernelupgrade i have lot of mysqldzombies, mysqlcheck means everything is ok! Werner
On Sun, 08 Aug 2004 18:35:32 +0200, Werner Amon wrote
Olivier Mueller schrieb:
Hello,
After installing the latest kernel update (from 6.Aug) and rebooting, I'm now getting hunderds of mysqld zombies (defunct) one some servers. I don't really see why yet... the only thing which changed was the kernel... The mysqld is a self-compiled one.
The services are still working fine, but that is still a bit disturbing... Anybody noticed the same thing?
regards, Olivier
Same problem here, after kernelupgrade i have lot of mysqldzombies, mysqlcheck means everything is ok!
By me too. (>120 mysqld processes (on SuSE 8.2)) ??? But seems to work OK.
Werner
-- Check the headers for your unsubscription address For additional commands, e-mail: suse-security-help@suse.com Security-related bug reports go to security@suse.de, not here
On Sun, 08 Aug 2004 18:35:32 +0200, Werner Amon wrote
Olivier Mueller schrieb:
Hello,
After installing the latest kernel update (from 6.Aug) and rebooting, I'm now getting hunderds of mysqld zombies (defunct) one some servers. I don't really see why yet... the only thing which changed was the kernel... The mysqld is a self-compiled one.
The services are still working fine, but that is still a bit disturbing... Anybody noticed the same thing?
regards, Olivier
Ooops, forgot somehing, maybe interesting: after kernel compile and reboot I had the > 120 mysqlzombies. rcmysql restart --> seems to be normal, but after an afternoon there are again 64 zombies. A restart helps only for a while.
Same problem here, after kernelupgrade i have lot of mysqldzombies, mysqlcheck means everything is ok!
Werner
-- Check the headers for your unsubscription address For additional commands, e-mail: suse-security-help@suse.com Security-related bug reports go to security@suse.de, not here
-- Yes of course I'm sure it's the red cable. I guarante[^%!/+)F#0c|'NO CARRIER -- STTS
participants (5)
-
Hubertus Haniel -
Marcus Meissner -
Mátyás Tibor -
Olivier Mueller -
Werner Amon