[Bug 516339] New: mysqld regularly crashes
http://bugzilla.novell.com/show_bug.cgi?id=516339 Summary: mysqld regularly crashes Classification: openSUSE Product: openSUSE 11.0 Version: Final Platform: x86-64 OS/Version: Other Status: NEW Severity: Critical Priority: P5 - None Component: Other AssignedTo: bnc-team-screening@forge.provo.novell.com ReportedBy: per.jessen@enidan.com QAContact: qa@suse.de Found By: --- User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.8.1.6) Gecko/20070730 SUSE/2.0.0.6-25 Firefox/2.0.0.6 I'm not sure where this report really belongs, but as I am running an openSUSE built mysql binary I thought I'd start here. Not quite every day but perhaps 3-4 times a week, my mysql server crashes. It happens at roughly the same time every day, during some morning batch processing. Usually the table 'quarantine_archive' will need repairing, but not always. It has some 38 million rows. The batchjob that is running does a few hundred queries like this: mysql -Ddb_spamchek -h localhost -u spamchek_batch --password=spamchek -s -B -e "INSERT IGNORE INTO quarantine_archive SELECT * FROM quarantine WHERE state=1 AND domain='example.com'; DELETE FROM quarantine WHERE state=1 AND domain='example.com'" They're not running in parallel, but one-by-one. I'll attach the mysqld.log and some hardware info. Reproducible: Sometimes -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=516339
User per.jessen@enidan.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c1
--- Comment #1 from Per Jessen
http://bugzilla.novell.com/show_bug.cgi?id=516339
User per.jessen@enidan.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c2
--- Comment #2 from Per Jessen
http://bugzilla.novell.com/show_bug.cgi?id=516339
User meissner@novell.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c3
Marcus Meissner
http://bugzilla.novell.com/show_bug.cgi?id=516339
User per.jessen@enidan.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c4
Per Jessen
http://bugzilla.novell.com/show_bug.cgi?id=516339
User meissner@novell.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c5
--- Comment #5 from Marcus Meissner
http://bugzilla.novell.com/show_bug.cgi?id=516339
User mhrusecky@novell.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c6
Michal Hrusecky
http://bugzilla.novell.com/show_bug.cgi?id=516339
User per.jessen@enidan.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c7
--- Comment #7 from Per Jessen
http://bugzilla.novell.com/show_bug.cgi?id=516339
Michal Hrusecky
http://bugzilla.novell.com/show_bug.cgi?id=516339
User per.jessen@enidan.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c8
--- Comment #8 from Per Jessen
http://bugzilla.novell.com/show_bug.cgi?id=516339
User per.jessen@enidan.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c9
--- Comment #9 from Per Jessen
http://bugzilla.novell.com/show_bug.cgi?id=516339
User per.jessen@enidan.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c10
--- Comment #10 from Per Jessen
http://bugzilla.novell.com/show_bug.cgi?id=516339
User per.jessen@enidan.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c11
--- Comment #11 from Per Jessen
http://bugzilla.novell.com/show_bug.cgi?id=516339
User per.jessen@enidan.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c12
Per Jessen
http://bugzilla.novell.com/show_bug.cgi?id=516339
Michal Hrusecky
http://bugzilla.novell.com/show_bug.cgi?id=516339
User per.jessen@enidan.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c13
--- Comment #13 from Per Jessen
http://bugzilla.novell.com/show_bug.cgi?id=516339
User mhrusecky@novell.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c14
--- Comment #14 from Michal Hrusecky
http://bugzilla.novell.com/show_bug.cgi?id=516339
User per.jessen@enidan.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c15
--- Comment #15 from Per Jessen
My finding so far:
Possibilities that came across my mind so far:
Somehow corrupted key file. If this is the case, dumping table, deleting it and recreating it should help. But as MySQL is trying fix key file after the crash (mentioned in logs) it looks like this is not very probable.
I did in fact dump the table, truncated it and have tried to reload it since. Just reloading it causes the problem, so it is in fact now very reproducable.
Some kind of race condition somewhere. This would explain why it is happening only under heavy load, but it will be hard to find and fix. Newer version differs in some mutexes so this is quite possible.
It is no longer only under heavy load, see above.
As problem seems to be MyISAM related, possible workaround may be to migrate affected table to different storage engine. For example InnoDB.
Out of the question.
Still trying to figure out what went wrong, but wanted to provide some feedback in the meantime.
Thanks, feedback much appreciated. -- Configure bugmail: http://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
http://bugzilla.novell.com/show_bug.cgi?id=516339
User per.jessen@enidan.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c16
--- Comment #16 from Per Jessen
http://bugzilla.novell.com/show_bug.cgi?id=516339
User mhrusecky@novell.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c17
Michal Hrusecky
http://bugzilla.novell.com/show_bug.cgi?id=516339
User per.jessen@enidan.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c18
Per Jessen
http://bugzilla.novell.com/show_bug.cgi?id=516339
User per.jessen@enidan.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c19
--- Comment #19 from Per Jessen
http://bugzilla.novell.com/show_bug.cgi?id=516339
User per.jessen@enidan.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c20
--- Comment #20 from Per Jessen
http://bugzilla.novell.com/show_bug.cgi?id=516339
User mhrusecky@novell.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c21
--- Comment #21 from Michal Hrusecky
http://bugzilla.novell.com/show_bug.cgi?id=516339
User per.jessen@enidan.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c22
--- Comment #22 from Per Jessen
http://bugzilla.novell.com/show_bug.cgi?id=516339
User per.jessen@enidan.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c23
--- Comment #23 from Per Jessen
http://bugzilla.novell.com/show_bug.cgi?id=516339
User mhrusecky@novell.com added comment
http://bugzilla.novell.com/show_bug.cgi?id=516339#c24
Michal Hrusecky
participants (1)
-
bugzilla_noreply@novell.com