[opensuse] Volume mount going mad..
I have this 11.0 server running as a VMware guest. The SUSE system runs only a MySQL database, and of course the most basic processes needed. Every now and then the MySQL server stops answering, and when I check the server, the mounted file system has turned read-only. The file system is a reiserfs 3, and one volume keeps all of the system in one partition. After reboot, reiserfsck cannot fix the file system without going to a complete 'rebuild-tree'. I havenn't been able to find any traces in any log to why it does this, so I'm puzzled and any ideas on this would be great. Anders. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Hey Anders,
I have this 11.0 server running as a VMware guest. The SUSE system runs only a MySQL database, and of course the most basic processes needed. Every now and then the MySQL server stops answering, and when I check the server, the mounted file system has turned read-only.
Which kernel version is running in the guest? And this is a vmware esx, by any chance? greets thomy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Hey Anders,
I have this 11.0 server running as a VMware guest. The SUSE system runs only a MySQL database, and of course the most basic processes needed. Every now and then the MySQL server stops answering, and when I check the server, the mounted file system has turned read-only.
Which kernel version is running in the guest? And this is a vmware esx, by any chance?
greets thomy
This is a SUSE 11.0 guest running kernel 2.6.25.20-0.1-default #1 SMP 2008-12-12 20:30:38 +0100 x86_64 in a dual CPU setup. The VMware layer is VMware Server 1.0.8 running on SUSE host with dual Opteron and kernel 2.6.18.8-0.13-default #1 SMP Thu Nov 6 13:35:13 UTC 2008 x86_64. Anders. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
I have this 11.0 server running as a VMware guest. The SUSE system the server, the mounted file system has turned read-only.
This is a SUSE 11.0 guest running kernel 2.6.25.20-0.1-default #1 SMP 2008-12-12 20:30:38 +0100 x86_64 in a dual CPU setup. The VMware layer is VMware Server 1.0.8 running on SUSE host with dual Opteron and kernel 2.6.18.8-0.13-default #1 SMP Thu Nov 6 13:35:13 UTC 2008 x86_64.
Hm, ok. I had the same problem you describe with a vmware esx, but maybe this can help you: the problem was the kernel scsi-driver in the guest (I assume your guest has also an msi-logic scsi adapter). When the host disk-system was on high traffic, the underlying guest couldn't write his journaling entries to disk in the right amount of time. Before writing nonsense to disk, the guest system decided to go read-only. Solution was to install a top-notch kernel. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
I have this 11.0 server running as a VMware guest. The SUSE system the server, the mounted file system has turned read-only.
This is a SUSE 11.0 guest running kernel 2.6.25.20-0.1-default #1 SMP 2008-12-12 20:30:38 +0100 x86_64 in a dual CPU setup. The VMware layer is VMware Server 1.0.8 running on SUSE host with dual Opteron and kernel 2.6.18.8-0.13-default #1 SMP Thu Nov 6 13:35:13 UTC 2008 x86_64.
Hm, ok.
I had the same problem you describe with a vmware esx, but maybe this can help you: the problem was the kernel scsi-driver in the guest (I assume your guest has also an msi-logic scsi adapter). When the host disk-system was on high traffic, the underlying guest couldn't write his journaling entries to disk in the right amount of time. Before writing nonsense to disk, the guest system decided to go read- only.
Solution was to install a top-notch kernel.
I doesn't sound all that unlikely.. The host has been quite busy "disk-wise" at times... I'll keep an eye on that, maybe I can see a connection next time it happens. Thanks a lot, at least I have something to look at now.. ;-) Anders. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday, 2009-02-02 at 17:35 +0100, Thorsten Moorhoff wrote:
Hm, ok.
I had the same problem you describe with a vmware esx, but maybe this can help you: the problem was the kernel scsi-driver in the guest (I assume your guest has also an msi-logic scsi adapter). When the host disk-system was on high traffic, the underlying guest couldn't write his journaling entries to disk in the right amount of time. Before writing nonsense to disk, the guest system decided to go read-only.
I might have what appears to be a related problem. I have an external disk, connected via usb (which uses the scsi subsystem), on this 11.0 machine, formatted as reiserfs, and it develops errors that force it to become read-only, and requires a long session of reiserfsck --rebuildtree. I see this on the log: Feb 2 02:11:45 nimrodel kernel: sd 0:0:0:0: [sda] Sense Key : No Sense [current] Feb 2 02:11:45 nimrodel kernel: Info fld=0x0 Feb 2 02:11:45 nimrodel kernel: sd 0:0:0:0: [sda] Add. Sense: No additional sense information Feb 2 02:11:45 nimrodel kernel: REISERFS warning: reiserfs-5090 is_tree_node: node level 60651 does not match to the expected one 1 Feb 2 02:11:45 nimrodel kernel: REISERFS error (device sda2): vs-5150 search_by_key: invalid format found in block 12648508. Fsck? Feb 2 02:11:45 nimrodel kernel: REISERFS (device sda2): Remounting filesystem read-only In the log I see hundred of these: Feb 2 16:54:44 nimrodel kernel: Info fld=0x0 Feb 2 16:54:44 nimrodel kernel: sd 1:0:0:0: [sda] Add. Sense: No additional sense information Feb 2 16:54:46 nimrodel kernel: sd 1:0:0:0: [sda] Sense Key : No Sense [current] Feb 2 16:54:46 nimrodel kernel: Info fld=0x0 Feb 2 16:54:46 nimrodel kernel: sd 1:0:0:0: [sda] Add. Sense: No additional sense information Feb 2 16:54:49 nimrodel kernel: sd 1:0:0:0: [sda] Sense Key : No Sense [current] Feb 2 16:54:49 nimrodel kernel: Info fld=0x0 Feb 2 16:54:49 nimrodel kernel: sd 1:0:0:0: [sda] Add. Sense: No additional sense information Feb 2 16:54:53 nimrodel kernel: sd 1:0:0:0: [sda] Sense Key : No Sense [current] Feb 2 16:54:53 nimrodel kernel: Info fld=0x0 Feb 2 16:54:53 nimrodel kernel: sd 1:0:0:0: [sda] Add. Sense: No additional sense information Feb 2 16:54:57 nimrodel kernel: sd 1:0:0:0: [sda] Sense Key : No Sense [current] Feb 2 16:54:57 nimrodel kernel: Info fld=0x0 Feb 2 16:54:57 nimrodel kernel: sd 1:0:0:0: [sda] Add. Sense: No additional sense information Feb 2 16:56:11 nimrodel kernel: sd 1:0:0:0: [sda] Sense Key : No Sense [current] Feb 2 16:56:11 nimrodel kernel: Info fld=0x0 Feb 2 16:56:11 nimrodel kernel: sd 1:0:0:0: [sda] Add. Sense: No additional sense information Feb 2 16:56:15 nimrodel kernel: sd 1:0:0:0: [sda] Sense Key : No Sense [current] Feb 2 16:56:15 nimrodel kernel: Info fld=0x0 Feb 2 16:56:15 nimrodel kernel: sd 1:0:0:0: [sda] Add. Sense: No additional sense information Feb 2 16:56:37 nimrodel kernel: sd 1:0:0:0: [sda] Sense Key : No Sense [current] Feb 2 16:56:37 nimrodel kernel: Info fld=0x0 In my case, I did not have the problem in opensuse 10.3, it apeared on 11.0. Did you fill a Bugzilla? - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkmHq5YACgkQtTMYHG2NR9UrIwCfcEHjxZvaiXeC5rcI78Fa7+eA c0QAnA8+Qh9uiM6sIizsAWYEgp7xy4xD =CWx+ -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Monday, 2009-02-02 at 17:35 +0100, Thorsten Moorhoff wrote:
Hm, ok.
I had the same problem you describe with a vmware esx, but maybe this can help you: the problem was the kernel scsi-driver in the guest (I assume your guest has also an msi-logic scsi adapter). When the host disk-system was on high traffic, the underlying guest couldn't write his journaling entries to disk in the right amount of time. Before writing nonsense to disk, the guest system decided to go read- only.
I might have what appears to be a related problem.
I have an external disk, connected via usb (which uses the scsi subsystem), on this 11.0 machine, formatted as reiserfs, and it develops errors that force it to become read-only, and requires a long session of reiserfsck --rebuildtree.
I see this on the log:
Feb 2 02:11:45 nimrodel kernel: sd 0:0:0:0: [sda] Sense Key : No Sense [current] Feb 2 02:11:45 nimrodel kernel: Info fld=0x0 Feb 2 02:11:45 nimrodel kernel: sd 0:0:0:0: [sda] Add. Sense: No additional sense information Feb 2 02:11:45 nimrodel kernel: REISERFS warning: reiserfs-5090 is_tree_node: node level 60651 does not match to the expected one 1 Feb 2 02:11:45 nimrodel kernel: REISERFS error (device sda2): vs-5150 search_by_key: invalid format found in block 12648508. Fsck? Feb 2 02:11:45 nimrodel kernel: REISERFS (device sda2): Remounting filesystem read-only
In the log I see hundred of these:
Feb 2 16:54:44 nimrodel kernel: Info fld=0x0 Feb 2 16:54:44 nimrodel kernel: sd 1:0:0:0: [sda] Add. Sense: No additional sense information Feb 2 16:54:46 nimrodel kernel: sd 1:0:0:0: [sda] Sense Key : No Sense [current] Feb 2 16:54:46 nimrodel kernel: Info fld=0x0 Feb 2 16:54:46 nimrodel kernel: sd 1:0:0:0: [sda] Add. Sense: No additional sense information Feb 2 16:54:49 nimrodel kernel: sd 1:0:0:0: [sda] Sense Key : No Sense [current] Feb 2 16:54:49 nimrodel kernel: Info fld=0x0 Feb 2 16:54:49 nimrodel kernel: sd 1:0:0:0: [sda] Add. Sense: No additional sense information Feb 2 16:54:53 nimrodel kernel: sd 1:0:0:0: [sda] Sense Key : No Sense [current] Feb 2 16:54:53 nimrodel kernel: Info fld=0x0 Feb 2 16:54:53 nimrodel kernel: sd 1:0:0:0: [sda] Add. Sense: No additional sense information Feb 2 16:54:57 nimrodel kernel: sd 1:0:0:0: [sda] Sense Key : No Sense [current] Feb 2 16:54:57 nimrodel kernel: Info fld=0x0 Feb 2 16:54:57 nimrodel kernel: sd 1:0:0:0: [sda] Add. Sense: No additional sense information Feb 2 16:56:11 nimrodel kernel: sd 1:0:0:0: [sda] Sense Key : No Sense [current] Feb 2 16:56:11 nimrodel kernel: Info fld=0x0 Feb 2 16:56:11 nimrodel kernel: sd 1:0:0:0: [sda] Add. Sense: No additional sense information Feb 2 16:56:15 nimrodel kernel: sd 1:0:0:0: [sda] Sense Key : No Sense [current] Feb 2 16:56:15 nimrodel kernel: Info fld=0x0 Feb 2 16:56:15 nimrodel kernel: sd 1:0:0:0: [sda] Add. Sense: No additional sense information Feb 2 16:56:37 nimrodel kernel: sd 1:0:0:0: [sda] Sense Key : No Sense [current] Feb 2 16:56:37 nimrodel kernel: Info fld=0x0
In my case, I did not have the problem in opensuse 10.3, it apeared on 11.0.
Did you fill a Bugzilla?
- -- Cheers, Carlos E. R.
Thanks for the feedback.. In fact, no I haven't filed any bug to this yet... Darn, I don't have a clue on what's happening on my system since I don't get anything whatsoever in the log files.. But I'll try to keep a closer look and see if I can trace some "external" events triggering this, like the host system being really active on the disk system.. Which I somehow doubt a little since it's a RAID-10 on SATA-300 with a 512MB RAID controller cache, but for now it's my only clue. Anders. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tuesday, 2009-02-03 at 07:22 +0100, Anders Norrbring wrote:
Thanks for the feedback..
In fact, no I haven't filed any bug to this yet... Darn, I don't have a clue on what's happening on my system since I don't get anything whatsoever in the log files..
Try increasing the kernel log verbosity. File "/etc/sysconfig/syslog", variable "KERNEL_LOGLEVEL=7". You have to do an "rcsyslog restart" afterwards. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkmIK3MACgkQtTMYHG2NR9WwtQCfdRc+/fkQe4pkpB04/JyrikIb U9kAnR6J43BMUoSaB3ZDoPVljYTMYFsL =SAww -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
I have this 11.0 server running as a VMware guest. The SUSE system the server, the mounted file system has turned read-only.
This is a SUSE 11.0 guest running kernel 2.6.25.20-0.1-default #1 SMP 2008-12-12 20:30:38 +0100 x86_64 in a dual CPU setup. The VMware layer is VMware Server 1.0.8 running on SUSE host with dual Opteron and kernel 2.6.18.8-0.13-default #1 SMP Thu Nov 6 13:35:13 UTC 2008 x86_64.
Hm, ok.
I had the same problem you describe with a vmware esx, but maybe this can help you: the problem was the kernel scsi-driver in the guest (I assume your guest has also an msi-logic scsi adapter). When the host disk-system was on high traffic, the underlying guest couldn't write his journaling entries to disk in the right amount of time. Before writing nonsense to disk, the guest system decided to go read- only.
Solution was to install a top-notch kernel.
Thinking of it, which SCSI emulation would you guess performs better under Linux? The LSI-Logic or the BusLogic? I haven't been experimenting with them at all, I've just gone along with the default LSI when setting up VMs. Anders. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
participants (3)
-
Anders Norrbring -
Carlos E. R. -
Thorsten Moorhoff