https://bugzilla.novell.com/show_bug.cgi?id=427264 User rodo@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=427264#c2 --- Comment #2 from Radek Doulik 2008-09-29 03:01:22 MDT --- Created an attachment (id=242175) --> (https://bugzilla.novell.com/attachment.cgi?id=242175) hdparm -I /dev/sde output -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=427264 User rodo@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=427264#c4 --- Comment #4 from Radek Doulik 2008-09-29 03:01:58 MDT --- Created an attachment (id=242177) --> (https://bugzilla.novell.com/attachment.cgi?id=242177) boot log -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=427264 Radek Doulik changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |ASSIGNED Info Provider|rodo@novell.com | -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=427264 Radek Doulik changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #242175|application/octet-stream |text/plain mime type| | -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=427264 Radek Doulik changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #242177|application/octet-stream |text/plain mime type| | -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=427264 User rodo@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=427264#c7 --- Comment #7 from Radek Doulik 2008-09-29 03:46:38 MDT --- Created an attachment (id=242185) --> (https://bugzilla.novell.com/attachment.cgi?id=242185) /var/log/messages from today I thought I will put it under stress a bit today, so I started by deleting 2 openoffice build trees at the same time (that's deleting of about 4-8 gigabytes on md0 device) After rm commands finished I checked the messages log and there are many ext3 problems reported followed by another "frozen" message. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=427264 User rodo@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=427264#c9 --- Comment #9 from Radek Doulik 2008-10-03 01:35:12 MDT --- I have tried secondary PSU and it looks like the ATA bus error freezes are gone. Thanks a lot! (the original/master PSU is 450W, so it looks like the 12V branch supporting the hard drives was overloaded) The timeouts are now happening regularly and only for sde, sdf. Every 30 minutes, probably when smartd wakes up and checks the disks. I will try to not have them go to standby mode to see whether it is related. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=427264 User teheo@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=427264#c11 --- Comment #11 from Tejun Heo 2008-10-03 20:33:53 MDT --- Thanks for confirming. Ah.. another PSU issue. It seems the rating printed on the PSU doesn't have much to do with actual performance and multi-lane PSUs are much worse than single lane one. The best PSU I've got for powering disk array is single lane 350W one which cost me around 15 bucks. :-P -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=427264 User teheo@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=427264#c13 Tejun Heo changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |ASSIGNED Info Provider|rodo@novell.com | --- Comment #13 from Tejun Heo 2008-10-03 20:36:38 MDT --- I missed comment #10. Clearing NEEDINFO. I now have several reports of SMART ENABLE OPERATIONS timeout on different drives. I'll investigate. Thanks. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=427264 User rodo@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=427264#c14 --- Comment #14 from Radek Doulik 2008-10-31 04:57:19 MDT --- Looks like the problems are back even with secondary PSU :( This is what I am getting now: Oct 31 10:58:23 rychlik smartd[4265]: Device: /dev/sda, SMART Usage Attribute: 194 Temperature_Celsius changed from 117 to 119 Oct 31 10:58:23 rychlik smartd[4265]: Device: /dev/sdb, SMART Usage Attribute: 194 Temperature_Celsius changed from 115 to 116 Oct 31 10:58:23 rychlik smartd[4265]: Device: /dev/sdc, SMART Usage Attribute: 194 Temperature_Celsius changed from 117 to 118 Oct 31 10:58:23 rychlik smartd[4265]: Device: /dev/sdd, 1 Currently unreadable (pending) sectors Oct 31 10:58:23 rychlik smartd[4265]: Device: /dev/sdd, 1 Offline uncorrectable sectors Oct 31 10:58:23 rychlik smartd[4265]: Device: /dev/sdd, SMART Usage Attribute: 194 Temperature_Celsius changed from 117 to 119 Oct 31 10:58:23 rychlik gnome-keyring-daemon[3890]: couldn't read 4 bytes from client: Oct 31 10:58:29 rychlik kernel: ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x2 frozen Oct 31 10:58:29 rychlik kernel: ata3.00: cmd b0/da:00:00:4f:c2/00:00:00:00:00/00 tag 0 Oct 31 10:58:29 rychlik kernel: res 40/00:00:01:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout) Oct 31 10:58:29 rychlik kernel: ata3.00: status: { DRDY } Oct 31 10:58:31 rychlik kernel: ata3: soft resetting link Oct 31 10:58:31 rychlik kernel: ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300) Oct 31 10:58:31 rychlik kernel: ata3.00: configured for UDMA/133 Oct 31 10:58:31 rychlik kernel: ata3: EH complete Oct 31 10:58:31 rychlik smartd[4265]: Device: /dev/sde, not capable of SMART self-check Oct 31 10:58:31 rychlik kernel: sd 2:0:0:0: [sde] 976773168 512-byte hardware sectors (500108 MB) Oct 31 10:58:31 rychlik kernel: sd 2:0:0:0: [sde] Write Protect is off Oct 31 10:58:31 rychlik kernel: sd 2:0:0:0: [sde] Mode Sense: 00 3a 00 00 Oct 31 10:58:31 rychlik kernel: sd 2:0:0:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Oct 31 10:58:31 rychlik smartd[4265]: Device: /dev/sde, SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 80 to 82 Oct 31 10:58:31 rychlik smartd[4265]: Device: /dev/sde, SMART Usage Attribute: 194 Temperature_Celsius changed from 178 to 184 Oct 31 10:58:37 rychlik kernel: ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x2 frozen Oct 31 10:58:37 rychlik kernel: ata4.00: cmd b0/da:00:00:4f:c2/00:00:00:00:00/00 tag 0 Oct 31 10:58:37 rychlik kernel: res 40/00:00:01:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout) Oct 31 10:58:37 rychlik kernel: ata4.00: status: { DRDY } Oct 31 10:58:38 rychlik kernel: ata4: soft resetting link Oct 31 10:58:39 rychlik kernel: ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300) Oct 31 10:58:39 rychlik kernel: ata4.00: configured for UDMA/133 Oct 31 10:58:39 rychlik kernel: ata4: EH complete Oct 31 10:58:39 rychlik kernel: sd 3:0:0:0: [sdf] 976773168 512-byte hardware sectors (500108 MB) Oct 31 10:58:39 rychlik kernel: sd 3:0:0:0: [sdf] Write Protect is off Oct 31 10:58:39 rychlik kernel: sd 3:0:0:0: [sdf] Mode Sense: 00 3a 00 00 Oct 31 10:58:39 rychlik kernel: sd 3:0:0:0: [sdf] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Oct 31 10:58:39 rychlik smartd[4265]: Device: /dev/sdf, not capable of SMART self-check Oct 31 10:58:39 rychlik smartd[4265]: Device: /dev/sdf, SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 81 to 82 Oct 31 10:58:39 rychlik smartd[4265]: Device: /dev/sdf, SMART Usage Attribute: 194 Temperature_Celsius changed from 181 to 184 Oct 31 10:59:23 rychlik gnome-keyring-daemon[3890]: couldn't read 4 bytes from client: Oct 31 11:00:16 rychlik syslog-ng[2066]: last message repeated 2 times Oct 31 11:00:23 rychlik gnome-keyring-daemon[3890]: couldn't read 4 bytes from client: Oct 31 11:01:23 rychlik gnome-keyring-daemon[3890]: couldn't read 4 bytes from client: Oct 31 11:02:25 rychlik gnome-keyring-daemon[3890]: couldn't read 4 bytes from client: Oct 31 11:02:25 rychlik gnome-keyring-daemon[3890]: couldn't read 4 bytes from client: Oct 31 11:07:37 rychlik kernel: ata2.00: exception Emask 0x10 SAct 0x0 SErr 0x4050002 action 0xa frozen Oct 31 11:07:37 rychlik kernel: ata2: SError: { RecovComm PHYRdyChg CommWake DevExch } Oct 31 11:07:37 rychlik kernel: ata2.00: cmd c8/00:20:97:45:c3/00:00:00:00:00/ec tag 0 dma 16384 in Oct 31 11:07:37 rychlik kernel: res 40/00:00:01:4f:c2/00:00:00:00:00/10 Emask 0x14 (ATA bus error) Oct 31 11:07:37 rychlik kernel: ata2.00: status: { DRDY } Oct 31 11:07:38 rychlik kernel: ata2: soft resetting link Oct 31 11:07:38 rychlik kernel: ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 300) Oct 31 11:07:38 rychlik kernel: ata2.00: configured for UDMA/100 Oct 31 11:07:38 rychlik kernel: ata2.01: configured for UDMA/100 Oct 31 11:07:38 rychlik kernel: ata2: EH complete Oct 31 11:07:38 rychlik kernel: sd 1:0:0:0: [sdc] 625142448 512-byte hardware sectors (320073 MB) Oct 31 11:07:38 rychlik kernel: sd 1:0:0:0: [sdc] Write Protect is off Oct 31 11:07:39 rychlik kernel: sd 1:0:0:0: [sdc] Mode Sense: 00 3a 00 00 Oct 31 11:07:39 rychlik kernel: sd 1:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Oct 31 11:07:39 rychlik kernel: sd 1:0:1:0: [sdd] 625142448 512-byte hardware sectors (320073 MB) Oct 31 11:07:39 rychlik kernel: sd 1:0:1:0: [sdd] Write Protect is off Oct 31 11:07:39 rychlik kernel: sd 1:0:1:0: [sdd] Mode Sense: 00 3a 00 00 Oct 31 11:07:39 rychlik kernel: sd 1:0:1:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Oct 31 11:07:39 rychlik kernel: sd 1:0:0:0: [sdc] 625142448 512-byte hardware sectors (320073 MB) Oct 31 11:07:39 rychlik kernel: sd 1:0:0:0: [sdc] Write Protect is off Oct 31 11:07:39 rychlik kernel: sd 1:0:0:0: [sdc] Mode Sense: 00 3a 00 00 Oct 31 11:07:39 rychlik kernel: sd 1:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Oct 31 11:07:39 rychlik kernel: sd 1:0:1:0: [sdd] 625142448 512-byte hardware sectors (320073 MB) Oct 31 11:07:40 rychlik kernel: sd 1:0:1:0: [sdd] Write Protect is off Oct 31 11:07:40 rychlik kernel: sd 1:0:1:0: [sdd] Mode Sense: 00 3a 00 00 Oct 31 11:07:40 rychlik kernel: sd 1:0:1:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Oct 31 11:08:23 rychlik gnome-keyring-daemon[3890]: couldn't read 4 bytes from client: Oct 31 11:08:23 rychlik gnome-keyring-daemon[3890]: couldn't read 4 bytes from client: Oct 31 11:09:23 rychlik gnome-keyring-daemon[3890]: couldn't read 4 bytes from client: Oct 31 11:09:23 rychlik gnome-keyring-daemon[3890]: couldn't read 4 bytes from client: Oct 31 11:14:18 rychlik kernel: attempt to access beyond end of device Oct 31 11:14:18 rychlik kernel: md0: rw=0, want=24965021704, limit=1701057920 Oct 31 11:14:18 rychlik kernel: attempt to access beyond end of device Oct 31 11:14:18 rychlik kernel: md0: rw=0, want=19586642472, limit=1701057920 Oct 31 11:14:18 rychlik kernel: attempt to access beyond end of device Oct 31 11:14:18 rychlik kernel: md0: rw=0, want=18388442552, limit=1701057920 Oct 31 11:14:18 rychlik kernel: attempt to access beyond end of device Oct 31 11:14:18 rychlik kernel: md0: rw=0, want=24965021704, limit=1701057920 Oct 31 11:14:18 rychlik kernel: attempt to access beyond end of device Oct 31 11:14:18 rychlik kernel: md0: rw=0, want=24965021704, limit=1701057920 Oct 31 11:14:18 rychlik kernel: attempt to access beyond end of device Oct 31 11:14:18 rychlik kernel: md0: rw=0, want=24965021704, limit=1701057920 Oct 31 11:14:18 rychlik kernel: attempt to access beyond end of device Oct 31 11:14:18 rychlik kernel: md0: rw=0, want=24965021704, limit=1701057920 Oct 31 11:14:18 rychlik kernel: attempt to access beyond end of device Oct 31 11:14:18 rychlik kernel: md0: rw=0, want=24965021704, limit=1701057920 Oct 31 11:14:18 rychlik kernel: attempt to access beyond end of device Oct 31 11:14:18 rychlik kernel: md0: rw=0, want=24965021704, limit=1701057920 Oct 31 11:14:18 rychlik kernel: attempt to access beyond end of device Oct 31 11:14:18 rychlik kernel: md0: rw=0, want=24965021704, limit=1701057920 Oct 31 11:14:18 rychlik kernel: attempt to access beyond end of device Oct 31 11:14:18 rychlik kernel: md0: rw=0, want=24965021704, limit=1701057920 .. I think I will try to turn off the smartd, to see if the ATA errros are related to timeouts. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=427264 User rodo@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=427264#c16 Radek Doulik changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEEDINFO |NEW Info Provider|rodo@novell.com | --- Comment #16 from Radek Doulik 2008-11-10 10:50:13 MST --- Created an attachment (id=251106) --> (https://bugzilla.novell.com/attachment.cgi?id=251106) smartctl -a /dev/sda -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.

https://bugzilla.novell.com/show_bug.cgi?id=427264 User teheo@novell.com added comment https://bugzilla.novell.com/show_bug.cgi?id=427264#c18 Tejun Heo changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |NEEDINFO Info Provider| |rodo@novell.com --- Comment #18 from Tejun Heo 2008-11-10 19:02:12 MST --- I sure hope so. :-P Hmmm... nothing really stands out in the smartctl ouput. Can you please do the followings? 1. Boot, record smartctl -a output. 2. Wait till an error occurs. 3. record smartctl -a output. If my brief powering-off theory is right, you should see Start_Stop_Count and/or Power_Cycle_Count increasing after such errors. Thanks. -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.