Re: [opensuse] MD5 Raid array does not start
(sorry, forgot to send to list too)
On Sunday 19 July 2009 18:12:31 Leen de Braal wrote:
When I do cat /proc/mdstat it says /dev/md0 is inactive. How can I activate it, and read/repair fs??
In the boot log, don't you have any messages from mdadm that say why it won't start up md0?
Yes. I have been checking some more and I think Murphy's Law is at hand here. Some more info on what happened. This morning I changed one of the disks, because it had failed with more than 180 unrecoverable sectors. After rebooting, the array started rebuilding. Somewhere in /var/log/messages (had a tail -f on a terminal) I saw that the rebuild was done this afternoon. When checking /proc/mdstat I saw to my horror that sda had fallen of the array, sdc was active, and sde (the new disk) was also out of the array. So apparently while rebuilding, a second disk in the array broke. In more or less panic I rebooted, thinking the array might be started with sdc and the new sde as second, and sda as failed disk. But now it seems, that sde has not been able to finish the rebuild, so there I have a lost raid5 (probably). Anyone know of a way to get the data back?
If the raid is degraded (e.g. if a disk is broken), it will refuse to
start up
the raid automatically.
Anders -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-- L. de Braal BraHa Systems NL - Terneuzen T +31 115 649333 F +31 115 649444 -- L. de Braal BraHa Systems NL - Terneuzen T +31 115 649333 F +31 115 649444 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Sunday 19 July 2009 18:48:08 Leen de Braal wrote:
Yes. I have been checking some more and I think Murphy's Law is at hand here. Some more info on what happened. This morning I changed one of the disks, because it had failed with more than 180 unrecoverable sectors. After rebooting, the array started rebuilding. Somewhere in /var/log/messages (had a tail -f on a terminal) I saw that the rebuild was done this afternoon. When checking /proc/mdstat I saw to my horror that sda had fallen of the array, sdc was active, and sde (the new disk) was also out of the array. So apparently while rebuilding, a second disk in the array broke. In more or less panic I rebooted, thinking the array might be started with sdc and the new sde as second, and sda as failed disk. But now it seems, that sde has not been able to finish the rebuild, so there I have a lost raid5 (probably).
Anyone know of a way to get the data back?
I don't think there is a way. With raid5, if two drives break in an array, your data is lost. I suspect you need to restore from backup (I hope you have one) Anders -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Sunday 19 July 2009 18:48:08 Leen de Braal wrote:
Yes. I have been checking some more and I think Murphy's Law is at hand here. Some more info on what happened. This morning I changed one of the disks, because it had failed with more than 180 unrecoverable sectors. After rebooting, the array started rebuilding. Somewhere in /var/log/messages (had a tail -f on a terminal) I saw that the rebuild was done this afternoon. When checking /proc/mdstat I saw to my horror that sda had fallen of the array, sdc was active, and sde (the new disk) was also out of the array. So apparently while rebuilding, a second disk in the array broke. In more or less panic I rebooted, thinking the array might be started with sdc and the new sde as second, and sda as failed disk. But now it seems, that sde has not been able to finish the rebuild, so there I have a lost raid5 (probably).
Anyone know of a way to get the data back?
I don't think there is a way. With raid5, if two drives break in an array, your data is lost. I suspect you need to restore from backup (I hope you have one)
No, this was the backup of a number of servers I maintain. Means all the backups have to rerun (more then 1 TB, will cost a lot of time), and of course the history is gone.
Anders
-- L. de Braal BraHa Systems NL - Terneuzen T +31 115 649333 F +31 115 649444 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
In <340cce3e641310b49e6c14c6fdaefecc.squirrel@mail.braha.nl>, Leen de Braal wrote:
On Sunday 19 July 2009 18:48:08 Leen de Braal wrote:
When checking /proc/mdstat I saw to my horror that sda had fallen of the array, sdc was active, and sde (the new disk) was also out of the array. So apparently while rebuilding, a second disk in the array broke. In more or less panic I rebooted, thinking the array might be started with sdc and the new sde as second, and sda as failed disk. But now it seems, that sde has not been able to finish the rebuild, so there I have a lost raid5 (probably).
Anyone know of a way to get the data back?
I don't think there is a way. With raid5, if two drives break in an array, your data is lost.
No, this was the backup of a number of servers I maintain. Means all the backups have to rerun (more then 1 TB, will cost a lot of time), and of course the history is gone.
Recommend the approach above, but there is another way. mdadm and be used to force-start an array, which will basically tell it to start back where it was just before the array lost the second disk. After you force-start the array, you can fsck the filesystem to correct any errors. Depending on how things are failing, you might get all your data back. No guarantees though. The filesystem might think things are fine and the backups be silently corrupted. Hence, I recommend you take the time to make new backups after getting your RAID5 in good shape. -- Boyd Stephen Smith Jr. ,= ,-_-. =. bss@iguanasuicide.net ((_/)o o(\_)) ICQ: 514984 YM/AIM: DaTwinkDaddy `-'(. .)`-' http://iguanasuicide.net/ \_/
participants (3)
-
Anders Johansson -
Boyd Stephen Smith Jr. -
Leen de Braal