[opensuse] Does openSuSE Ever run fsck on disks in dmraid array with nvidia controller?
Filesystem gurus, When checking the filesystem boot count or when faced with a filesystem error, "Does openSuSE ever actually fsck 'the disks' in the array?" If so how? It is impossible to properly fsck the individual disks in an array on some controllers, like nvidia, without first disabling the dmraid BIOS function and you cannot update the 'bad block table' of an array. How is this handled? Or,... is this just an unmentioned disadvantage of dmraid with some controllers? I ask because I just stumbled through confirming this for myself with a nvidia dmraid chipset on one box. I know I have been able to fsck individual disks with the 'Promise' dmraid controller. What about those like nvidia? Even after booting from install media or a live-CD, e2fsck will refuse to run on the individual disks claiming this the disk is 'mounted or under the exclusive control of another process'. It is not until the array is destroyed that fsck will run. Is there some software workaround employed by openSuSE? -- David C. Rankin, J.D.,P.E. Rankin Law Firm, PLLC 510 Ochiltree Street Nacogdoches, Texas 75961 Telephone: (936) 715-9333 Facsimile: (936) 715-9339 www.rankinlawfirm.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Fri, 2010-01-22 at 09:58 -0600, David C. Rankin wrote:
Filesystem gurus,
When checking the filesystem boot count or when faced with a filesystem error, "Does openSuSE ever actually fsck 'the disks' in the array?" If so how? It is impossible to properly fsck the individual disks in an array on some controllers, like nvidia, without first disabling the dmraid BIOS function and you cannot update the 'bad block table' of an array.
The file system check (fsck) works, as the name implies, on the file system level. Whatever lies underneath the file system is completely unknown to fsck, as long as it presents it with an API it can use. You run fsck on the block device containing the file system, and this does not mean the single disks in a RAID, it means the top-level RAID device As far as I know, fsck never does badblock analysis Anders -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Friday, 2010-01-22 at 17:23 +0100, Anders Johansson wrote: ...
As far as I know, fsck never does badblock analysis
As a matter of fact, it does. Kind of. For instance, for ext3 it calls e2fsck, and this one has options for badblock handling. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAktZ2f0ACgkQtTMYHG2NR9WtHgCeJZ3vt/R9CkZknlY8/xC3u2XN Bg8AnRkIDDRybk7S1etoq3CAFmacoqZp =Z/0W -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On 01/22/2010 11:01 AM, Carlos E. R. wrote:
On Friday, 2010-01-22 at 17:23 +0100, Anders Johansson wrote:
...
As far as I know, fsck never does badblock analysis
As a matter of fact, it does. Kind of. For instance, for ext3 it calls e2fsck, and this one has options for badblock handling.
-- Cheers, Carlos E. R.
There in lies an Achilles' of dmraid that I am interested in. If there isn't a specific kernel level workaround to temporarily disable a dmraid array to check and correct any normally easily correctable 'disk' errors such as bad blocks, etc.., then that means a dmraid setup will suppress/prevent the correction of disk errors on each 'disk' in the array allowing simple correctable errors to propagate or grow into multiple compounded errors resulting in disk deterioration to the point of data loss. Simply put, if small disk errors go uncorrected in the dmraid array and are allowed to remain uncorrected and multiply, then dmraid has a gaping hole in its robustness making it look far inferior to software raid. I can't believe that it the way dmraid works. In the past, I have actually preferred it to software raid because I can rebuild a new disk following a drive failure and remake the array before I ever have to boot the operating system. But, if you can never e2fsck -fcy /dev/sda, sdb, etc.. without first disabling the array in the bios, you are basically playing Russian roulette with disk errors on any single disk in the array. Does anybody know more or have any links to detailed information on how bad blocks are handled in dmraid?? If not, I guess I'll forward the discussion to the dmraid maintainer over at RedHat and see what he has to say as well. Thanks. All of this discussion is prompted by 2 recent supposed disk 'failures' in dmraid setups where one disk will be pristine, but the other will have hundreds of simply corrected errors that have never been fixed over the life of the array, that after disabling the array and running e2fsck are fully correctable. In all the dmraid documentation I have read, I have never seen a warning or note that says: "Periodic disabling of the dmraid array will be required to allow fsck or other disk maintenance processes to be run on the individual disks of the array because enabling dmraid in the BIOS prevents disk maintenance on individual disks in the array." I'm concerned that something like that is needed. The question is, "Is that true?" -- David C. Rankin, J.D.,P.E. Rankin Law Firm, PLLC 510 Ochiltree Street Nacogdoches, Texas 75961 Telephone: (936) 715-9333 Facsimile: (936) 715-9339 www.rankinlawfirm.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Saturday, 2010-01-23 at 13:20 -0600, David C. Rankin wrote:
On 01/22/2010 11:01 AM, Carlos E. R. wrote:
On Friday, 2010-01-22 at 17:23 +0100, Anders Johansson wrote:
...
As far as I know, fsck never does badblock analysis
As a matter of fact, it does. Kind of. For instance, for ext3 it calls e2fsck, and this one has options for badblock handling.
There in lies an Achilles' of dmraid that I am interested in. If there isn't a specific kernel level workaround to temporarily disable a dmraid array to check and correct any normally easily correctable 'disk' errors such as bad blocks, etc.., then that means a dmraid setup will suppress/prevent the correction of disk errors on each 'disk' in the array allowing simple correctable errors to propagate or grow into multiple compounded errors resulting in disk deterioration to the point of data loss.
Whoa! Hold on. The posibility of using fsck to mark badblocks is _NOT_ used on contemporary hard disks. Period :-) Badblocks are left to be managed by the hard disk firmware internally and transparently, when attemting to write to a known (by the HD) bad block.
Simply put, if small disk errors go uncorrected in the dmraid array and are allowed to remain uncorrected and multiply, then dmraid has a gaping hole in its robustness making it look far inferior to software raid.
I can't believe that it the way dmraid works. In the past, I have actually preferred it to software raid because I can rebuild a new disk following a drive failure and remake the array before I ever have to boot the operating system. But, if you can never e2fsck -fcy /dev/sda, sdb, etc.. without first disabling the array in the bios, you are basically playing Russian roulette with disk errors on any single disk in the array.
If you do that, you will corrupt your filesystem, and not be able to reenable the array. Both images will be different, and, I guess, when the array is reenabled the newer copy will simply be overwritten to the older copy, including the badblocks; ie, bad blocks in side 0 will be copied and marked bad to side 1, even if there are none there. And perhaps, good blocks on 0 will overwrite bad blocks on 1. Notice that fsck bad blocks works on the filesystem level, it does never see raid elements, which is correct. The raid, any type of raid, presents a unique device to the filesystem layer. If you disable the array and run fsck on the elements, fsck will work on the apparent filesystem, both sides will be different as a result, and the array will be corrupted in the end. As best, the newer side will overwrite the older side. Do not attempt to correct badblocks from the operating system on an array; leave that to the disk firmware. Just use smartctl to trigger the long test (on each element), which includes a surface scan. If there are uncorrected sectors after that, the procedure would be to rewrite the affected sectors (or the entire disk). - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAktbVNwACgkQtTMYHG2NR9UHRwCcDDp2+4EkNQs2ceUgs6pHpY4O 2+oAnirvL/sfAYI0xhjexYbi1rbkblcV =mJM4 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Carlos E. R. wrote:
On Saturday, 2010-01-23 at 13:20 -0600, David C. Rankin wrote:
On 01/22/2010 11:01 AM, Carlos E. R. wrote:
On Friday, 2010-01-22 at 17:23 +0100, Anders Johansson wrote:
...
As far as I know, fsck never does badblock analysis
As a matter of fact, it does. Kind of. For instance, for ext3 it calls e2fsck, and this one has options for badblock handling.
There in lies an Achilles' of dmraid that I am interested in. If there isn't a specific kernel level workaround to temporarily disable a dmraid array to check and correct any normally easily correctable 'disk' errors such as bad blocks, etc.., then that means a dmraid setup will suppress/prevent the correction of disk errors on each 'disk' in the array allowing simple correctable errors to propagate or grow into multiple compounded errors resulting in disk deterioration to the point of data loss.
Whoa! Hold on.
The posibility of using fsck to mark badblocks is _NOT_ used on contemporary hard disks.
Yep. You have to go pretty far back for that to be a realistic option. Pre-IDE days, I would say. /Per -- Per Jessen, Zürich (-0.8°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sunday, 2010-01-24 at 10:01 +0100, Per Jessen wrote:
Carlos E. R. wrote:
Whoa! Hold on.
The posibility of using fsck to mark badblocks is _NOT_ used on contemporary hard disks.
Yep. You have to go pretty far back for that to be a realistic option. Pre-IDE days, I would say.
Maybe not so far back. Only since SMART appeared, and did remapping in hardware. Remember that the msdos FAT checkdisk utility did this remapping, on the filesystem, by default, since day one: it was an absolute necessity for floppies, and even hard disks. The first hard disk I owned (32MB) came with a label listing bad blocks. However, ext3 or reiser fsck did not, till a few years back: this feature was added relatively recently. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAktcS70ACgkQtTMYHG2NR9XZKACbBvLiYQnekRk2PhSJKfcwZbGO 6WsAnR+MIO/tKrp5uNe24HOv150zSAq7 =vNq9 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Carlos E. R. wrote:
On Sunday, 2010-01-24 at 10:01 +0100, Per Jessen wrote:
Carlos E. R. wrote:
Whoa! Hold on.
The posibility of using fsck to mark badblocks is _NOT_ used on contemporary hard disks.
Yep. You have to go pretty far back for that to be a realistic option. Pre-IDE days, I would say.
Maybe not so far back. Only since SMART appeared, and did remapping in hardware.
IDE (PATA) drives have always done remapping of bad blocks themselves. SMART is just the monitoring part of that.
Remember that the msdos FAT checkdisk utility did this remapping, on the filesystem, by default, since day one: it was an absolute necessity for floppies, and even hard disks. The first hard disk I owned (32MB) came with a label listing bad blocks.
Presumably, none of those were IDE/ATA drives? /Per -- Per Jessen, Zürich (-0.8°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Per Jessen wrote:
Carlos E. R. wrote:
On Sunday, 2010-01-24 at 10:01 +0100, Per Jessen wrote:
Carlos E. R. wrote:
Whoa! Hold on.
The posibility of using fsck to mark badblocks is _NOT_ used on contemporary hard disks.
Yep. You have to go pretty far back for that to be a realistic option. Pre-IDE days, I would say.
Maybe not so far back. Only since SMART appeared, and did remapping in hardware.
IDE (PATA) drives have always done remapping of bad blocks themselves. SMART is just the monitoring part of that.
Remember that the msdos FAT checkdisk utility did this remapping, on the filesystem, by default, since day one: it was an absolute necessity for floppies, and even hard disks. The first hard disk I owned (32MB) came with a label listing bad blocks.
Presumably, none of those were IDE/ATA drives?
Just clarify - when I said pre-IDE, I meant disks with ST-506 or ESDI interfaces, where the actual disk controlling logic sat on a disk controller, as opposed to the ATA/IDE drives with integrated electronics (IDE). /Per -- Per Jessen, Zürich (-0.8°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sunday, 2010-01-24 at 15:12 +0100, Per Jessen wrote:
Per Jessen wrote:
Yep. You have to go pretty far back for that to be a realistic option. Pre-IDE days, I would say.
Maybe not so far back. Only since SMART appeared, and did remapping in hardware.
IDE (PATA) drives have always done remapping of bad blocks themselves. SMART is just the monitoring part of that.
I'm not sure of that. I only learnt of that feature after the turn of the century.
Remember that the msdos FAT checkdisk utility did this remapping, on the filesystem, by default, since day one: it was an absolute necessity for floppies, and even hard disks. The first hard disk I owned (32MB) came with a label listing bad blocks.
Presumably, none of those were IDE/ATA drives?
No, the 32MB unit is pre-ATA, intelligence in the driver card. Step motor.
Just clarify - when I said pre-IDE, I meant disks with ST-506 or ESDI interfaces, where the actual disk controlling logic sat on a disk controller, as opposed to the ATA/IDE drives with integrated electronics (IDE).
Yes, but I'm not sure that the IDE HDs in the 90's had remapping capability. Maybe some had, but I don't remember the units I used or owned (80 MB, 500 MB, 8GB...) having it. It was handled by MsDos checkdisk. Maybe it had to be activated? :-? - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAktcfwIACgkQtTMYHG2NR9WORgCfSRs6TDuLjtw0GYa5GypMvzLY /fgAnR1DeLbM7dz850Q/mRGhHzS2iUXy =X/0y -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Carlos E. R. wrote:
On Sunday, 2010-01-24 at 15:12 +0100, Per Jessen wrote:
Per Jessen wrote:
Yep. You have to go pretty far back for that to be a realistic option. Pre-IDE days, I would say.
Maybe not so far back. Only since SMART appeared, and did remapping in hardware.
IDE (PATA) drives have always done remapping of bad blocks themselves. SMART is just the monitoring part of that.
I'm not sure of that. I only learnt of that feature after the turn of the century.
Okay, maybe not always, but after harddrives switched to GMR (thanks Jülich and IBM), they definitely did the bad block remapping themselves. Maybe early 90s, I'm not sure. SMART dates back to that time too. /Per -- Per Jessen, Zürich (-0.8°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sunday, 2010-01-24 at 20:41 +0100, Per Jessen wrote:
Carlos E. R. wrote:
IDE (PATA) drives have always done remapping of bad blocks themselves. SMART is just the monitoring part of that.
I'm not sure of that. I only learnt of that feature after the turn of the century.
Okay, maybe not always, but after harddrives switched to GMR (thanks Jülich and IBM), they definitely did the bad block remapping themselves. Maybe early 90s, I'm not sure. SMART dates back to that time too.
Maybe not all brands had the same features. This wikipedia article says more or less what we have been saying: <http://en.wikipedia.org/wiki/Bad_sector> and here: <http://www.mjm.co.uk/sectorremapping.html> But the references I see say "modern hard disks", with no date specified. Curious :-) - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAktc5JoACgkQtTMYHG2NR9XmIwCeKq3fjWCU2mPP1pdqowgmP/eL PXIAniToCghyRKALLwDDxoaVRPSEE3WA =WiRS -----END PGP SIGNATURE-----
On 01/23/2010 01:58 PM, Carlos E. R. wrote:
I can't believe that it the way dmraid works. In the past, I have actually preferred it to software raid because I can rebuild a new disk following a drive failure and remake the array before I ever have to boot the operating system. But, if you can never e2fsck -fcy /dev/sda, sdb, etc.. without first disabling the array in the bios, you are basically playing Russian roulette with disk errors on any single disk in the array. If you do that, you will corrupt your filesystem, and not be able to reenable the array. Both images will be different, and, I guess, when the array is reenabled the newer copy will simply be overwritten to the older copy, including the badblocks; ie, bad blocks in side 0 will be copied and marked bad to side 1, even if there are none there. And perhaps, good blocks on 0 will overwrite bad blocks on 1.
Happily, there is some magic somewhere in dmraid that makes this work OK. The key is to boot with some other media so that the 2 disks in question are not mounted or used by the booted OS. After fsck'ing the disk in question, mount both disks say under /mnt/a and /mnt/b. Then if 'b' was the drive fsck'ed, do a cp -a /mnt/a/* /mnt/b. Then re-enable the array in the bios on the next boot and all is well. As for the 'bad block' terminology, I may be off. What I'm talking about having to fsck to cure that isn't happening in dmraid is the offline uncorrectable errors: 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 207 I had been adding the bad block scan to fsck, perhaps wrongly so, but to force a surface scan. The error above was the error that prompted the response to fsck the disk with the bad block check enabled. What ever errors fsck finds and fixes with the -fcy option are the errors that are NOT being fixed when the disks are configured in a dmraid array. That's what was the genesis of the post and concern about the robustness of dmraid over time. The question is still pending though. How are (whatever these errors are) handled by suse when a disk is part of an dmraid array? You cannot fsck the disk when it is part of the array because fsck exits and throws the error that the disk is 'under the exclusive control of another process' and refuses to test the disk. My experience here is that the disk is FINE and just needs a simple fsck and so far that has been true in two out of three dmraid arrays I have had errors on during the past year. (in one case, the disk was cratering). What I want to know is: "Is there any individual disk level error correction performed on disks in dmraid arrays, or is it as it looks -- no fsck error correction is ever performed on individual disks in a dmraid array?" The ultimate question is should I ditch dmraid entirely and go with mdraid? dmraid has been absolutely bullet-proof as has been my mdraid installs. But, if mdraid can handle single disk error correction within an array where dmraid can't, that would be enough justification for me to dump dmraid in favor of mdraid. dmraid and mdraid are incredibly similar in the flexibility they offer as a raid solution. In either case you have mirrored copies, where upon failure, or just for kicks, you can rip one disk out of the array and boot and run on a single disk, or put it in another box without any concern about raid incompatibility. Both just use normal partitions and disks, the only primary difference is in how they are joined, either through a bios function or through a software function. I have liked dmraid because I can easily mirror /, /boot, /home and swap on each disk so that in the event of disk failure, the most work one disk can ever need to act as a standalone is to reinstall grub in the mbr if the boot loader code just happened to be other disk. (small potatoes) But, if there is a fundamental difference in the single disk maintenance/error correction area, that would make a big difference in the robustness of one versus the other. I just don't know the answer to that question and after extensive googling it doesn't seem like anyone else does either (or they just haven't discussed it) Thus my post -- Anybody know the answer?? Anders, are you a raid guy too?? (P.S. -- you know it is a good problem when the question is hard to frame :-) -- David C. Rankin, J.D.,P.E. Rankin Law Firm, PLLC 510 Ochiltree Street Nacogdoches, Texas 75961 Telephone: (936) 715-9333 Facsimile: (936) 715-9339 www.rankinlawfirm.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
David C. Rankin wrote:
The question is still pending though. How are (whatever these errors are) handled by suse when a disk is part of an dmraid array? You cannot fsck the disk when it is part of the array because fsck exits and throws the error that the disk is 'under the exclusive control of another process' and refuses to test the disk.
You do not have a need to fsck any individual disk that is part of an array. fsck checks the filesystem, so you run fsck on the array device that holds your file system. That's it. Any errors with an individual drive will be reported by the array controller (whether software or hardware).
What I want to know is: "Is there any individual disk level error correction performed on disks in dmraid arrays, or is it as it looks -- no fsck error correction is ever performed on individual disks in a dmraid array?"
The latter.
The ultimate question is should I ditch dmraid entirely and go with mdraid? dmraid has been absolutely bullet-proof as has been my mdraid installs. But, if mdraid can handle single disk error correction within an array where dmraid can't, that would be enough justification for me to dump dmraid in favor of mdraid.
AFAICT, mdraid does not "handle" a drive error any better or any worse. It would report it and run the array in degraded mode. What dmraid does will presumably depend on what your hardware controller does. I.e. when your RAID chipset decides the mirror is bad, how does it inform you?
dmraid and mdraid are incredibly similar in the flexibility they offer as a raid solution. In either case you have mirrored copies, where upon failure, or just for kicks, you can rip one disk out of the array and boot and run on a single disk, or put it in another box without any concern about raid incompatibility. Both just use normal partitions and disks, the only primary difference is in how they are joined, either through a bios function or through a software function.
I could be way wrong, but my understanding is that dmraid is an interface to various hardware/on-board RAID controllers, whereas mdraid actively does the mirroring.
I have liked dmraid because I can easily mirror /, /boot, /home and swap on each disk so that in the event of disk failure, the most work one disk can ever need to act as a standalone is to reinstall grub in the mbr if the boot loader code just happened to be other disk. (small potatoes)
FYI, mdraid will do that too, no big deal. Isn't that the whole point? I think the primary difference between the two is that mdraid takes up CPU cycles doing the mirroring, dmraid does not. /Per -- Per Jessen, Zürich (-1.9°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thursday, 2010-01-28 at 09:12 +0100, Per Jessen wrote:
FYI, mdraid will do that too, no big deal. Isn't that the whole point? I think the primary difference between the two is that mdraid takes up CPU cycles doing the mirroring, dmraid does not.
Both do, AFAIK. Motherboard, bios raid, is not a real hardware raid, it needs a driver that does the real work in the cpu, with some help from the chipset. That's why it is called "fake raid". - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAktiA/QACgkQtTMYHG2NR9V9AwCfeSbwJnci7/ZhXDummTyLoaqf ih4AnAqR0yMdo5gtNz0oMwE9Ze7JCYdU =NYoP -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Thu, Jan 28, 2010 at 4:38 PM, Carlos E. R. <robin.listas@telefonica.net> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Thursday, 2010-01-28 at 09:12 +0100, Per Jessen wrote:
FYI, mdraid will do that too, no big deal. Isn't that the whole point? I think the primary difference between the two is that mdraid takes up CPU cycles doing the mirroring, dmraid does not.
Both do, AFAIK. Motherboard, bios raid, is not a real hardware raid, it needs a driver that does the real work in the cpu, with some help from the chipset. That's why it is called "fake raid".
- -- Cheers, Carlos E. R.
I don't know the specifics, but mdraid for sure and possibly dmraid have been getting updates in the last year or so to offload the parity logic. I assume that some raid cards now expose their xor engine and thus the data blocks can be sent to them to handle the xor logic. The patches I noticed were for IBM hardware iirc. I think the really big difference between dmraid and mdraid is compatibility. MS Windows only supports the equivalent of dmraid, so if you want to create a dual boot raid system, you need to go that way. If Windows is not a concern, I think mdraid is the obvious choice but there may be some subtleties I don't know about. Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thursday, 2010-01-28 at 17:17 -0500, Greg Freemyer wrote:
On Thu, Jan 28, 2010 at 4:38 PM, Carlos E. R. <> wrote:
Both do, AFAIK. Motherboard, bios raid, is not a real hardware raid, it needs a driver that does the real work in the cpu, with some help from the chipset. That's why it is called "fake raid".
I don't know the specifics, but mdraid for sure and possibly dmraid have been getting updates in the last year or so to offload the parity logic. I assume that some raid cards now expose their xor engine and thus the data blocks can be sent to them to handle the xor logic.
That would be interesting.
The patches I noticed were for IBM hardware iirc.
Then it is less interesting >:-)
I think the really big difference between dmraid and mdraid is compatibility. MS Windows only supports the equivalent of dmraid, so if you want to create a dual boot raid system, you need to go that way.
If Windows is not a concern, I think mdraid is the obvious choice but there may be some subtleties I don't know about.
What happens, I understand, is that the bios is capable of reading, and thus booting, from the fakeraid, which means that Windows is capable of booting. When the system is up, it loads the driver and gains write capability. This doesn't happen wit a linux software raid, we have to use tricks that allow grub or lilo to load the kernel first (like having /boot outside of the raid array). A better explanation, here: <http://en.wikipedia.org/wiki/Fakeraid#Firmware.2Fdriver-based_RAID_.28.22FakeRAID.22.29> Firmware/driver-based RAID ("FakeRAID") Operating system-based RAID doesn't always protect the boot process and is generally impractical on desktop versions of Windows (as described above). Hardware RAID controllers are expensive and proprietary. To fill this gap, cheap "RAID controllers" were introduced that do not contain a RAID controller chip, but simply a standard disk controller chip with special firmware and drivers. During early stage bootup the RAID is implemented by the firmware; when a protected-mode operating system kernel such as Linux or a modern version of Microsoft Windows is loaded the drivers take over. These controllers are described by their manufacturers as RAID controllers, and it is rarely made clear to purchasers that the burden of RAID processing is borne by the host computer's central processing unit, not the RAID controller itself, thus introducing the aforementioned CPU overhead from which hardware controllers don't suffer. Firmware controllers often can only use certain types of hard drives in their RAID arrays (e.g. SATA for Intel Matrix RAID, as there is neither SCSI nor PATA support in modern Intel ICH southbridges; however, motherboard makers implement RAID controllers outside of the southbridge on some motherboards). Before their introduction, a "RAID controller" implied that the controller did the processing, and the new type has become known by some as "fake RAID" even though the RAID itself is implemented correctly. Adaptec calls them "HostRAID". - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAktiKysACgkQtTMYHG2NR9XmIQCfSh9yQJ5PHZq9OECspwpNvxou o+4Anih34LI8qMiWKjy3LXsPfDngdhzY =UdZk -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On 01/28/2010 04:17 PM, Greg Freemyer wrote:
On Thu, Jan 28, 2010 at 4:38 PM, Carlos E. R. <robin.listas@telefonica.net> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Thursday, 2010-01-28 at 09:12 +0100, Per Jessen wrote:
FYI, mdraid will do that too, no big deal. Isn't that the whole point? I think the primary difference between the two is that mdraid takes up CPU cycles doing the mirroring, dmraid does not.
Both do, AFAIK. Motherboard, bios raid, is not a real hardware raid, it needs a driver that does the real work in the cpu, with some help from the chipset. That's why it is called "fake raid".
- -- Cheers, Carlos E. R.
I don't know the specifics, but mdraid for sure and possibly dmraid have been getting updates in the last year or so to offload the parity logic. I assume that some raid cards now expose their xor engine and thus the data blocks can be sent to them to handle the xor logic.
The patches I noticed were for IBM hardware iirc.
I think the really big difference between dmraid and mdraid is compatibility. MS Windows only supports the equivalent of dmraid, so if you want to create a dual boot raid system, you need to go that way.
If Windows is not a concern, I think mdraid is the obvious choice but there may be some subtleties I don't know about.
Greg
Right you are Greg, There has been an incredible amount of work done on dmraid in the past 6-7 months. RedHat maintains the code and there have been 4-5 major releases since August. The only pickle was whether you needed to add (or not add) a single 'p' at the end of the device mapper ID. In the dmraid 2.0.x flavor - the 'p' is there :p -- David C. Rankin, J.D.,P.E. Rankin Law Firm, PLLC 510 Ochiltree Street Nacogdoches, Texas 75961 Telephone: (936) 715-9333 Facsimile: (936) 715-9339 www.rankinlawfirm.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wednesday, 2010-01-27 at 23:54 -0600, David C. Rankin wrote:
Happily, there is some magic somewhere in dmraid that makes this work OK. The key is to boot with some other media so that the 2 disks in question are not mounted or used by the booted OS. After fsck'ing the disk in question, mount both disks say under /mnt/a and /mnt/b. Then if 'b' was the drive fsck'ed, do a cp -a /mnt/a/* /mnt/b. Then re-enable the array in the bios on the next boot and all is well.
Run smartctl --test=long /dev/sda, then sdb, etc. After it finishes, fsck the array. Notice that copying one disk to the other, ie, rewriting a disk, is what is done to force the disk to remap sectors when there are uncorrectable or pending errors.
What I want to know is: "Is there any individual disk level error correction performed on disks in dmraid arrays, or is it as it looks -- no fsck error correction is ever performed on individual disks in a dmraid array?"
As we said, fsck should not be used to correct those errors. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAktiBeAACgkQtTMYHG2NR9WShwCfY/yeAcfpw6f+dkrnZ2E83hph 23AAmgNLX/S2K2gPK+tY0OqJi4CJhKd6 =Mj+O -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
David C. Rankin wrote:
Simply put, if small disk errors go uncorrected in the dmraid array and are allowed to remain uncorrected and multiply, then dmraid has a gaping hole in its robustness making it look far inferior to software raid.
Surely you can still use SMART to monitor the drives, right?
Does anybody know more or have any links to detailed information on how bad blocks are handled in dmraid??
It's done automagically by the IDE drive.
All of this discussion is prompted by 2 recent supposed disk 'failures' in dmraid setups where one disk will be pristine, but the other will have hundreds of simply corrected errors that have never been fixed over the life of the array, that after disabling the array and running e2fsck are fully correctable.
If I understand this right, you have a two-disk RAID1 array run by a hardware/on-board controller that you access via dmraid. Unless something is wrong with the mirroring, I don't see how one disk can have file-system errors that the other does not.
In all the dmraid documentation I have read, I have never seen a warning or note that says:
"Periodic disabling of the dmraid array will be required to allow fsck or other disk maintenance processes to be run on the individual disks of the array because enabling dmraid in the BIOS prevents disk maintenance on individual disks in the array."
I'm concerned that something like that is needed. The question is, "Is that true?"
Unless your RAID setup also prevents SMART monitoring, no. /Per -- Per Jessen, Zürich (-0.8°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On 01/24/2010 03:08 AM, Per Jessen wrote:
David C. Rankin wrote:
<snip>
If I understand this right, you have a two-disk RAID1 array run by a hardware/on-board controller that you access via dmraid. Unless something is wrong with the mirroring, I don't see how one disk can have file-system errors that the other does not.
<snip>
/Per
Per, Thank You!!! That is the question I needed to be asking all along. How can one drive in the array have filesystem problems that are easily correctable by fsck while the other disk is perfect with no errors at all?? And further, would these errors be capable of being corrected by fsck in a mdraid array but not correctable in a dmraid array? Whew.... I knew we would finally get down to a way to ask the right question. Carlos, Anders, Per, all -- anybody got an answer to that one? -- David C. Rankin, J.D.,P.E. Rankin Law Firm, PLLC 510 Ochiltree Street Nacogdoches, Texas 75961 Telephone: (936) 715-9333 Facsimile: (936) 715-9339 www.rankinlawfirm.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
David C. Rankin wrote:
On 01/24/2010 03:08 AM, Per Jessen wrote:
David C. Rankin wrote:
<snip>
If I understand this right, you have a two-disk RAID1 array run by a hardware/on-board controller that you access via dmraid. Unless something is wrong with the mirroring, I don't see how one disk can have file-system errors that the other does not.
<snip>
/Per
Per,
Thank You!!! That is the question I needed to be asking all along. How can one drive in the array have filesystem problems that are easily correctable by fsck while the other disk is perfect with no errors at all??
The only explanation is that they are _not_ mirror copies of each other. Does your RAID1 hardware have any utilities for diagnostics? Or just an indication of the state of mirror?
And further, would these errors be capable of being corrected by fsck in a mdraid array but not correctable in a dmraid array?
Well, for either one, the way to "correct" the problem is to remove one drive, leaving the one with the copy you believe to be correct. Then add the drive back, and wait for the mirroring to catch up. If you have a drive hardware issue, the mirror will not work, and you'll need to replace the faulty drive. /Per -- Per Jessen, Zürich (-2.1°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thursday, 2010-01-28 at 08:56 +0100, Per Jessen wrote:
Well, for either one, the way to "correct" the problem is to remove one drive, leaving the one with the copy you believe to be correct. Then add the drive back, and wait for the mirroring to catch up. If you have a drive hardware issue, the mirror will not work, and you'll need to replace the faulty drive.
Right. And this operation, as it rewrites one of the disks, will remap all pending sectors. Probably best done after doing a surface scan using smart long test. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAktiBt4ACgkQtTMYHG2NR9UznQCgk1t5SSfnePgWy9l4eruMIoDY vaAAoJCQ9Ni8wsi6sTcfOqZPjdcpDoX7 =y/PV -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
participants (5)
-
Anders Johansson -
Carlos E. R. -
David C. Rankin -
Greg Freemyer -
Per Jessen