-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Friday, 2010-01-22 at 17:23 +0100, Anders Johansson wrote: ...

As far as I know, fsck never does badblock analysis

As a matter of fact, it does. Kind of. For instance, for ext3 it calls e2fsck, and this one has options for badblock handling. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAktZ2f0ACgkQtTMYHG2NR9WtHgCeJZ3vt/R9CkZknlY8/xC3u2XN Bg8AnRkIDDRybk7S1etoq3CAFmacoqZp =Z/0W -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Saturday, 2010-01-23 at 13:20 -0600, David C. Rankin wrote:

On 01/22/2010 11:01 AM, Carlos E. R. wrote:

...

On Friday, 2010-01-22 at 17:23 +0100, Anders Johansson wrote:

...

...

As far as I know, fsck never does badblock analysis

As a matter of fact, it does. Kind of. For instance, for ext3 it calls e2fsck, and this one has options for badblock handling.

There in lies an Achilles' of dmraid that I am interested in. If there isn't a specific kernel level workaround to temporarily disable a dmraid array to check and correct any normally easily correctable 'disk' errors such as bad blocks, etc.., then that means a dmraid setup will suppress/prevent the correction of disk errors on each 'disk' in the array allowing simple correctable errors to propagate or grow into multiple compounded errors resulting in disk deterioration to the point of data loss.

Whoa! Hold on. The posibility of using fsck to mark badblocks is _NOT_ used on contemporary hard disks. Period :-) Badblocks are left to be managed by the hard disk firmware internally and transparently, when attemting to write to a known (by the HD) bad block.

Simply put, if small disk errors go uncorrected in the dmraid array and are allowed to remain uncorrected and multiply, then dmraid has a gaping hole in its robustness making it look far inferior to software raid.

I can't believe that it the way dmraid works. In the past, I have actually preferred it to software raid because I can rebuild a new disk following a drive failure and remake the array before I ever have to boot the operating system. But, if you can never e2fsck -fcy /dev/sda, sdb, etc.. without first disabling the array in the bios, you are basically playing Russian roulette with disk errors on any single disk in the array.

If you do that, you will corrupt your filesystem, and not be able to reenable the array. Both images will be different, and, I guess, when the array is reenabled the newer copy will simply be overwritten to the older copy, including the badblocks; ie, bad blocks in side 0 will be copied and marked bad to side 1, even if there are none there. And perhaps, good blocks on 0 will overwrite bad blocks on 1. Notice that fsck bad blocks works on the filesystem level, it does never see raid elements, which is correct. The raid, any type of raid, presents a unique device to the filesystem layer. If you disable the array and run fsck on the elements, fsck will work on the apparent filesystem, both sides will be different as a result, and the array will be corrupted in the end. As best, the newer side will overwrite the older side. Do not attempt to correct badblocks from the operating system on an array; leave that to the disk firmware. Just use smartctl to trigger the long test (on each element), which includes a surface scan. If there are uncorrected sectors after that, the procedure would be to rewrite the affected sectors (or the entire disk). - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAktbVNwACgkQtTMYHG2NR9UHRwCcDDp2+4EkNQs2ceUgs6pHpY4O 2+oAnirvL/sfAYI0xhjexYbi1rbkblcV =mJM4 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sunday, 2010-01-24 at 10:01 +0100, Per Jessen wrote:

Carlos E. R. wrote:

...

Whoa! Hold on.

The posibility of using fsck to mark badblocks is _NOT_ used on contemporary hard disks.

Yep. You have to go pretty far back for that to be a realistic option. Pre-IDE days, I would say.

Maybe not so far back. Only since SMART appeared, and did remapping in hardware. Remember that the msdos FAT checkdisk utility did this remapping, on the filesystem, by default, since day one: it was an absolute necessity for floppies, and even hard disks. The first hard disk I owned (32MB) came with a label listing bad blocks. However, ext3 or reiser fsck did not, till a few years back: this feature was added relatively recently. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAktcS70ACgkQtTMYHG2NR9XZKACbBvLiYQnekRk2PhSJKfcwZbGO 6WsAnR+MIO/tKrp5uNe24HOv150zSAq7 =vNq9 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Per Jessen wrote:

Carlos E. R. wrote:

...

On Sunday, 2010-01-24 at 10:01 +0100, Per Jessen wrote:

...

Carlos E. R. wrote:

...

...

Whoa! Hold on.

The posibility of using fsck to mark badblocks is _NOT_ used on contemporary hard disks.

Yep. You have to go pretty far back for that to be a realistic option. Pre-IDE days, I would say.

Maybe not so far back. Only since SMART appeared, and did remapping in hardware.

IDE (PATA) drives have always done remapping of bad blocks themselves. SMART is just the monitoring part of that.

...

Remember that the msdos FAT checkdisk utility did this remapping, on the filesystem, by default, since day one: it was an absolute necessity for floppies, and even hard disks. The first hard disk I owned (32MB) came with a label listing bad blocks.

Presumably, none of those were IDE/ATA drives?

Just clarify - when I said pre-IDE, I meant disks with ST-506 or ESDI interfaces, where the actual disk controlling logic sat on a disk controller, as opposed to the ATA/IDE drives with integrated electronics (IDE). /Per -- Per Jessen, Zürich (-0.8°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

Carlos E. R. wrote:

On Sunday, 2010-01-24 at 15:12 +0100, Per Jessen wrote:

...

Per Jessen wrote:

...

...

...

...

Yep. You have to go pretty far back for that to be a realistic option. Pre-IDE days, I would say.

Maybe not so far back. Only since SMART appeared, and did remapping in hardware.

IDE (PATA) drives have always done remapping of bad blocks themselves. SMART is just the monitoring part of that.

I'm not sure of that. I only learnt of that feature after the turn of the century.

Okay, maybe not always, but after harddrives switched to GMR (thanks Jülich and IBM), they definitely did the bad block remapping themselves. Maybe early 90s, I'm not sure. SMART dates back to that time too. /Per -- Per Jessen, Zürich (-0.8°C) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

On 01/23/2010 01:58 PM, Carlos E. R. wrote:

...

...

I can't believe that it the way dmraid works. In the past, I have actually preferred it to software raid because I can rebuild a new disk following a drive failure and remake the array before I ever have to boot the operating system. But, if you can never e2fsck -fcy /dev/sda, sdb, etc.. without first disabling the array in the bios, you are basically playing Russian roulette with disk errors on any single disk in the array. If you do that, you will corrupt your filesystem, and not be able to reenable the array. Both images will be different, and, I guess, when the array is reenabled the newer copy will simply be overwritten to the older copy, including the badblocks; ie, bad blocks in side 0 will be copied and marked bad to side 1, even if there are none there. And perhaps, good blocks on 0 will overwrite bad blocks on 1.

Happily, there is some magic somewhere in dmraid that makes this work OK. The key is to boot with some other media so that the 2 disks in question are not mounted or used by the booted OS. After fsck'ing the disk in question, mount both disks say under /mnt/a and /mnt/b. Then if 'b' was the drive fsck'ed, do a cp -a /mnt/a/* /mnt/b. Then re-enable the array in the bios on the next boot and all is well. As for the 'bad block' terminology, I may be off. What I'm talking about having to fsck to cure that isn't happening in dmraid is the offline uncorrectable errors: 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 207 I had been adding the bad block scan to fsck, perhaps wrongly so, but to force a surface scan. The error above was the error that prompted the response to fsck the disk with the bad block check enabled. What ever errors fsck finds and fixes with the -fcy option are the errors that are NOT being fixed when the disks are configured in a dmraid array. That's what was the genesis of the post and concern about the robustness of dmraid over time. The question is still pending though. How are (whatever these errors are) handled by suse when a disk is part of an dmraid array? You cannot fsck the disk when it is part of the array because fsck exits and throws the error that the disk is 'under the exclusive control of another process' and refuses to test the disk. My experience here is that the disk is FINE and just needs a simple fsck and so far that has been true in two out of three dmraid arrays I have had errors on during the past year. (in one case, the disk was cratering). What I want to know is: "Is there any individual disk level error correction performed on disks in dmraid arrays, or is it as it looks -- no fsck error correction is ever performed on individual disks in a dmraid array?" The ultimate question is should I ditch dmraid entirely and go with mdraid? dmraid has been absolutely bullet-proof as has been my mdraid installs. But, if mdraid can handle single disk error correction within an array where dmraid can't, that would be enough justification for me to dump dmraid in favor of mdraid. dmraid and mdraid are incredibly similar in the flexibility they offer as a raid solution. In either case you have mirrored copies, where upon failure, or just for kicks, you can rip one disk out of the array and boot and run on a single disk, or put it in another box without any concern about raid incompatibility. Both just use normal partitions and disks, the only primary difference is in how they are joined, either through a bios function or through a software function. I have liked dmraid because I can easily mirror /, /boot, /home and swap on each disk so that in the event of disk failure, the most work one disk can ever need to act as a standalone is to reinstall grub in the mbr if the boot loader code just happened to be other disk. (small potatoes) But, if there is a fundamental difference in the single disk maintenance/error correction area, that would make a big difference in the robustness of one versus the other. I just don't know the answer to that question and after extensive googling it doesn't seem like anyone else does either (or they just haven't discussed it) Thus my post -- Anybody know the answer?? Anders, are you a raid guy too?? (P.S. -- you know it is a good problem when the question is hard to frame :-) -- David C. Rankin, J.D.,P.E. Rankin Law Firm, PLLC 510 Ochiltree Street Nacogdoches, Texas 75961 Telephone: (936) 715-9333 Facsimile: (936) 715-9339 www.rankinlawfirm.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thursday, 2010-01-28 at 09:12 +0100, Per Jessen wrote:

FYI, mdraid will do that too, no big deal. Isn't that the whole point? I think the primary difference between the two is that mdraid takes up CPU cycles doing the mirroring, dmraid does not.

Both do, AFAIK. Motherboard, bios raid, is not a real hardware raid, it needs a driver that does the real work in the cpu, with some help from the chipset. That's why it is called "fake raid". - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAktiA/QACgkQtTMYHG2NR9V9AwCfeSbwJnci7/ZhXDummTyLoaqf ih4AnAqR0yMdo5gtNz0oMwE9Ze7JCYdU =NYoP -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thursday, 2010-01-28 at 17:17 -0500, Greg Freemyer wrote:

On Thu, Jan 28, 2010 at 4:38 PM, Carlos E. R. <> wrote:

...

Both do, AFAIK. Motherboard, bios raid, is not a real hardware raid, it needs a driver that does the real work in the cpu, with some help from the chipset. That's why it is called "fake raid".

I don't know the specifics, but mdraid for sure and possibly dmraid have been getting updates in the last year or so to offload the parity logic. I assume that some raid cards now expose their xor engine and thus the data blocks can be sent to them to handle the xor logic.

That would be interesting.

The patches I noticed were for IBM hardware iirc.

Then it is less interesting >:-)

I think the really big difference between dmraid and mdraid is compatibility. MS Windows only supports the equivalent of dmraid, so if you want to create a dual boot raid system, you need to go that way.

If Windows is not a concern, I think mdraid is the obvious choice but there may be some subtleties I don't know about.

What happens, I understand, is that the bios is capable of reading, and thus booting, from the fakeraid, which means that Windows is capable of booting. When the system is up, it loads the driver and gains write capability. This doesn't happen wit a linux software raid, we have to use tricks that allow grub or lilo to load the kernel first (like having /boot outside of the raid array). A better explanation, here: http://en.wikipedia.org/wiki/Fakeraid#Firmware.2Fdriver-based_RAID_.28.22Fak... Firmware/driver-based RAID ("FakeRAID") Operating system-based RAID doesn't always protect the boot process and is generally impractical on desktop versions of Windows (as described above). Hardware RAID controllers are expensive and proprietary. To fill this gap, cheap "RAID controllers" were introduced that do not contain a RAID controller chip, but simply a standard disk controller chip with special firmware and drivers. During early stage bootup the RAID is implemented by the firmware; when a protected-mode operating system kernel such as Linux or a modern version of Microsoft Windows is loaded the drivers take over. These controllers are described by their manufacturers as RAID controllers, and it is rarely made clear to purchasers that the burden of RAID processing is borne by the host computer's central processing unit, not the RAID controller itself, thus introducing the aforementioned CPU overhead from which hardware controllers don't suffer. Firmware controllers often can only use certain types of hard drives in their RAID arrays (e.g. SATA for Intel Matrix RAID, as there is neither SCSI nor PATA support in modern Intel ICH southbridges; however, motherboard makers implement RAID controllers outside of the southbridge on some motherboards). Before their introduction, a "RAID controller" implied that the controller did the processing, and the new type has become known by some as "fake RAID" even though the RAID itself is implemented correctly. Adaptec calls them "HostRAID". - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAktiKysACgkQtTMYHG2NR9XmIQCfSh9yQJ5PHZq9OECspwpNvxou o+4Anih34LI8qMiWKjy3LXsPfDngdhzY =UdZk -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wednesday, 2010-01-27 at 23:54 -0600, David C. Rankin wrote:

Happily, there is some magic somewhere in dmraid that makes this work OK. The key is to boot with some other media so that the 2 disks in question are not mounted or used by the booted OS. After fsck'ing the disk in question, mount both disks say under /mnt/a and /mnt/b. Then if 'b' was the drive fsck'ed, do a cp -a /mnt/a/* /mnt/b. Then re-enable the array in the bios on the next boot and all is well.

Run smartctl --test=long /dev/sda, then sdb, etc. After it finishes, fsck the array. Notice that copying one disk to the other, ie, rewriting a disk, is what is done to force the disk to remap sectors when there are uncorrectable or pending errors.

What I want to know is: "Is there any individual disk level error correction performed on disks in dmraid arrays, or is it as it looks -- no fsck error correction is ever performed on individual disks in a dmraid array?"

As we said, fsck should not be used to correct those errors. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAktiBeAACgkQtTMYHG2NR9WShwCfY/yeAcfpw6f+dkrnZ2E83hph 23AAmgNLX/S2K2gPK+tY0OqJi4CJhKd6 =Mj+O -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

On 01/24/2010 03:08 AM, Per Jessen wrote:

David C. Rankin wrote:

<snip>

If I understand this right, you have a two-disk RAID1 array run by a hardware/on-board controller that you access via dmraid. Unless something is wrong with the mirroring, I don't see how one disk can have file-system errors that the other does not.

<snip>

/Per

Per, Thank You!!! That is the question I needed to be asking all along. How can one drive in the array have filesystem problems that are easily correctable by fsck while the other disk is perfect with no errors at all?? And further, would these errors be capable of being corrected by fsck in a mdraid array but not correctable in a dmraid array? Whew.... I knew we would finally get down to a way to ask the right question. Carlos, Anders, Per, all -- anybody got an answer to that one? -- David C. Rankin, J.D.,P.E. Rankin Law Firm, PLLC 510 Ochiltree Street Nacogdoches, Texas 75961 Telephone: (936) 715-9333 Facsimile: (936) 715-9339 www.rankinlawfirm.com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thursday, 2010-01-28 at 08:56 +0100, Per Jessen wrote:

Well, for either one, the way to "correct" the problem is to remove one drive, leaving the one with the copy you believe to be correct. Then add the drive back, and wait for the mirroring to catch up. If you have a drive hardware issue, the mirror will not work, and you'll need to replace the faulty drive.

Right. And this operation, as it rewrites one of the disks, will remap all pending sectors. Probably best done after doing a surface scan using smart long test. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAktiBt4ACgkQtTMYHG2NR9UznQCgk1t5SSfnePgWy9l4eruMIoDY vaAAoJCQ9Ni8wsi6sTcfOqZPjdcpDoX7 =y/PV -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org