Re: [opensuse] Large Raids with lots of disks [Was: Reiserfs unstable under load using large LV]

On Thursday 07 June 2007 16:08:54 Greg Freemyer wrote:

On 6/6/07, Matthew Stringer wrote:

...

Hi,

I've a server with 24 500GB SATA 2 hard drives with RAID5 running 10.2.

*** Warning, the hp trained storage engineer part of me is coming out.****

A 24 disk raid5 is not normally recommended. It is much more likely to fail than a raid50 where you are striping across several smaller raid5's.

Even more reliable is raid6. Personally I like raid10 for most situations. raid10 is the most expensive, but the most reliable one highest performing.

As an example, assume you have a raid50 made of 6 4-disk raid5 sets striped together. Assume the chance of a first disk failure in a year is X. Assume the chance of a second disk failure during the window of vulnerability is Y.

FYI: The window of vulnerability is however long it takes to replace a failed drive and rebuild the array. On big arrays, just the rebuild time can be a couple days. And if the array is both very large and in extremely heavy use 24/7 it can be a couple weeks.

Back to the example: The overall failure likely hood is [(4 * X) * ((4-1) * Y)] * 6 or 72 * X * Y.

Hopefully you can figure out where the numbers came from.

Now for a single 24 disk array the failure likelihood is: (24 * X) * ((24 -1) * Y) or 552 * X * Y.

That is the big raid5 array is roughly 7.5 times more likely to have a total failure than a raid50 with the same number of drives would be.

(Admittedly the raid50 holds only 75% of raw disk capacity and the raid5 is going to hold a little more than 95%.)

Given the price of drives, you might want to consider replacing all of those 500GB drives with 1000GB drives and doing raid10. (about $500 each I think).

Then you have the same capacity and the failure likelyhood would be: 24 * X * 1 * Y (ie any of the 24 can initially fail, but only its partner failing causes a data loss).

So your online data would be about 20 times safer for $12K. If I had that much data, I would seriously consider what raid strategy I would use, but I really doubt I would end up choosing a 24-disk raid5.

Greg -- Greg Freemyer The Norcross Group Forensics for the 21st Century

All you say is true however, budget dictates, I needed >10TB of storage for £6K. You're suggesting spending another £12K on new drives alone! If I'd bought larger disks and used RAID50 I'd have half the disk I/O too which isn't desirable. In my case the data isn't that critical hence the solution I put in place, which incidentally with OpenSUSE 10.2 and XFS has been spot on! Matthew -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org