On Thursday 09 November 2006 05:22, Arie Reynaldi Z wrote:

...

Software RAID level 1 is slower than 1 harddisk? Not true. Writing and Reading speed is good. But, it's the most expensive solution in terms of money per MB. Software RAID level 5 is slower than 1 harddisk? Partly true. Writing performance is a bit slower since it must calculate the parity. Reading speed is good.

I googled and found this: http://www.catux.org/index.php?contingut=articles&num=101

It shows that using software RAID 1 is slower then others.. Can someone explain about this ? It used Bonnie++ just like Joachim said.

A mirror uses two drives. Data has to be written to both drives. File systems/buffers help hide this to some extent, but in the end twice as much work must be done, so it is reasonable to expect it to be slower for writes. But, as the charts on that web page show, the mirror is not twice as slow. So, a software Raid mirror is a good compromise for reliability and performance without spending big money on enterprise class hardware raid.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Thursday 2006-11-09 at 12:01 +0700, Fajar Priyanto wrote:

Software RAID level 5 is slower than 1 harddisk? Partly true. Writing performance is a bit slower since it must calculate the parity. Reading speed is good.

But write operations are distributed over the three disks and parallelized, so it should be faster. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFFUydatTMYHG2NR9URAnRDAKCFaROwLTY9CBJPvlVw7v6g/130GQCeMG/e Oo1A4NcIQ+Zvqc3SRC4PBnU= =Q4Dc -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The Thursday 2006-11-09 at 08:30 -0600, Stan Glasoe wrote:

Not. RAID5 has to first write the data, then it has to calculate the parity, then it writes the parity. It can't do all three in parallel. It has to actually write the data portion so it has the necessary information to calculate the parity and then write the parity.

I don't think that is correct. So says the HOWTO: Both read and write performance usually increase, but can be hard to predict how much. Reads are similar to RAID-0 reads, writes can be either rather expensive (requiring read-in prior to write, in order to be able to calculate the correct parity information), or similar to RAID-1 writes. The write efficiency depends heavily on the amount of memory in the machine, and the usage pattern of the array. Heavily scattered writes are bound to be more expensive. - -- Cheers, Carlos E. R. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) Comment: Made with pgp4pine 1.76 iD8DBQFFU07TtTMYHG2NR9URAkgQAJ0aHAfY7FMG9O/vlKyXc8dxMPGwtACdG0/+ tq/yMD1vN4V+YHlZ7wE9cIA= =Iflu -----END PGP SIGNATURE-----

On 11/9/06, Stan Glasoe wrote:

On Thursday November 9 2006 9:52 am, Carlos E. R. wrote:

...

The Thursday 2006-11-09 at 08:30 -0600, Stan Glasoe wrote:

...

Not. RAID5 has to first write the data, then it has to calculate the parity, then it writes the parity. It can't do all three in parallel. It has to actually write the data portion so it has the necessary information to calculate the parity and then write the parity.

I don't think that is correct.

So says the HOWTO:

Both read and write performance usually increase, but can be hard to predict how much. Reads are similar to RAID-0 reads, writes can be either rather expensive (requiring read-in prior to write, in order to be able to calculate the correct parity information), or similar to RAID-1 writes. The write efficiency depends heavily on the amount of memory in the machine, and the usage pattern of the array. Heavily scattered writes are bound to be more expensive.

Carlos E. R.

The How To in this case gives incomplete information. If there is a write at all on RAID5 then it isn't similar to RAID1. All the calculations may be done in system or controller card memory but as far as benchmark software timing is concerned RAID5 still has to make 2 writes versus RAID1 making 1 write before that I/O is considered done.

RAID1 can write to one disk and then when time and work load permits, mirror that write to the other drive. Both drives typically aren't written to at the same time. So once the data is committed to either disk the benchmark doesn't track the other I/O but may be affected since that background I/O may delay a read or write that is next in queue. That background write probably doesn't interfere with the next I/O since if that drive is busy then the other drive most likely isn't and the next I/O the benchmark is counting goes to the available drive; not always but often enough.

RAID5 writes have to calculate and write the changed parity. That adds I/O delay and it is always more than RAID1. A RAID5 write has to write a data chunk, read the other data chunk, calculate parity and then write the parity chunk. Now the I/O is committed to disk and the benchmark can call it complete. 2 writes are involved; changed data and changed parity.

Hence my opinion that RAID5 writes always suck in the performance area. They do, they can't help it. Depending on the system, is that performance hit of any concern versus the survivability of the data?

Stan

The calculation is not normally done that way. To compare different RAID efficiencies the standard process is to calculate or compare the number of logical writes/second the array will sustain under heavy load and a well utilized cache to minimize wasted work. I don't know the details of how linux implements it but conceptually for the worse case with raid 5 (ie. scattered writes) to update a single block you have to: Read original block (even if your updating it 100%) Read Parity Block Back-out the original block from the parity calculation (Simple XOR logic) Put the new blocks data into the parity calculation (Simple XOR logic) Write the new data block Write the new parity block So it takes 4 i/o's to do a single logical write assuming scattered disk writes. The good news is you get to spread those i/o across all your disks. A Raid 5 made of 4 truly independent disks (ie. no master/slave IDE conflicts, etc.) can do 4 i/o's simultaneously. So a busy 4-disk Raid5 can handle the same number of logical writes per second as a single disk. On the read side it would be 4 times faster. (A 3-disk raid set can only handle 3/4 as many logical writes per second, but 3x as many reads/second.) For the best case with RAID-5 (full stripe updates) the OS should determine that the entire stripe set is being updated so it does not care about any of the prior data and is able to skip the reads and only has to write the parity block once. So for a 4-disk raid5 set it takes 4 i/o's to handle 3 logical block writes. Since all 4 of those i/o's could happen simultaneously a 4-disk Raid 5 can handle 3x the logical writes/second that a single disk could. FYI: RAID 10 should be even better/faster. Primarily because you would need 6 disks to hold the capacity of data as a 4-disk Raid5. And with the extra disks you can do more physical i/o's per second. Again, all of the above assumes your disk activity is trully independent. You don't typically get that with IDE bus based drives. SCSI is typically independent and I don't know about the SATA controllers. I haven't seen any simultaneous disk activity benchmarks yet for them. Greg -- Greg Freemyer The Norcross Group Forensics for the 21st Century

On 11/9/06, Carlos E. R. wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

The Thursday 2006-11-09 at 15:54 -0500, Greg Freemyer wrote:

...

...

I don't know the details of how linux implements it but conceptually for the worse case with raid 5 (ie. scattered writes) to update a single block you have to:

Read original block (even if your updating it 100%) Read Parity Block Back-out the original block from the parity calculation (Simple XOR logic) Put the new blocks data into the parity calculation (Simple XOR logic) Write the new data block Write the new parity block

So it takes 4 i/o's to do a single logical write assuming scattered disk writes.

Question.

The data block, is it written to one disk, or is it split over two disks?

Again I don't know how Linux does it, but I just did some detail reconstruction work on a 3-disk hardware raid5 using a dell perc controller. IIRC, the blocks were laid out like: 1 2 p 3 p 4 p 5 6 7 8 p Each block was 64K (ie. 128 sectors), and full stripe was 2x64K or 128K. So block 1 is on the first disk in the array. block 2 is on the second disk in the array, and the first parity block is on the third disk in the array. Block 3 is back on the first disk, and so on and so on. Now if only a 4K page is being updated, I don't know if just a single 4K page is read from the data block and parity blocks and then updates done. Or if the logic only works with full 64K blocks at a time. FYI, I was working at this low level because the hardware had failed but I was able to extract fairly good dd images of the drives and I used a low-cost windows program to rebuild the discrete dd images into a single logical dd image. The tool was "Raid Reconstructor" and I think is was less than $100. Greg -- Greg Freemyer The Norcross Group Forensics for the 21st Century

Greg Freemyer wrote:

On 11/9/06, Stan Glasoe wrote:

...

On Thursday November 9 2006 9:52 am, Carlos E. R. wrote:

...

The Thursday 2006-11-09 at 08:30 -0600, Stan Glasoe wrote:

...

Not. RAID5 has to first write the data, then it has to calculate the parity, then it writes the parity. It can't do all three in parallel. It has to actually write the data portion so it has the necessary information to calculate the parity and then write the parity.

I don't think that is correct.

So says the HOWTO:

Both read and write performance usually increase, but can be hard to predict how much. Reads are similar to RAID-0 reads, writes can be either rather expensive (requiring read-in prior to write, in order to be able to calculate the correct parity information), or similar to RAID-1 writes. The write efficiency depends heavily on the amount of memory in the machine, and the usage pattern of the array. Heavily scattered writes are bound to be more expensive.

Carlos E. R.

The How To in this case gives incomplete information. If there is a write at all on RAID5 then it isn't similar to RAID1. All the calculations may be done in system or controller card memory but as far as benchmark software timing is concerned RAID5 still has to make 2 writes versus RAID1 making 1 write before that I/O is considered done.

RAID1 can write to one disk and then when time and work load permits, mirror that write to the other drive. Both drives typically aren't written to at the same time. So once the data is committed to either disk the benchmark doesn't track the other I/O but may be affected since that background I/O may delay a read or write that is next in queue. That background write probably doesn't interfere with the next I/O since if that drive is busy then the other drive most likely isn't and the next I/O the benchmark is counting goes to the available drive; not always but often enough.

RAID5 writes have to calculate and write the changed parity. That adds I/O delay and it is always more than RAID1. A RAID5 write has to write a data chunk, read the other data chunk, calculate parity and then write the parity chunk. Now the I/O is committed to disk and the benchmark can call it complete. 2 writes are involved; changed data and changed parity.

Hence my opinion that RAID5 writes always suck in the performance area. They do, they can't help it. Depending on the system, is that performance hit of any concern versus the survivability of the data?

Stan

The calculation is not normally done that way. To compare different RAID efficiencies the standard process is to calculate or compare the number of logical writes/second the array will sustain under heavy load and a well utilized cache to minimize wasted work.

IMHO, this should not be stated in absolute terms. E.g., on typical database servers, one uses almost always unbuffered (synchronous) writes. Only the DBMS caches and the filesystem cache is turned off. Because there, reliability (the D in ACID) is much more important than any performance improvement. Cache is a not a factor in disk benchmarking then, the beef is in fast synchronous writes when one selects a disk subsystem. The DBMS cache efficiency will be the same for all RAID variants, the log tablespace performance limit the overall write efficiency. This is one of the reasons why one does not use plain IDE or SATA disks without battery buffers for such systems, but real SCSI disks where one can control such capabilities in detail. E.g., one typically selects fs sync flags differently for the temp and the log tablespaces. (Btw, I prefer EMC storage subsystems for their reliability and their feature set. This is not a plea for Linux md.) In real storage selection scenarios, performance benchmarking depends highly on the application that this server is used on. I do this for a living, and at the situations where I'm involved (typically mission-critical servers) RAID-5 is more and more uninteresting. Current storage prices make RAID-1 practical for TB-sized storage systems; only at the PB-level we need different scenarios (HSMs, actually). Joachim -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Joachim Schrod Email: jschrod@acm.org Roedermark, Germany