[opensuse] SATA RAID recommendations?
I'm looking at building a SATA II RAID-5 system, and I'd like to get some thoughts on controllers since there are a multitude out there.. I'm going for either 4 or 8 ports. The rig should be run by a 10.1 x86_64 system with dual opterons. The manufacturers I'm looking at are: Adaptec Highpoint 3Ware Promise Can someone with knowledge about these tell me the pros and cons with each of them? -- Anders Norrbring Norrbring Consulting
On Sunday 04 March 2007, Anders Norrbring wrote:
I'm looking at building a SATA II RAID-5 system, and I'd like to get some thoughts on controllers since there are a multitude out there.. I'm going for either 4 or 8 ports. The rig should be run by a 10.1 x86_64 system with dual opterons.
The manufacturers I'm looking at are:
Adaptec Highpoint 3Ware Promise
Can someone with knowledge about these tell me the pros and cons with each of them?
Why use any of these. Software raid is robust, drop dead simple to set up with SUSE and usually outperforms any of the raid controllers due to use of your far more powerfull CPU. The CPU requirements are minimal, usually less than 1% according to top and performance is great. I've been using software raid in production machines since Suse 7.1 and its rock solid. I even have raid controllers on which I have turned off the raid, and simply use the card's independent controller channels. Performance for me on these older CMD cards was better with the controller turned off. YMMV. -- _____________________________________ John Andersen -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
John Andersen skrev:
On Sunday 04 March 2007, Anders Norrbring wrote:
I'm looking at building a SATA II RAID-5 system, and I'd like to get some thoughts on controllers since there are a multitude out there.. I'm going for either 4 or 8 ports. The rig should be run by a 10.1 x86_64 system with dual opterons.
The manufacturers I'm looking at are:
Adaptec Highpoint 3Ware Promise
Can someone with knowledge about these tell me the pros and cons with each of them?
Why use any of these. Software raid is robust, drop dead simple to set up with SUSE and usually outperforms any of the raid controllers due to use of your far more powerfull CPU.
The CPU requirements are minimal, usually less than 1% according to top and performance is great. I've been using software raid in production machines since Suse 7.1 and its rock solid.
I even have raid controllers on which I have turned off the raid, and simply use the card's independent controller channels. Performance for me on these older CMD cards was better with the controller turned off. YMMV.
You definately have a point there John.. I guess I'm a bit too used to running SCSI rigs on Adaptec cards to even consider software RAID.. ;) I'll look around for some SATA-II expansion cards to take a look at.. I do prefer to run PCI-X or 64bit since I have lots of those on board, but no SATA connections. -- Anders Norrbring Norrbring Consulting
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Anders Norrbring wrote: ...
The manufacturers I'm looking at are: Adaptec Highpoint 3Ware Promise
Can someone with knowledge about these tell me the pros and cons with each of them?
Definitely stay away from Promise, they only make pseudo-hardware-RAID
devices where the (windows) driver does 90% of the work.
You'd be much better off using the kernel's software RAID instead.
3ware make some good, real hardware RAID controllers, but they're pretty
expensive. And still, software RAID works great on Linux.
Anyhow, stay away from any RAID controller that is cheaper than 1000 EUR
because it's crap. And yet, software RAID on Linux outperforms most of them.
Umm.. does that answer your question ? ;)
cheers
- --
-o) Pascal Bleser http://linux01.gwdg.de/~pbleser/
/\\
Pascal Bleser skrev:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Anders Norrbring wrote: ...
The manufacturers I'm looking at are: Adaptec Highpoint 3Ware Promise
Can someone with knowledge about these tell me the pros and cons with each of them?
Definitely stay away from Promise, they only make pseudo-hardware-RAID devices where the (windows) driver does 90% of the work. You'd be much better off using the kernel's software RAID instead.
3ware make some good, real hardware RAID controllers, but they're pretty expensive. And still, software RAID works great on Linux.
Anyhow, stay away from any RAID controller that is cheaper than 1000 EUR because it's crap. And yet, software RAID on Linux outperforms most of them.
Umm.. does that answer your question ? ;)
Yep, it does.. This and John's reply.. Actually, I was looking more to Adaptec than to 3Ware, but then I only have experience with LSI Logic and Adaptec before, and only on SCSI. -- Anders Norrbring Norrbring Consulting
On Sun, March 4, 2007 22:35, Pascal Bleser wrote:
Anyhow, stay away from any RAID controller that is cheaper than 1000 EUR because it's crap. And yet, software RAID on Linux outperforms most of them.
1000 EUR? http://froogle.google.de/froogle?q=adaptec+2820SA&btnG=Froogle-Suche And this card is definitely not crap. Hans -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Hans du Plooy skrev:
On Sun, March 4, 2007 22:35, Pascal Bleser wrote:
Anyhow, stay away from any RAID controller that is cheaper than 1000 EUR because it's crap. And yet, software RAID on Linux outperforms most of them.
1000 EUR?
http://froogle.google.de/froogle?q=adaptec+2820SA&btnG=Froogle-Suche
And this card is definitely not crap.
Hans
Right.. I'm actually looking at this http://www.adaptec.com/en-GB/products/sas/raid/SAS-4800/ -- Anders Norrbring Norrbring Consulting
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hans du Plooy wrote:
On Sun, March 4, 2007 22:35, Pascal Bleser wrote:
Anyhow, stay away from any RAID controller that is cheaper than 1000 EUR because it's crap. And yet, software RAID on Linux outperforms most of them.
1000 EUR? http://froogle.google.de/froogle?q=adaptec+2820SA&btnG=Froogle-Suche And this card is definitely not crap.
Well, yeah, 300-400 EUR would have been a better limit.
Actually: stay away from pseudo-hardware RAID controllers like Promise.
cheers
- --
-o) Pascal Bleser http://linux01.gwdg.de/~pbleser/
/\\
Hans du Plooy skrev:
On Sun, March 4, 2007 22:35, Pascal Bleser wrote:
Anyhow, stay away from any RAID controller that is cheaper than 1000 EUR because it's crap. And yet, software RAID on Linux outperforms most of them.
1000 EUR?
http://froogle.google.de/froogle?q=adaptec+2820SA&btnG=Froogle-Suche
And this card is definitely not crap.
Hans
I've been doing some digging... Seems like ALL Adaptec SATA and SAS RAID (or not RAID) controllers make use of Adaptec's own closed-source drivers.. *sigh* They provide a "SHIM" kit for users to be able to build a driver for their running kernel, but IMHO that sucks bigtime.. Then I need to bust my ass every time there's a kernel update available. I also looked at Highpoint (this, for example: http://www.highpoint-tech.com/USA/rr1820a.htm), apparently they provide source code for drivers, and also precompiled for SUSE up to 10.1 as of now. Anyone with opinions on Highpoint vs. Adaptec? -- Anders Norrbring Norrbring Consulting
At 02:25 PM 3/5/2007, Anders Norrbring wrote:
Hans du Plooy skrev:
On Sun, March 4, 2007 22:35, Pascal Bleser wrote:
Anyhow, stay away from any RAID controller that is cheaper than 1000 EUR because it's crap. And yet, software RAID on Linux outperforms most of them. 1000 EUR? http://froogle.google.de/froogle?q=adaptec+2820SA&btnG=Froogle-Suche And this card is definitely not crap. Hans
I've been doing some digging... Seems like ALL Adaptec SATA and SAS RAID (or not RAID) controllers make use of Adaptec's own closed-source drivers.. *sigh* They provide a "SHIM" kit for users to be able to build a driver for their running kernel, but IMHO that sucks bigtime.. Then I need to bust my ass every time there's a kernel update available. I will have to second the votes for using 3ware. Suse has the drivers built into the system for 3ware raid cards. I have used both ide an sata raid cards from them with no problems here at work an home. they work like a charm out of the box for me so far. I just do a clean install on a system that I insert the card into an they are seen an recognized with no problems at all. I use both raid 0 an 10 here with the.
jack -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Jack Malone skrev:
Hans du Plooy skrev:
On Sun, March 4, 2007 22:35, Pascal Bleser wrote:
Anyhow, stay away from any RAID controller that is cheaper than 1000 EUR because it's crap. And yet, software RAID on Linux outperforms most of them. 1000 EUR? http://froogle.google.de/froogle?q=adaptec+2820SA&btnG=Froogle-Suche And this card is definitely not crap. Hans
I've been doing some digging... Seems like ALL Adaptec SATA and SAS RAID (or not RAID) controllers make use of Adaptec's own closed-source drivers.. *sigh* They provide a "SHIM" kit for users to be able to build a driver for their running kernel, but IMHO that sucks bigtime.. Then I need to bust my ass every time there's a kernel update available. I will have to second the votes for using 3ware. Suse has the drivers built into the system for 3ware raid cards. I have used both ide an sata raid cards from them with no problems here at work an home. they work
At 02:25 PM 3/5/2007, Anders Norrbring wrote: like a charm out of the box for me so far. I just do a clean install on a system that I insert the card into an they are seen an recognized with no problems at all. I use both raid 0 an 10 here with the.
jack
I'd like to have an option for non-raid cards as well, 4-8 channels, but they seem to be hard to find.. I don't care much if it's a SAS or SATA-II card, and I don't care much about the RAID functionality either since many people say the software RAID in Linux is really good. -- Anders Norrbring Norrbring Consulting
On Mon, 5 Mar 2007, Anders Norrbring wrote:
I'd like to have an option for non-raid cards as well, 4-8 channels, but they seem to be hard to find..
I don't care much if it's a SAS or SATA-II card, and I don't care much about the RAID functionality either since many people say the software RAID in Linux is really good.
Does anybody have anything to say about the Addonics stuff? They've got
a whole plethora of stuff that looks pretty useful.
--
Carpe diem - Seize the day.
Carp in denim - There's a fish in my pants!
Jon Nelson
Jon Nelson skrev:
On Mon, 5 Mar 2007, Anders Norrbring wrote:
I'd like to have an option for non-raid cards as well, 4-8 channels, but they seem to be hard to find..
I don't care much if it's a SAS or SATA-II card, and I don't care much about the RAID functionality either since many people say the software RAID in Linux is really good.
Does anybody have anything to say about the Addonics stuff? They've got a whole plethora of stuff that looks pretty useful.
-- Carpe diem - Seize the day. Carp in denim - There's a fish in my pants!
Jon Nelson
Those aren't an option for me, my distributors carry Highpoint, Adaptec, 3ware and Promise (and the latter is dead according to those who tried them). -- Anders Norrbring Norrbring Consulting
On 3/5/07, Anders Norrbring
Jack Malone skrev:
Hans du Plooy skrev:
On Sun, March 4, 2007 22:35, Pascal Bleser wrote:
Anyhow, stay away from any RAID controller that is cheaper than 1000 EUR because it's crap. And yet, software RAID on Linux outperforms most of them. 1000 EUR? http://froogle.google.de/froogle?q=adaptec+2820SA&btnG=Froogle-Suche And this card is definitely not crap. Hans
I've been doing some digging... Seems like ALL Adaptec SATA and SAS RAID (or not RAID) controllers make use of Adaptec's own closed-source drivers.. *sigh* They provide a "SHIM" kit for users to be able to build a driver for their running kernel, but IMHO that sucks bigtime.. Then I need to bust my ass every time there's a kernel update available. I will have to second the votes for using 3ware. Suse has the drivers built into the system for 3ware raid cards. I have used both ide an sata raid cards from them with no problems here at work an home. they work
At 02:25 PM 3/5/2007, Anders Norrbring wrote: like a charm out of the box for me so far. I just do a clean install on a system that I insert the card into an they are seen an recognized with no problems at all. I use both raid 0 an 10 here with the.
jack
I'd like to have an option for non-raid cards as well, 4-8 channels, but they seem to be hard to find.. I don't care much if it's a SAS or SATA-II card, and I don't care much about the RAID functionality either since many people say the software RAID in Linux is really good.
Before you totally commit to software raid think about what happens with a drive failure or if you need to boot the SUSE DVD into rescue mode. I don't actually know the answers, but those are always my concerns with software raid. ie. For booting software raid does not normally provide any tolerance, and for rescue mode issues I just get nervous. Basically I don't mind software raid for non-boot drives, but for boot drives I have always avoided it. FYI: I've been running RAID-1 3ware boot drives for years. The 2-port 3ware card is about $100 IIRC. Greg -- Greg Freemyer The Norcross Group Forensics for the 21st Century -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Greg Freemyer wrote:
Before you totally commit to software raid think about what happens with a drive failure or if you need to boot the SUSE DVD into rescue mode.
I don't actually know the answers, but those are always my concerns with software raid. It works with no problems. I presently boot from a software raid 1 here at home (PATA) and at the office (SATA). The 10.2 rescue system works without a problem (though I did have problems with the rescue system in 10.1 and the raid). When I was running 10.1, I had to use my 9.3 DVD's rescue system, which worked fine as well.
ie. For booting software raid does not normally provide any tolerance, and for rescue mode issues I just get nervous. Booting period is intolerant of mistakes. I haven't found any difference with running the raid. I initially ran a boot partition, but haven't now for quite a while.
Basically I don't mind software raid for non-boot drives, but for boot drives I have always avoided it.
I have read that later versions of GRUB will natively support software raid. -- Joe Morris Registered Linux user 231871 running openSUSE 10.2 x86_64 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Tuesday 06 March 2007 00:42, Joe Morris (NTM) wrote:
Greg Freemyer wrote:
Before you totally commit to software raid think about what happens with a drive failure or if you need to boot the SUSE DVD into rescue mode.
I don't actually know the answers, but those are always my concerns with software raid.
It works with no problems. I presently boot from a software raid 1 here at home (PATA) and at the office (SATA). The 10.2 rescue system works without a problem (though I did have problems with the rescue system in 10.1 and the raid). When I was running 10.1, I had to use my 9.3 DVD's rescue system, which worked fine as well.
ie. For booting software raid does not normally provide any tolerance, and for rescue mode issues I just get nervous.
Booting period is intolerant of mistakes. I haven't found any difference with running the raid. I initially ran a boot partition, but haven't now for quite a while.
Basically I don't mind software raid for non-boot drives, but for boot drives I have always avoided it.
I have read that later versions of GRUB will natively support software raid.
-- Joe Morris Registered Linux user 231871 running openSUSE 10.2 x86_64 Greg, there seams to be one point you missed, Even though the software raid works in as a boot drive (in all situations I've ever tried), I seldom use the DVD recovery and such.
The partions on a software raid 1 are not controller specific, so you can take out a good disk, pop it into a usb-box and restore it on any machine! If find this functionality missing in hardware raids. This is has been my standard upgrades procedure for years. When I upgrade I've always bought 2 new Disks, install the system from DVD, and recover all my data from USB. I get nervous when using hardware raid, as I cannot pull the disks and get the data.... Jerry -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Monday 05 March 2007, Jerome R. Westrick wrote:
I get nervous when using hardware raid, as I cannot pull the disks and get the data....
But is that because hardware raid controllers are seldom set up as raid-1? Are we comparing apples to apples? I know some scsi controllers will write data in such a way that the same drive can't be read by another contoller (how DO they do that) but I've never seen this on ATA or SATA. -- _____________________________________ John Andersen
Greg Freemyer skrev:
On 3/5/07, Anders Norrbring
wrote: Jack Malone skrev:
Hans du Plooy skrev:
On Sun, March 4, 2007 22:35, Pascal Bleser wrote:
Anyhow, stay away from any RAID controller that is cheaper than 1000 EUR because it's crap. And yet, software RAID on Linux outperforms most of them. 1000 EUR? http://froogle.google.de/froogle?q=adaptec+2820SA&btnG=Froogle-Suche And this card is definitely not crap. Hans
I've been doing some digging... Seems like ALL Adaptec SATA and SAS RAID (or not RAID) controllers make use of Adaptec's own closed-source drivers.. *sigh* They provide a "SHIM" kit for users to be able to build a driver for their running kernel, but IMHO that sucks bigtime.. Then I need to bust my ass every time there's a kernel update available. I will have to second the votes for using 3ware. Suse has the drivers built into the system for 3ware raid cards. I have used both ide an sata raid cards from them with no problems here at work an home. they work
At 02:25 PM 3/5/2007, Anders Norrbring wrote: like a charm out of the box for me so far. I just do a clean install on a system that I insert the card into an they are seen an recognized with no problems at all. I use both raid 0 an 10 here with the.
jack
I'd like to have an option for non-raid cards as well, 4-8 channels, but they seem to be hard to find.. I don't care much if it's a SAS or SATA-II card, and I don't care much about the RAID functionality either since many people say the software RAID in Linux is really good.
Before you totally commit to software raid think about what happens with a drive failure or if you need to boot the SUSE DVD into rescue mode.
I don't actually know the answers, but those are always my concerns with software raid.
ie. For booting software raid does not normally provide any tolerance, and for rescue mode issues I just get nervous.
Basically I don't mind software raid for non-boot drives, but for boot drives I have always avoided it.
FYI: I've been running RAID-1 3ware boot drives for years. The 2-port 3ware card is about $100 IIRC.
Greg
Thanks, but this array will be used as storage space only. The system itself with all important system functions and the /home directories are run off SCSI RAID-1 and RAID-5 arrays on LSI Logic SCSI-320 RAID controllers, so that's not an issue here. -- Anders Norrbring Norrbring Consulting
On Mon, 2007-03-05 at 21:25 +0100, Anders Norrbring wrote:
Hans du Plooy skrev:
http://froogle.google.de/froogle?q=adaptec+2820SA&btnG=Froogle-Suche
And this card is definitely not crap.
Hans
I've been doing some digging... Seems like ALL Adaptec SATA and SAS RAID (or not RAID) controllers make use of Adaptec's own closed-source drivers.. *sigh* Where did you read that? Most of Adaptec's hardware raid cards use the aacraid driver that's part of the kernel. The link I posted certainly shows a card that does - I've used many of them, as well as their SCSI cousins (I can't comment on SAS - never tried it)
The only issue when you use a card to upgrade the disc setup on an older installation, the aacraid module might not have support for the newer card. Mantel's kernels used to be useful for that.
Anyone with opinions on Highpoint vs. Adaptec? Adaptec. My experience with Highpoint has been no better than with Promise. But that's just my 2c
Hans -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Hans du Plooy skrev:
On Mon, 2007-03-05 at 21:25 +0100, Anders Norrbring wrote:
Hans du Plooy skrev:
http://froogle.google.de/froogle?q=adaptec+2820SA&btnG=Froogle-Suche
And this card is definitely not crap.
Hans
I've been doing some digging... Seems like ALL Adaptec SATA and SAS RAID (or not RAID) controllers make use of Adaptec's own closed-source drivers.. *sigh* Where did you read that? Most of Adaptec's hardware raid cards use the aacraid driver that's part of the kernel. The link I posted certainly shows a card that does - I've used many of them, as well as their SCSI cousins (I can't comment on SAS - never tried it)
I downloaded their driver packs to see what was in them, and this is in the README file: 1. Overview ------------ The SHIM (Adaptec HostRAID re-compilable package ) code model allows the end user to recompile the code for Adaptec HostRAID without requiring access to Adaptec closed source RAID code. The proprietary code is supplied in a pre-compiled object module that can be linked to the recompiled source code to provide a fully functioning Linux device driver. This procedure can be used to recompile the SAS HostRAID driver (adp94xx) or the SATA2 drivers (aar81xx/adp3132). The SAS driver, adp94xx, supports the ASC-48300, ASC-44300 and ASC-58300 SAS controllers. The SATA2 driver, aar81xx, supports the AAR-1420SA and AAR-1430SA controllers and adp3132 supports AAR-1220SA Controller. Seems like it's not the aacraid driver in any of those, and I was actually aiming for the 48300 (http://www.adaptec.com/en-US/products/sas/host/SAS-48300/) which isn't even a RAID controller... Then "HostRAID" isn't h/w RAID, it's software, much like the infamous nVidia chipsets and Promise. I don't know if those driver packs are needed to just communicate with the drives, if not, I'm home safe with those controllers if I should use Linux' own software RAID. I should take a look at 1420SA and 2420SA as well, I just noticed that they exist, I don't have a clue about what they are/do yet.
The only issue when you use a card to upgrade the disc setup on an older installation, the aacraid module might not have support for the newer card. Mantel's kernels used to be useful for that.
Anyone with opinions on Highpoint vs. Adaptec? Adaptec. My experience with Highpoint has been no better than with Promise. But that's just my 2c
With emphasis on the words *used to be*, Huberts kernels are no more... :( I actually haven't heard anything negative about Highpoint so far, not much positive either... At least they do manufacture quite some chips for Adaptec, so I guess their stuff shouldn't stink too much? -- Anders Norrbring Norrbring Consulting
On Tue, March 6, 2007 06:12, Anders Norrbring wrote:
I downloaded their driver packs to see what was in them, and this is in the README file:
1. Overview ------------
The SHIM (Adaptec HostRAID re-compilable package ) code model allows
OK, I wasn't aware of this - haven't worked with any new cards in about a year.
Then "HostRAID" isn't h/w RAID, it's software, much like the infamous nVidia chipsets and Promise. I don't know if those driver packs are needed to just communicate with the drives, if not, I'm home safe with those controllers if I should use Linux' own software RAID.
http://www.adaptec.com/en-US/products/sata_cards/entry/AAR-1220SA/_details/1... Make sense then to avoid these
I should take a look at 1420SA and 2420SA as well, I just noticed that they exist, I don't have a clue about what they are/do yet.
According to their website the 1420SA is a HostRAID card. The 2420SA Is a normal hardware raid card - aacraid driver - I've used this one.
With emphasis on the words *used to be*, Huberts kernels are no more... :(
Sad that. My more_than_one_year_old_notebook's ACPI is still not sorted out (although there are some groundbraking patches in the upcoming 2.6.21). I'm still not having a 100% successful experiencing building vanilla kernels on SUSE - there's always something bitching at me.
I actually haven't heard anything negative about Highpoint so far, not much positive either... At least they do manufacture quite some chips for Adaptec, so I guess their stuff shouldn't stink too much?
Which chips do they manufacture? The RAID controller (shudder) or the SATA controller? I remember the LSI SATA-160 card had the LSI RAID card had three Sil3112 chips on (to control the six drives) and the LSI RAID controller on top of that. I assume the RAID controller is the same one used on the SCSI cards I used (had the same crap performance). Hans -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Hans du Plooy skrev:
On Tue, March 6, 2007 06:12, Anders Norrbring wrote:
I downloaded their driver packs to see what was in them, and this is in the README file:
1. Overview ------------
The SHIM (Adaptec HostRAID re-compilable package ) code model allows
OK, I wasn't aware of this - haven't worked with any new cards in about a year.
Then "HostRAID" isn't h/w RAID, it's software, much like the infamous nVidia chipsets and Promise. I don't know if those driver packs are needed to just communicate with the drives, if not, I'm home safe with those controllers if I should use Linux' own software RAID.
http://www.adaptec.com/en-US/products/sata_cards/entry/AAR-1220SA/_details/1... Make sense then to avoid these
I was afraid of that too.. I'll call my distributor to try to sort it out. As usual, Adaptec aren't too responsive to e-mail questions... (Even though I'm a Adaptec partner..)
I should take a look at 1420SA and 2420SA as well, I just noticed that they exist, I don't have a clue about what they are/do yet.
According to their website the 1420SA is a HostRAID card. The 2420SA Is a normal hardware raid card - aacraid driver - I've used this one.
I'll look into that one a bit closer. But I'd really like a 8-channel card for future expansion, preferably a SAS card, but I'm not so sure anymore...
With emphasis on the words *used to be*, Huberts kernels are no more... :(
Sad that. My more_than_one_year_old_notebook's ACPI is still not sorted out (although there are some groundbraking patches in the upcoming 2.6.21). I'm still not having a 100% successful experiencing building vanilla kernels on SUSE - there's always something bitching at me.
Hmm... I'm not seeing that, but on the other hand, I don't use ACPI.. (But the wife is bitching at me instead).
I actually haven't heard anything negative about Highpoint so far, not much positive either... At least they do manufacture quite some chips for Adaptec, so I guess their stuff shouldn't stink too much?
Which chips do they manufacture? The RAID controller (shudder) or the SATA controller? I remember the LSI SATA-160 card had the LSI RAID card had three Sil3112 chips on (to control the six drives) and the LSI RAID controller on top of that. I assume the RAID controller is the same one used on the SCSI cards I used (had the same crap performance).
I can't say, I saw a press info on the subject a while back, I guess our friend Google might clear it out.. ;) -- Anders Norrbring Norrbring Consulting
On Tue, March 6, 2007 09:26, Anders Norrbring wrote:
I'll look into that one a bit closer. But I'd really like a 8-channel card for future expansion, preferably a SAS card, but I'm not so sure anymore...
Not SAS, but same as the 2420SA - just with 8 ports: http://www.adaptec.com/en-US/products/sata_tech/value/AAR-2820SA/
Sad that. My more_than_one_year_old_notebook's ACPI is still not sorted out (although there are some groundbraking patches in the upcoming 2.6.21). I'm still not having a 100% successful experiencing building vanilla kernels on SUSE - there's always something bitching at me.
Hmm... I'm not seeing that, but on the other hand, I don't use ACPI.. (But the wife is bitching at me instead).
Ha ha! Problem on my notebooks is my fans don't switch on. Whic hwas OK before CPU frequency scaling started working, because the chip was underclocked and barely ever hit 60 degrees celcius. But at some point that was fixed (the frequency scaling) which now gives me a problem. Hans -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Hans du Plooy skrev:
On Tue, March 6, 2007 09:26, Anders Norrbring wrote:
I'll look into that one a bit closer. But I'd really like a 8-channel card for future expansion, preferably a SAS card, but I'm not so sure anymore...
Not SAS, but same as the 2420SA - just with 8 ports: http://www.adaptec.com/en-US/products/sata_tech/value/AAR-2820SA/
Sad that. My more_than_one_year_old_notebook's ACPI is still not sorted out (although there are some groundbraking patches in the upcoming 2.6.21). I'm still not having a 100% successful experiencing building vanilla kernels on SUSE - there's always something bitching at me. Hmm... I'm not seeing that, but on the other hand, I don't use ACPI.. (But the wife is bitching at me instead).
Ha ha! Problem on my notebooks is my fans don't switch on. Whic hwas OK before CPU frequency scaling started working, because the chip was underclocked and barely ever hit 60 degrees celcius. But at some point that was fixed (the frequency scaling) which now gives me a problem.
Hans
Maybe I'm just cheap and stupid, but actually the Highpoint 1820A looks like a decent alternative.. http://www.highpoint-tech.com/USA/rr1820a.htm It's almost exactly half the price as the Adaptec 2820SA, and performance looks similar. The "only" thing not included are Adaptec's managment stuff (which I never use on my SCSI arrays anyway). -- Anders Norrbring Norrbring Consulting
Hans du Plooy wrote:
Ha ha! Problem on my notebooks is my fans don't switch on. Whic hwas OK before CPU frequency scaling started working, because the chip was underclocked and barely ever hit 60 degrees celcius. But at some point that was fixed (the frequency scaling) which now gives me a problem.
ACPI is definitely a mixed bag. I don't think I've ever seen a system where it worked completely right, unlike APM, which was pretty mature. On my Thinkpad T22, CPU speed switching and screen dimming work OK, but the machine doesn't suspend properly. Actually, it doesn't even power off when I shut down, except on rare occasions when it's on battery and I have no PCMCIA cards plugged in. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Tue, March 6, 2007 18:56, David Brodbeck wrote:
ACPI is definitely a mixed bag. I don't think I've ever seen a system where it worked completely right, unlike APM, which was pretty mature. Worked like a bomb on my Compaq Armada m700 - too bad the notebook could only take 512mb memory, otherwise I would still have had it.
Hans -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Tuesday 06 March 2007, David Brodbeck wrote:
ACPI is definitely a mixed bag. I don't think I've ever seen a system where it worked completely right, unlike APM, which was pretty mature.
Most laptops now days need it. Period. End of story. If you don't run it you get stuttering sound, jerky mouse pointers, dead or flaky usb ports, etc, etc. Too many devices. So few interrupts. For my laptops (sony, dell) its worked very well since SuSE 8.2. Before that, it was a crap shoot. -- _____________________________________ John Andersen
John Andersen wrote:
On Tuesday 06 March 2007, David Brodbeck wrote:
ACPI is definitely a mixed bag. I don't think I've ever seen a system where it worked completely right, unlike APM, which was pretty mature.
Most laptops now days need it. Period. End of story. If you don't run it you get stuttering sound, jerky mouse pointers, dead or flaky usb ports, etc, etc.
Oh, I know. My Thinkpad T22 is sort of one of them. Under SUSE 10.1, if I turned off ACPI and turned on APM, I could get working suspend, but I had to disable CPU frequency switching in the BIOS or things got wonky when I switched from AC to battery. In 10.2, suspend doesn't work in APM, either. Suspend/resume have been gradually getting less reliable for me with every new release, which makes me wonder if the Linux developers have sort of given up on the whole thing as a bad job. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On 3/7/07, David Brodbeck
John Andersen wrote:
On Tuesday 06 March 2007, David Brodbeck wrote:
ACPI is definitely a mixed bag. I don't think I've ever seen a system where it worked completely right, unlike APM, which was pretty mature.
Most laptops now days need it. Period. End of story. If you don't run it you get stuttering sound, jerky mouse pointers, dead or flaky usb ports, etc, etc.
Oh, I know. My Thinkpad T22 is sort of one of them. Under SUSE 10.1, if I turned off ACPI and turned on APM, I could get working suspend, but I had to disable CPU frequency switching in the BIOS or things got wonky when I switched from AC to battery. In 10.2, suspend doesn't work in APM, either. Suspend/resume have been gradually getting less reliable for me with every new release, which makes me wonder if the Linux developers have sort of given up on the whole thing as a bad job.
I know the SATA driver team has been working on suspend / resume since last summer. I don't think it was really supported with sata before that. They definately don't have it working perfectly yet, but at least they are testing it and getting new fixes in with every release of the vanilla kernel. The ide driver was officially unmaintained from June 06 until a month or two ago, so IDE support had likely been going backwards during that time (2.6.17 thru 2.6.19 I think saw no real IDE driver work). FYI: Laptops started shipping with internal sata drives last summer as well. I think that was the impetuous behind the sata driver team getting to work on it. Greg -- Greg Freemyer The Norcross Group Forensics for the 21st Century -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Hans du Plooy skrev: [8<]
I actually haven't heard anything negative about Highpoint so far, not much positive either... At least they do manufacture quite some chips for Adaptec, so I guess their stuff shouldn't stink too much?
Which chips do they manufacture? The RAID controller (shudder) or the SATA controller? I remember the LSI SATA-160 card had the LSI RAID card had three Sil3112 chips on (to control the six drives) and the LSI RAID controller on top of that. I assume the RAID controller is the same one used on the SCSI cards I used (had the same crap performance).
I just placed an order for the Highpoint 1820 8-channel controller and 6x500 WD disks.. I'll report here how I feel about the controller when it's installed in a couple of days. :) -- Anders Norrbring Norrbring Consulting
Anders Norrbring skrev:
Hans du Plooy skrev: [8<]
I actually haven't heard anything negative about Highpoint so far, not much positive either... At least they do manufacture quite some chips for Adaptec, so I guess their stuff shouldn't stink too much?
Which chips do they manufacture? The RAID controller (shudder) or the SATA controller? I remember the LSI SATA-160 card had the LSI RAID card had three Sil3112 chips on (to control the six drives) and the LSI RAID controller on top of that. I assume the RAID controller is the same one used on the SCSI cards I used (had the same crap performance).
I just placed an order for the Highpoint 1820 8-channel controller and 6x500 WD disks.. I'll report here how I feel about the controller when it's installed in a couple of days. :)
Alright folks.. STAY AWAY from this piece of junk! I'll put out the Highpoint on eBay for a dollar, it's a no-good piece of trash, which of course doesn't show up ANYWHERE until you actually try to use it. I'll order a Adaptec instead. -- Anders Norrbring Norrbring Consulting
On Saturday 10 March 2007, Anders Norrbring wrote:
I just placed an order for the Highpoint 1820 8-channel controller and 6x500 WD disks.. I'll report here how I feel about the controller when it's installed in a couple of days. :)
Alright folks.. STAY AWAY from this piece of junk! I'll put out the Highpoint on eBay for a dollar, it's a no-good piece of trash, which of course doesn't show up ANYWHERE until you actually try to use it.
Oh, I dono,,, Its probably fine if you find the jumper to disable the on board raid controller and just use it for all the channels with software raid. /running and ducking... -- _____________________________________ John Andersen -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
John Andersen skrev:
On Saturday 10 March 2007, Anders Norrbring wrote:
I just placed an order for the Highpoint 1820 8-channel controller and 6x500 WD disks.. I'll report here how I feel about the controller when it's installed in a couple of days. :)
Alright folks.. STAY AWAY from this piece of junk! I'll put out the Highpoint on eBay for a dollar, it's a no-good piece of trash, which of course doesn't show up ANYWHERE until you actually try to use it.
Oh, I dono,,, Its probably fine if you find the jumper to disable the on board raid controller and just use it for all the channels with software raid.
/running and ducking...
Don't run.. Hehe.. It's not even that hard (finding the jumper), it's actually a software based piece of trash with a BIOS onboard. So, to use software RAID is just a matter of choice between Linux RAID or Highpoint RAID, software in both cases. It's out of my box, and I will never, ever recommend Highpoint to any of our clients. Geez. At least "the others" has the decency to inform presumptive customers about the software fact when you check their product info pages on web sites, but not Highpoint. I wrote them a very angry letter as well as requested a RMA from my distributor... Oh, I also placed an order for a Adaptec 2820SA HBA which is a real RAID controller.... -- Anders Norrbring Norrbring Consulting
Anders Norrbring wrote:
actually a software based piece of trash with a BIOS onboard.
Oh, I also placed an order for a Adaptec 2820SA HBA which is a real RAID controller....
sqorry if I seem silly, but how can any raid controller work without any software? if not in user space nor kernel space; it must be somewhere onboard? any scsi controller (any hdd, by the way) have some sort of software onboard :-) of course there is good and bad software (evan the microprocessor have software :-) jdd -- http://www.dodin.net Lucien Dodin, inventeur http://lucien.dodin.net/index.shtml -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
jdd wrote:
Anders Norrbring wrote:
actually a software based piece of trash with a BIOS onboard.
Oh, I also placed an order for a Adaptec 2820SA HBA which is a real RAID controller....
sqorry if I seem silly, but how can any raid controller work without any software? if not in user space nor kernel space; it must be somewhere onboard?
The real question is "which functions reside in the module and which reside in the raid hba". In earlier times software raid was frowned upon because the server cpu had to compute the necessary raid striping, which degraded performance, but today most cpus are not taxed enough to validate that argument. I have a dual boot installation Suse/WinXP on a hardware raid5, I dare you to try that with software raid. (^-^) -- Sandy List replies only please! Please address PMs to: news-reply2 (@) japantest (.) homelinux (.) com -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Sandy Drobic skrev:
jdd wrote:
Anders Norrbring wrote:
actually a software based piece of trash with a BIOS onboard. Oh, I also placed an order for a Adaptec 2820SA HBA which is a real RAID controller....
sqorry if I seem silly, but how can any raid controller work without any software? if not in user space nor kernel space; it must be somewhere onboard?
The real question is "which functions reside in the module and which reside in the raid hba". In earlier times software raid was frowned upon because the server cpu had to compute the necessary raid striping, which degraded performance, but today most cpus are not taxed enough to validate that argument.
I have a dual boot installation Suse/WinXP on a hardware raid5, I dare you to try that with software raid. (^-^)
Also, on top of that, I absolutely need 100% transparent hot-spare functionality, no matter the O/S being running or not. So, off-board RAID software isn't an option. That means, bye-bye Highpoint... -- Anders Norrbring Norrbring Consulting
Anders Norrbring skrev:
Sandy Drobic skrev:
jdd wrote:
Anders Norrbring wrote:
actually a software based piece of trash with a BIOS onboard. Oh, I also placed an order for a Adaptec 2820SA HBA which is a real RAID controller....
sqorry if I seem silly, but how can any raid controller work without any software? if not in user space nor kernel space; it must be somewhere onboard?
The real question is "which functions reside in the module and which reside in the raid hba". In earlier times software raid was frowned upon because the server cpu had to compute the necessary raid striping, which degraded performance, but today most cpus are not taxed enough to validate that argument.
I have a dual boot installation Suse/WinXP on a hardware raid5, I dare you to try that with software raid. (^-^)
Also, on top of that, I absolutely need 100% transparent hot-spare functionality, no matter the O/S being running or not.
So, off-board RAID software isn't an option. That means, bye-bye Highpoint...
Oh yeah..... Today the Adaptec 2820SA adapter arrived... What a difference! Everything runs on the controller, and it's MORE than twice as fast than the Highpoint. -- Anders Norrbring Norrbring Consulting -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Fri, March 16, 2007 11:09, Anders Norrbring wrote:
Oh yeah..... Today the Adaptec 2820SA adapter arrived... What a difference! Everything runs on the controller, and it's MORE than twice as fast than the Highpoint.
Glad to hear! Hans -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Hans du Plooy skrev:
On Fri, March 16, 2007 11:09, Anders Norrbring wrote:
Oh yeah..... Today the Adaptec 2820SA adapter arrived... What a difference! Everything runs on the controller, and it's MORE than twice as fast than the Highpoint.
Glad to hear!
Hans
You get what you pay for.. ;) This little sucker was about €100 more than the Highpoint.. Anyway, now it feels like the disk subsystem can be both used and trusted. -- Anders Norrbring Norrbring Consulting -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
* jdd (jdd@dodin.org) [20070311 13:01]:
sorry if I seem silly, but how can any raid controller work without any software? if not in user space nor kernel space; it must be somewhere onboard?
That answer isn't that silly :) The hardware RAID controllers also need software, but in the form of firmware onoard the controller, running on the controllers (embedded) CPU. This means that all operations are transparent to the OS, e.g. the OS only sees the RAID device(s) and not the underlying drives. Philipp -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
* Pascal Bleser (pascal.bleser@skynet.be) [20070304 23:36]:
Anyhow, stay away from any RAID controller that is cheaper than 1000 EUR because it's crap. And yet, software RAID on Linux outperforms most of them.
Hmm, AFAICS, with one exception (24 Port PCIe) *all* of the 3ware controllers cost less then 1000 €. Philipp -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Philipp Thomas skrev:
* Pascal Bleser (pascal.bleser@skynet.be) [20070304 23:36]:
Anyhow, stay away from any RAID controller that is cheaper than 1000 EUR because it's crap. And yet, software RAID on Linux outperforms most of them.
Hmm, AFAICS, with one exception (24 Port PCIe) *all* of the 3ware controllers cost less then 1000 €.
Yep.. Nobody knows where that price came from.. Anyway, I got myself a 8-port Highpoint, unfrotunately it's only 1.5Mbit/s (SATA and not SATA II), but at least it was priced half of what the Adaptec costed. Anders.
* Anders Norrbring (lists@norrbring.se) [20070309 17:56]:
Anyway, I got myself a 8-port Highpoint, unfortunately it's only 1.5Mbit/s (SATA and not SATA II), but at least it was priced half of what the Adaptec costed.
Might I ask what you paid for it? I myself had decided to go with either 9650SE-4LPML (~ 300 €) or 9650SE-8LPML (~450 €) (data sheet here: http://www.3ware.com/products/pdf/9650SE_DS_101106.pdf). But after checking prices, I'm not sure if I shouldn't also go with the HighPoint RocketRAID 2320, given that this 8 port card costs approx. half of the AMCC/3ware one. Philipp -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Philipp Thomas skrev:
* Anders Norrbring (lists@norrbring.se) [20070309 17:56]:
Anyway, I got myself a 8-port Highpoint, unfortunately it's only 1.5Mbit/s (SATA and not SATA II), but at least it was priced half of what the Adaptec costed.
Might I ask what you paid for it? I myself had decided to go with either 9650SE-4LPML (~ 300 €) or 9650SE-8LPML (~450 €) (data sheet here: http://www.3ware.com/products/pdf/9650SE_DS_101106.pdf). But after checking prices, I'm not sure if I shouldn't also go with the HighPoint RocketRAID 2320, given that this 8 port card costs approx. half of the AMCC/3ware one.
Philipp
I paid just below €200 for it, but I can't say yet how it is, I'm about to install it this weekend.. -- Anders Norrbring Norrbring Consulting
i use 3ware if you are using a lot of drives, i use 8-12 in a raid 5 config, and its pci-x or pci-e you are better off using the 3ware over linux RAID, because with linux you'd have to get a expansion sata pci-x/e expansion card anyways, and thus you may as well get the 3ware with hardware raid 5 or even 50 built in. 3ware also have linux command line monitoring and admin tools, also you can get battery backup modules and the top 3ware in the line i use has a 512MB cache (with battery you can use it for wrtie cache), and this definitely can speed things up depending on what your server is doing. I have about 6 3wares installed now, all have been 100% perfect. I used 3-4 promise's in the past, they are slow, and don't have decent linux command line monitoring and admin. -tl On Sun, 2007-03-04 at 22:28 +0100, Anders Norrbring wrote:
I'm looking at building a SATA II RAID-5 system, and I'd like to get some thoughts on controllers since there are a multitude out there.. I'm going for either 4 or 8 ports. The rig should be run by a 10.1 x86_64 system with dual opterons.
The manufacturers I'm looking at are:
Adaptec Highpoint 3Ware Promise
Can someone with knowledge about these tell me the pros and cons with each of them?
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
The Areca cards are faster, but the drivers are not well integrated into the mainline kernels. So, if you want easy, go with 3Ware... if you want performance and are willing to build your own drivers, then I would highly recommend Areca. My $0.02 - Herman tleslie wrote:
i use 3ware
if you are using a lot of drives, i use 8-12 in a raid 5 config, and its pci-x or pci-e
you are better off using the 3ware over linux RAID, because with linux you'd have to get a expansion sata pci-x/e expansion card anyways, and thus you may as well get the 3ware with hardware raid 5 or even 50 built in.
3ware also have linux command line monitoring and admin tools,
also you can get battery backup modules and the top 3ware in the line i use has a 512MB cache (with battery you can use it for wrtie cache), and this definitely can speed things up depending on what your server is doing.
I have about 6 3wares installed now, all have been 100% perfect. I used 3-4 promise's in the past, they are slow, and don't have decent linux command line monitoring and admin.
-tl
On Sun, 2007-03-04 at 22:28 +0100, Anders Norrbring wrote:
I'm looking at building a SATA II RAID-5 system, and I'd like to get some thoughts on controllers since there are a multitude out there.. I'm going for either 4 or 8 ports. The rig should be run by a 10.1 x86_64 system with dual opterons.
The manufacturers I'm looking at are:
Adaptec Highpoint 3Ware Promise
Can someone with knowledge about these tell me the pros and cons with each of them?
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
On Sunday 04 March 2007, tleslie wrote:
you are better off using the 3ware over linux RAID, because with linux you'd have to get a expansion sata pci-x/e expansion card anyways, and thus you may as well get the 3ware with hardware raid 5 or even 50 built in.
Well that depends on the hardware. The last Dell low-end server I purchased had 6 sata headers on board, and that was standard. The expansion board for 6 more was only $50 AIR. For industrial use there might (MIGHT) be a reason to go 3ware, but by and large any home or small business use is better off long term with software raid. Did you know you can mix drive types with software raid? Some scsi, some sata, some ATA? Yast sets it up perfectly. -- _____________________________________ John Andersen -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
Not sure about hardware, but I just built a software RAID-1 system a few days back that boots directly from the Array. This was hand-built kernel on Debian, but I'd think it'll work under SuSE. IIRC, they include the MD driver in the kernel. See Documentation/md.txt in the kernel source. This is the menu.lst entry I used. title Debian GNU/Linux, kernel 2.6.20 root (hd0,0) kernel /boot/vmlinuz-2.6.20 root=/dev/md_d0p1 ro console=tty0 md=d0,/dev/sda,/dev/sdb Thinking on it, I'm not sure the root directive is "right". I'm not sure how one would tell grub to look at a RAID device. It may be it only works because this is RAID-1. These confirm this: http://gentoo-wiki.com/Talk:HOWTO_Gentoo_Install_on_Software_RAID#Raid_5_and... http://www.digitalmapping.sk.ca/Networks/ExpandingRAID.htm Anders Norrbring wrote:
I'm looking at building a SATA II RAID-5 system, and I'd like to get some thoughts on controllers since there are a multitude out there.. I'm going for either 4 or 8 ports. The rig should be run by a 10.1 x86_64 system with dual opterons.
The manufacturers I'm looking at are:
Adaptec Highpoint 3Ware Promise
Can someone with knowledge about these tell me the pros and cons with each of them?
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
I've got about 6 large sata raids (16 drives in each) and many smaller
guys - all with 3ware cards in them - frankly with no problems at all.
The 3ware cards are more expensive but if you really want a raid that
works day in and day out they are the way to go -
M--
On 3/4/07, Anders Norrbring
I'm looking at building a SATA II RAID-5 system, and I'd like to get some thoughts on controllers since there are a multitude out there.. I'm going for either 4 or 8 ports. The rig should be run by a 10.1 x86_64 system with dual opterons.
The manufacturers I'm looking at are:
Adaptec Highpoint 3Ware Promise
Can someone with knowledge about these tell me the pros and cons with each of them? --
Anders Norrbring Norrbring Consulting
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org
participants (17)
-
Anders Norrbring
-
David Brodbeck
-
Greg Freemyer
-
Hans du Plooy
-
Herman Knief
-
Jack Malone
-
jdd
-
Jerome R. Westrick
-
Joe Morris (NTM)
-
John Andersen
-
Jon Nelson
-
Michael Folsom
-
Pascal Bleser
-
Philipp Thomas
-
Russell Jones
-
Sandy Drobic
-
tleslie