Adam Tauno Williams wrote:
On Mon, 2009-07-06 at 08:42 -0700, Lew Wolfgang wrote:
I've been working on a project for several years now that requires lots of disk space. We've got three servers, each with 40 directly connected 1-TB SATA disks. Now, the project is expanding and will require as many as 98 directly connected disks. I know that I can get the hardware and RAID (JBOD) controllers to mechanize this, but I'm not sure about the OS. How many individual disk (/dev/sdxx) drives can Linux support?
Last number I saw was 2,304. But I think that kind of architecture (and even your number) is unmanageable/unsustainable. Your going to need some high-end controllers to get near that number.
Hi Adam, I was thinking about the 3Ware 9690SA-8I, which supports up to 128 individual disks. I could divide the disks among as many controllers as I can fit in one motherboard. This application is rather non-standard. It basically sorts and shuffles lots of data from one disk to another in strings of six. Not much data processing, but lots of disk I/O. The disks are all individual (no RAID) and have to be hot-swappable and accessible from the front of the rack. The reason for this is that data is fed into it by inserting 1-TB disks from another process. Processed data is delivered to subsequent systems by physically moving some of the disks.
BTW, we need directly connected disks because of the bandwidth limits that NFS throws up. I'd be happy to listen to alternatives.
Fiber channel? Although I really suspect that if you tested iSCSI you'd find the bandwidth sufficient (I've just heard the cannot-do-it-due-to-bandwidth allot and found it to be very rarely true).
I tried iSCSI via Fiber Channel recently and found the bandwidth lacking. Maybe I wasn't doing it right? (It was RHEL 5.2!)
There is really no way to introduce HSM (hierarchical storage management) into your app? If the answer is no, I don't mean to be harsh, but your application is effectively broken (it certainly will stop eventually). Maybe use DASD for recent/active data-sets and migrate less active data-sets to iSCSI attached storage?
Some here might argue that the app is broken! :-) But we have to live within the constraints demanded by the project. I don't think that HSM is a fit for this particular requirement. Thanks for your suggestions. Lew -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org