Hello All,
I've been having a heck of a time with luns presented from Hitachi Array's ever since we've switched over to Sles9 from 8. First we couldn't recognize greater then 8 luns out of the box, which we got past once we were informed that "scsi_mod.dev_flags=HITACHI:OPEN-3:0x240,HITACHI:OPEN-E:0x240" had to be placed at the end of our boot string in the grub.conf. Then we bumped into the issue where not all of the luns presented to the OS were being reported within /proc/scsi/scsi, which put a dig damper on things due to the fact that we're big on scsidev to prevent device slippage. This was fixed in build -201 of the kernel.
Now, what I'm hitting seems to be our next big road block. We're building a 10 node Oracle10gR2 cluster. We have dual qla's in the machines, minus two machines that have two singles. We're presenting the first 4 luns to each machine as non-shared storage, that will be configured for the oracle binary installs and some user directories. The next 5 luns are presented as shared storage to all 10 machines & will be bound as raw devices for the oracle registry & voting disks, and the remainder of the disks will be presented as shared to all 10 as well and will be the raw database devices. Now, for the database devices, their being presented down 5 channels to the fibre switch for performance reasons. We're also presenting storage down both ports of the qlogic card, but not multipathed. Just balancing out what we want down each path for I/O reasons. Now the issue's we're seeing with this setup though, is that on boot, not all of the devices are presented to the OS. I can see all of the devices sitting at /proc/scsi/qla2xxx/*, but they are not reported within lsscsi, or /proc/partitions. We're wondering if since the 10 machines are in a shared zone, that if each of the machines is actually taking in account for the luns that it sees, as well as those same luns presented to the other 9 machines, therefore making us hit a limitation much earlier in the game. (just a theory)
Once SP3 was released, we jumped on that to see if maybe something was patched that would just miraculously fix our problem, but it wasn't the case.(we're not that lucky =]) I really don't know what else to do at this point. I'll gladly provide any more information that I might have left out here if anyone would have any idea's on where to start with this one. Or if anyone knows somebody that's running sles9 hooked up to any hitahi arrays and use more then just a few luns here and there. It's not unlikely to see 400-500 luns presented to some of our machines. With 10g we'll obviously be cutting back on that number quite a bit due to presenting larger luns for the raw database devices. I'm just going to stop writing and hit send to see if I can spark any interest. If I do, like I said, I'll gladly fill in some more blanks.
Thanks In Advance All, Mike K