Lew Wolfgang wrote:
Hi Folks,
I've got, at this point, a rather ill-defined requirement that I thought I'd run by you. I thought about posting on the off-topic list, but this is a real world task that I hope SuSE is up to.
SUSE won't have a problem with handling it - there are distros better suited for this type of computation, but that usually means they've got the tools pre-selected etc.
The number of elements for a fully populated array is huge (2.5e+13). There are ways to reduce the number of elements, maybe by the sources being in a smaller patch than the receivers. It might also be possible to use a polar grid about each source coordinate with perhaps 50 radials. It's thought that the total array size could be pared to 10-TB or less.
You've neglected to mention how fast or how often you need to process this amount of data. Storing it, on disk on in memory is not a problem - 10 x 1Tb SATA drives and/or 640 nodes with 16Gb of memory each.
Do you think this kind of problem could be hosted on anything affordable by a small company (~$30,000)?
I think quite possibly, yes. USD30K will buy you a decent amount of hardware, certainly plenty of space for holding your data. Here's an article about someone building an HPC cluster: http://jessen.ch/ammonite/ It's a bit old, so you'll certainly get more for your money today.
I know it's rather an ill-defined problem at this point, but I'd guess that HDF5 would make the most sense.
I think perhaps your question is better asked on the beowulf list, where problems such as these are dealt with regularly. That list has some very knowledgeable people. http://www.beowulf.org/mailman/listinfo/beowulf /Per Jessen, Zürich -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org For additional commands, e-mail: opensuse+help@opensuse.org