On Fri, Jun 3, 2016 at 2:14 PM, Lew Wolfgang wrote:

E7-8893v3

Per Intel: Recommended Customer Price $6841.00 So, over $25K just for the 4 CPUs Probably over $50K for the RAM. This is one hell of a machine you are talking about. All quad (or dual) socket servers are NUMA as I understand it. If you read the link I sent a big part of the issue is if one of the sockets runs out of work to do (ie. there aren't 8 threads scheduled to it), then the scheduler may fail to move new threads to it for seconds. That is even in the case of all the threads being independent. The Linux scheduler in the mainline kernel is simply not doing a good job of manager the NUMA thread pools. == next question == Is there much disk i/o? If so, NVMe SSDs are what you "want" for storage. The price is way down. A 128GB NVMe SSD from Samsung is under $100. (Scale up as needed.) NVMe solutions were in the thousands minimum a couple years ago. read these 2 posts about NVMe design from last night on the factory list: http://markmail.org/message/owyjv7di4ilss7bk http://markmail.org/message/wqpnm3yhyugm34ek == Highlevel data storage info == I've let myself get behind, but I think this is right. => SCSI /dev/sdx Been around forever, very feature rich. /dev/sdx originally meant SCSI disk x => IDE /dev/hdx The old ide drivers. Very popular 2000-2005. => SATA /dev/sdx Sata was IDE on steroids from a feature perspective. So much so, it was layered under the SCSI driver as libata instead of living in the old ide driver code. => NVMe /dev/nvme The newest thing under the sun. Designed explicitly to support highspeed SSDs and have direct connections to the PCIe bus. An entire new linux driver had to be developed. /dev/nvme0 is the 1st NVMe drive in your system, /dev/nvme0n1 is the 1st namespace (similar to a SCSI LUN) on that drive /dev/nvme0n1p1 is the 1st partition in that namespace. NVMe has a theoretical max speed of 10 GB/sec (iirc) Real world is above 1 GB/sec at affordable costs. I saw a laptop earlier this week with a 500GB NVMe SSD. (Made by Alienware). Greg -- Greg Freemyer www.IntelligentAvatar.net -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

On 06/03/2016 03:40 PM, Lew Wolfgang wrote:

On 06/03/2016 12:04 PM, Greg Freemyer wrote:

...

On Fri, Jun 3, 2016 at 2:14 PM, Lew Wolfgang wrote:

...

E7-8893v3 Per Intel: Recommended Customer Price $6841.00

So, over $25K just for the 4 CPUs

Probably over $50K for the RAM.

This is one hell of a machine you are talking about.

The "pie-in-the-sky" quote was about $175,000 US. This included 400-GB of NVMe for the operating system.

The data disks would be a RAID of spinning SAS. We've lots of experience here and can manage 1.4-GB/sec writing to an 11-disk RAID6 array.

But it looks today like they choked on the price and are now considering modification of the display algorithms to run on smaller machines. Still, it would have been fun...

Thanks for your help Greg, the vendor I was working with is going to look at the link you sent too.

Regards, Lew

Sheeze! And I thought I had seen a big box when I ended up with the hand-me-down Supermicro H8QM8-2. (the plus side to the older hardware was spare Opteron 8360 SE's were only $14 each, and an extra 32G of ram was another $40, I also splurged for a pair of LSI Megaraid 8888ELP cards $12 each [very little benefit in SATA III for platter drives] :-) Why anyone would ever need more than 64K of RAM?... Seriously, the limitations for the amount of memory that can be addressed/managed by each processor is ... well processor dependent. As for the Linux scheduler divvying out instructions into pipelines and the processor's scheduler and control of separate regions of RAM, there seems to be no one-size fits all solution. That's why you system designer folks make the big bucks :) That's about where my eyes glaze over. The discussions of NUMA addressing the processor stall problem inherent with SMP providing each core full access to all available memory makes sense, along with the NUMA nodes and memory zones, but then throw in the software dependency and differences needed to tailor different handling for small app small percentage of overall memory verses large application large percentage of ram use.... and I'll punt. A quick look and I found several articles that helped make it clear as mud: Non-uniform memory access https://en.wikipedia.org/wiki/Non-uniform_memory_access NUMA (Non-Uniform Memory Access): An Overview http://queue.acm.org/detail.cfm?id=2513149 I'm just glad I'm not the one that has to make the call on putting together six or seven figure boxes. I'll take comfort being the one that gets the call when something doesn't quite work out as expected :) -- David C. Rankin, J.D.,P.E. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org