Dima
I have S2895 too. I was able to make OS see all the system memory (8 Gb) enabling "memory node interleave" and "memory hole mapping: software" in BIOS. That probably will penalize your performance a bit .. I guess the missing memory is what is called PCI memory hole ...
I have an S2885, and with both the BIOS versions where the "memory
hole remapping: software" was available, memory copy bandwidth in the
bottom 4 GB (including the remapped portion) was considerably
decreased, from about 1.7GB/sec in the non-remapped case, to about
1.2GB/sec. YMMV, but it seems there is either a performance issue with
the software remapping in general or a bug with the implementation.
I've informed Tyan of the issue. (My server has 8GB physically
installed, but only 7170-ish MB is available, with remapping it puts
the memory between 3 & 4GB up at the end of the address space, so
physical memory appears to go up to the 9GB mark)
Note that for the newest Opterons available (including the dual-core
chips), there is a new "hardware remapping" feature which may not
have the performance hit. I haven't been able to test it but bet
it probably doesn't have that problem.
Some people seem to know what the "hole before 4GB" is, and others
do not, so a short tutoral (which those who know can tune out):
-- Hardware with RAM or memory-mapped I/O needs to be able to map
it somewhere.
-- 32-bit OSes *and* 32-bit addressable hardware (which is the
majority of the PCI hardware out there right now) both need
to be able to work for the general case.
So, a "hole" at 4GB which grows downward depending on the total size
of the addresses used is punched. It consists of (usually more or
less in this order growing downward from 4GB):
-- x86 ROM/BIOS (this really is required to be aligned against
the top of the 4GB boundary)
-- IO APIC
-- PCI/AGP devices as mapped in PCI config space numerically
(but not required to be in that order) NOTE each of the
PCI domains are required to be aligned to their native size.
-- AGP aperture (NOTE: the agp aperture is required to be aligned
to it's native size.
The point of the "native size alignment" comments above is that, for
example, if you have, say a video card with a 256MB video buffer, then
at minimum, you'll have at least a 512MB "PCI hole", since the BIOS
needs to be the last part before 4GB and the video buffer must be
aligned to a 256MB (i.e. native size) boundary. If it was a 512MB
video buffer, then you'd be guaranteed to lose at least 1GB.
So, it's easier then you'd imagine to lose 0.5GB or even a full 1GB
in the PCI hole with some high-resource add-in adapters. My server
box loses about 870MB without memory remapping enabled. Though I
guess I asked for it with that 256MB video card and the other add-in
adapters I use.
This is not a Linux vs. MS-Windows nor an AMD vs. Intel thing. It's
just a fact of how the PC architecture works.
Modern Intel and AMD server hardware both support remapping the extra
memory to the high end of the physical address space, but I've never
tried the Intel version, so I don't know if it has any caveats
like the performance hit for the "software" version.
--
Erich Stefan Boleyn