Dima <bryga66@yahoo.com> wrote:
I have S2895 too. I was able to make OS see all the system memory (8 Gb) enabling "memory node interleave" and "memory hole mapping: software" in BIOS. That probably will penalize your performance a bit .. I guess the missing memory is what is called PCI memory hole ...
I have an S2885, and with both the BIOS versions where the "memory hole remapping: software" was available, memory copy bandwidth in the bottom 4 GB (including the remapped portion) was considerably decreased, from about 1.7GB/sec in the non-remapped case, to about 1.2GB/sec. YMMV, but it seems there is either a performance issue with the software remapping in general or a bug with the implementation. I've informed Tyan of the issue. (My server has 8GB physically installed, but only 7170-ish MB is available, with remapping it puts the memory between 3 & 4GB up at the end of the address space, so physical memory appears to go up to the 9GB mark) Note that for the newest Opterons available (including the dual-core chips), there is a new "hardware remapping" feature which may not have the performance hit. I haven't been able to test it but bet it probably doesn't have that problem. Some people seem to know what the "hole before 4GB" is, and others do not, so a short tutoral (which those who know can tune out): -- Hardware with RAM or memory-mapped I/O needs to be able to map it somewhere. -- 32-bit OSes *and* 32-bit addressable hardware (which is the majority of the PCI hardware out there right now) both need to be able to work for the general case. So, a "hole" at 4GB which grows downward depending on the total size of the addresses used is punched. It consists of (usually more or less in this order growing downward from 4GB): -- x86 ROM/BIOS (this really is required to be aligned against the top of the 4GB boundary) -- IO APIC -- PCI/AGP devices as mapped in PCI config space numerically (but not required to be in that order) NOTE each of the PCI domains are required to be aligned to their native size. -- AGP aperture (NOTE: the agp aperture is required to be aligned to it's native size. The point of the "native size alignment" comments above is that, for example, if you have, say a video card with a 256MB video buffer, then at minimum, you'll have at least a 512MB "PCI hole", since the BIOS needs to be the last part before 4GB and the video buffer must be aligned to a 256MB (i.e. native size) boundary. If it was a 512MB video buffer, then you'd be guaranteed to lose at least 1GB. So, it's easier then you'd imagine to lose 0.5GB or even a full 1GB in the PCI hole with some high-resource add-in adapters. My server box loses about 870MB without memory remapping enabled. Though I guess I asked for it with that 256MB video card and the other add-in adapters I use. This is not a Linux vs. MS-Windows nor an AMD vs. Intel thing. It's just a fact of how the PC architecture works. Modern Intel and AMD server hardware both support remapping the extra memory to the high end of the physical address space, but I've never tried the Intel version, so I don't know if it has any caveats like the performance hit for the "software" version. -- Erich Stefan Boleyn <erich@uruk.org> http://www.uruk.org/ "Reality is truly stranger than fiction; Probably why fiction is so popular"