Thank you for the info. Still, I am puzzled. From hwinfo, the memory table is
memory.1: main memory size kcore mem: 0x200000000 klog mem 0: 0x1eca77000 klog mem 1: 0x0 klog mem: 0x1eca77000 -- <4> BIOS-e820: 0000000000000000 - 000000000009f800 (usable)
-- <4> BIOS-e820: 000000000009f800 - 00000000000a0000 (reserved)
-- <4> BIOS-e820: 00000000000cc000 - 0000000000100000 (reserved)
-- <4> BIOS-e820: 0000000000100000 - 00000000fbf80000 (usable)
-- <4> BIOS-e820: 00000000fbf80000 - 00000000fc000000 (reserved)
-- <4> BIOS-e820: 00000000fec00000 - 00000000fec00400 (reserved)
-- <4> BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
-- <4> BIOS-e820: 00000000fff80000 - 0000000100000000 (reserved)
-- <4> BIOS-e820: 0000000100000000 - 0000000200000000 (usable)
-- <4>kernel direct mapping tables upto 10200000000 @ 8000-11000
bios mem: 0x1fbf1f800
meminfo: 0x1d0a71000
If I read the table correctly, there are about 66 MB that are either
reserved or holes below 4GB. So, bios mem reports 8GB-66MB,
but meminfo reports only 7.5 GB. There is also a 448 MB difference between
klog mem and meminfo. What is the origin of this discrepancy? At this
point, it is mostly curiosity, as I do not expect Celestica to come up with
a fix soon, though I received a posting from a korean guy who says they are
missing close to 1 GB on a 16 GB system. Also, for future reference, it
would be interesting to find out if populating the banks with smaller DIMMS
(say 512 MB) would ameliorate the problem.
Thank you
Alberto Scotti
--On Thursday, March 25, 2004 2:39 AM +0100 Andi Kleen
On Wed, 24 Mar 2004 20:21:58 -0500 ascotti@email.unc.edu wrote:
Hi,
I have recently purchased a quad-opteron. The MB is a Quartet manufactured by Celestica. The system has 8 GB of RAM (8 dimms, 2 GB per node). The puzzling thing is that only 7.5 GB show up, both during the BIOS startup, and by the OS (SLES 8). Playing around with the dimms, I have found that up to 3 GB everything looks normal. Adding a fourth dimm to the system results in an effective memory of 3.5 GB. Adding extra modules increase the count by 1GB as expected. The dimms are good, and nothing changes by switching dimms or populating the nodes differently. The vendor tells me that this is a known "feature" of the AMD chipset, even for Athlons. I looked up the thing, and found nothing. I am curious to see if this is just a problem with the Opteron chipset, or a more general problem. 0.5 GB seems a lot to disappear...
It depends on the BIOS. The hardware can only map a single DIMM to an continuous address. Your BIOS decides to map the first three DIMMs at 0,1GB,2GB. Then there is the PCI memory hole directly below 4GB (PCI memory mappings, AGP aperture, IOMMU etc., can be several hundred MB). Your BIOS could chose to leave 3GB-4GB free and only put the IO mappings there and put the next DIMM at 4GB, but it chose instead to map the DIMM at 3GB and so part of its memory is lost to the IO hole.
There may be subtle reasons why it does that.
Complain to your BIOS vendor about this, there is nothing Linux has to do with it.
-Andi