Erich, this was a very very helpful explanation! Thanks a lot! I think I've understood most of the issues. Erich Boleyn wrote
The remapping would take a part at the end of the address space and map that into the RAM behind the PCI hole. New picture for a machine with 4GB of RAM total:
0 ==== 640K - 1GB ==== (PCI hole start) --- 4GB === (4GB + pci hole size)
Yes, that's what I did not understand before. So remapping means just taking the RAM that would be made invisible by the PCI hole and move it to some other place so that it is accessible. First I though that the PCI devices were taking some amount from the memory, so that it would always be lost and didn't understand why it could be available ("suddenly" :-)) when using remapping.
The BIOS could be punting and not trying to map the spaces correctly because of the complexity of MTRR overlap rules. Before it only had cacheable memory below the PCI hole, now it has it on either size (above and below), and this could be making it punt and just mark them all as uncacheable.
I strongly suspect it's just a bug when the remapping is enabled. ... There are multiple MTRRs available (generally) on modern x86 hardware to set memory types. You would want to take one of the available MTRRs and set the range of memory for the video card to be write-combining. ... PCI devices in general need their memory addresses to be marked as uncacheable, so I'm sure that's what the BIOS does by default. That's where they are in the physical address space.
Ok, so I tried to play around with that a little bit. With memory remapping, /proc/mtrr looks like that: I can disable reg3 and reg2, but trying to create some new write-combining mtrr starting at 3.5GB (and recreating uncacheable ones behind the write-combining) always fails. The kernel complains "mtrr: type mismatch for e8000000,8000000 old: write-back new: write-combining" (That also happens already when X starts and loads the fglrx driver) and indeed no new mtrr range appears in /proc/mtrr. So it looks like I can't create overlapping mtrrs when write-back mtrrs are setup for the whole memory from 0-4GB. So I disabled reg0 also (disabling reg1 first crashes the PC immediately) and tried to set up a write-back mtrr at 0 with size 3,5GB, then a write-combining mtrr at 3.5 with size 128MB (my video ram), and the rest up to 4gb as uncachebable. But trying to create the write-back mtrr with echo "base=0x00000000 size=0xe0000000 type=write-back" >| /proc/mtrr the kernel complains that "base(0x0000) is not aligned on a size(0xe0000000) boundary". Do you have an idea what I'm doing wrong here? Even if not, thanks a lot again for your explanation :-) cu, Frank -- Dipl.-Inform. Frank Steiner Web: http://www.bio.ifi.lmu.de/~steiner/ Lehrstuhl f. Bioinformatik Mail: http://www.bio.ifi.lmu.de/~steiner/m/ LMU, Amalienstr. 17 Phone: +49 89 2180-4049 80333 Muenchen, Germany Fax: +49 89 2180-99-4049 * Rekursion kann man erst verstehen, wenn man Rekursion verstanden hat. *