On Wed, Nov 05, 2003 at 01:03:10AM -0800, John McCorquodale wrote:

...

Most likely the aperture is there and you are messing with the IOMMU mappings.

Yeah, I'm investigating that now. Figuring out how to read the aperture setup out of the 8151. I've been talking to Tyan, they don't seem to have a solid AGP configuration picture.

Look in /var/log/boot.msg The kernel reads it and prints it early in boot. There are unfortunately multiple ways to store this (latest kernel should handle all this), it can be put into any northbridge and into the AGP bridge. Windows centric BIOS put it into the AGP bridge.

...

Does it go away when you boot with iommu=memaper=2 (this will make your system lose some memory)

I will try this next time I'm in my office. It's worth noting that I've tried most ATI video cards and the bad performance results are always the same, even with AGP110 cards ('tho the 110 gets slightly faster due presumably to the wider datapath; it's not a factor of two 'tho).

I'll post a big writeup of this; I'm getting it together now but it'll spill into tomorrow (getting sleepy).

But while I've got you here, two questions:

1) It couldn't be that the AGP framebuffer is getting indirected through the IOMMU when I map it through PCI-space or something like that could it?

The frame buffer is normally not redirected through the AGP aperture. AGP aperture/IOMMU is to remap multiple mainboard ram areas to a single continuous bus area as seen from the AGP/PCI side. Frame buffer is a direct mmio area pointing to memory on the card.

2) Any clue why Tyan's BIOS would set the AGP aperture (and surrounding address space to uncachable in the MTRR)?

Setting the AGP aperture and the PCI hole (that is around it) to uncachable is standard and ok. Write combining MTRRs are set by the OS (the Xserver on Linux)

It's also worth noting that changing video cards around changes the size of the entry in the MTRR: sometimes it's 512MB, sometimes it's 1GB. The "missing memory" varies with the size of this MTRR entry.

That is because they have different sized PCI resources that are mapped there.

...

Overall it sounds more like hardware/BIOS issues than a Linux problem. I would recommend to contact Tyan. You could also try a different video card.

I agree that the MTRR/missing memory is probably BIOS. But how is it possible

Missing memory is a probably a minor BIOS problem or maybe unavoidable depending on your memory setup. The memory controller can map a single DIMM only in a single chunk and there has to be holeless memory < 4GB (Windows among others require this). The aperture and MMIO mappings also have to be <4GB. If the BIOS is unable to map the DIMMs completely around the hole containing aperture/MMIO you will lose memory.

to misconfigure hardware to the point that mere accesses to physical address space get so much slower? It's a bus; it's just clocked. It seems to get

Using cached access on a framebuffer/aperture is normally incredible slow. Frame buffer access is best using write combining.

Still in the dark really, but that's not reason not to send a verbose message to the list! :)

Just that you did not cc it to the list... Doing that now because the explanations may be useful to others. -Andi