When booting our TW kernel on a POWER9 system in bare metal or in KVM mode, it
fails to boot because MMIO accesses break. The reason for that is that Cell
support enables indirect MMIO support which hides bits in the address which
does not work in Radix mode.

For full details, I'll paste the email from Michael Ellermann.

The easiest fix for this would be to disable Cell support in our BE kernel.
That way we wouldn't be able to run on Cell systems (does anyone do that
still?), but at least we would be able to fully leverage POWER9 and above
machines.

The alternative to this would be to disable Radix Tree support in our BE
kernel. I doubt that's a good idea though.


---


> qemu-system-ppc64 -nographic -cpu host -enable-kvm -kernel linux -vga 
> none -drive file=linux,if=virtio,snapshot=on -m 1G
>
> with the guest kernel taken from 
> http://download.opensuse.org/ports/ppc/factory/repo/oss/boot/ppc64/linux
>
> The same kernel compiled as ppc64le works correctly.

It looks like you've got CONFIG_PPC_INDIRECT_MMIO enabled, and looking
at the code it simply can't work with Radix. It masks off some of the
high bits of the address to use as a token, but those bits are used as
part of the address with Radix.

You can see in the reg dump that r4 = c00c000080000014  which is an
address from ioremap().

But then we do:
  rldicl  r10,r4,4,12
  rotldi  r10,r10,60
  hwsync
  stb     r3,0(r10)

Which clears bits 59-48 leaving us with c000000080000014, which would be
in the linear mapping except you mustn't have that much memory. But it's
still the wrong address.

You get PPC_INDIRECT_MMIO selected because you still build CELL support 

We might be able to rework the PPC_INDIRECT_MMIO stuff to work with
Radix, but I'll have to see if there are enough bits left for its token
thing.

The other option is we make PPC_INDIRECT_MMIO depend on !RADIX. That
might break Cell if you enable Radix, but possibly only on the IBM Cell
Blades which you probably don't care about anymore.

cheers