This is with an 8500-12 SATA raid controller. I should also mention that using the 32-bit SUSE 9.0 Professional on the same machine does not exhibit the corruption.
Yes, I have this problem as well. I have reported it to 3ware (which you should do too; duplicate bug reports add legitimacy to problems) with no luck. There was some theorization on the x86-64 kernel list a few months ago that it was an IOMMU problem, but I never had luck with tweeking IOMMU parameters.
The problem I saw was that reads/writes with blocksize under 4k-N (where N was 40 bytes or something like that) seemed to work fine, but would return all zeros (or bogus data) in the buffers above 4k-N or in some cases (M*4k)-N. Also, the first 16k (if I remember correctly) of the partition always reads as zero.
The K8W appears to have serious BIOS problems setting up memory mappings (particularly in regard to AGP setup) when 8GB is installed, so drop your RAM to 1GB and see if you have better luck.
After I'm done here at Supercomputing, I'll try to come up with a kernel patch to reorganize memory mappings to sanify Tyan's BIOS's mess for 8GB, but I don't know that this approach will work. I'm hoping it will solve the 3ware problem and my AGP problem simultaneously, but I am used to engaging in wishful thinking. :)
I have been in contact with Tyan about this problem, and have had no useful response so far.
It is worth noting that there are two other AGP/PCIX Opteron boards on display in the AMD booth at Supercomputing (and the Tyan board is absent). You might check out the Gigabyte 7A8WD (1 32 bit, 2 PCIX, AGP, 4 DDR on 1 proc), Iwill DK8X (2 32bit, 3 PCIX, AGP, 8 DDR).