Unfortunately, the xdm setting issue is not the problem that I am seeing. I have Xorg crashing on me rather hard. regardless of which login manager I use. KDM,GDM crash Xorg as well!
I can do xinit and have X working without any window manager, but manually starting kde or gnome crashes X. Currently, the only error in my Xorg.0.log file regaring the libfb.so module lacking "fbmmx". That made me think about my cpu (A celeron 2.0GA).
Discussed here quite a few times in the last two or three months. A google list search will provide you with the proper procedure to solve this problem.