Werner, you're really using a weird setup. Two Xservers, both accessing the NVIDIA devices could be part of the problem. But, yeah. Try the latest NVIDIA driver G05/440.61. Maybe it fixes the issue.