Surprise - this has something to do with gdm. If I replace gdm with a different display manager (tried lightdm so far), the problem doesn't occur any more, even if I'm logged in under GNOME.