weird mouse problems

Hello people, I have this weird problem with my mouse: When the computer starts up, the mouse cursor (with gpm, in the console) works perfectly, but then, if I switch the virtual terminal or enter and exit X, it gets lagged. It starts working exessivly(sp?) slow. Anyone knows why this might happen? Regards, Carlos.