The patch seems just slightly changes the issue. It's still exists, but probably the time of the lag less than with no patch. Under XOrg it's almost insensible. Now I'm trying with kernel parameters turning vulnerability mitigations off, as Atri proposed. Not sure yet, I need some more time for testing. But the messages about negative offset still in the journal.