OK, now I'm building a 5.12.x kernel with the revert of the patch. It's built in OBS home:tiwai:bsc1187472 repo. Please give it a try later (the build finishes usually after an hour or so). The revert wasn't clean and I had to get rid of some other chunk as well, so it's just for verifying whether the issue is relevant with that. The real fix should be done in a better way in the upstream, in anyway.