I have first copied all sound folder, and then removed modules (with a bisect like approach), ran depmod and rebooted afterwards. I could pinpoint the issue to one file sound/soc/snd-soc-core.ko.zst if I copy this from .33 to .38 I am not able to reproduce the problem. Will use the system like this for now if it is the timeout that has changed but I don't think so, it looks like the issue is gone.