I wanted to mention that the issue did not occur now for quite some weeks. Therefore, I think the issue might be solved in newer kernels. Currently running kernel 6.7.0-rc1, but the one before also did not show the issue.