I tested kernel version 5.11-rc6 from Kernel:HEAD and the brightness issue still exists. It seems like I should mainline this thing myself. Never expected this from amd, even NVIDIA drivers works great.