This effect seems to be caused by the VDPAU setting in VLC. When using VLC and I set the output to XCB, the resolution is back at is intended level. There should be a way to pass this argument from kaffeine to the libVLC, but I have not found a way to do this; any setting for --vout or -V seems to be ignored. Using Radeon RX480 as graphics card (amdgpu).