I'm in contact with the upstream maintainer of the cyberjack driver. This is related to hardware specifics and apparently can be avoided by having pcscd running permanently which is the case on Windows and MacOS as well. In that case the device should not enter the unresponsive state. A workaround would be a USB reset but this will reset all other connected USB devices as well which can be an issue.