You need the latest u-boot and the kernel was missing the mutex fix in sifive_errata_patch_func. Please try again with kernel-default-6.3~rc1-18.1.