Sorry for yet another comment, but it seems I can enter it there but it may not work. Entering it on the console prompt before the switch happens does work though. Thankfully my passphrase is not too long to be entered in that short time.