some more details kernel 5.13.8-1 (current TW kernel) 1. kexec from 470.63.01 into system with 470.42.01 - failed kexec 2. power cycle and boot directly into system with 470.42.01 - working kexec with 470.42.01, 3. kexec from 470.42.01 into system with 470.57.02 - breaks already when 470.57.02 tries to boot From the log it looks like thie step was working still with 5.13.2 ``` root@fortress ~ # for boot in 28 27 26 25 24 23 22 21 ; do echo "boot ID: -${boot}" ; journalctl -b -${boot} --no-tail | rg -i "(NVRM: loading NVIDIA UNIX x86_64 Kernel Module|kernel: Linux version|INVALID_DEVICE_REQUEST)" ; done boot ID: -28 Jul 18 02:27:17 fortress kernel: Linux version 5.13.1-1-default (geeko@buildhost) (gcc (SUSE Linux) 11.1.1 20210625 [revision 62bbb113ae68a7e724255e17143520735bcb9ec9], GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.36.1.20210326-4) #1 SMP Mon Jul 12 06:35:58 UTC 2021 (72aabc2) Jul 18 02:27:26 fortress kernel: NVRM: loading NVIDIA UNIX x86_64 Kernel Module 470.42.01 Tue Jun 15 21:26:37 UTC 2021 boot ID: -27 Jul 19 11:55:44 fortress kernel: Linux version 5.13.2-1-default (geeko@buildhost) (gcc (SUSE Linux) 11.1.1 20210625 [revision 62bbb113ae68a7e724255e17143520735bcb9ec9], GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.36.1.20210326-4) #1 SMP Thu Jul 15 03:36:02 UTC 2021 (89416ca) Jul 19 11:56:25 fortress kernel: NVRM: loading NVIDIA UNIX x86_64 Kernel Module 470.42.01 Tue Jun 15 21:26:37 UTC 2021 boot ID: -26 Jul 19 20:17:24 fortress kernel: Linux version 5.13.2-1-default (geeko@buildhost) (gcc (SUSE Linux) 11.1.1 20210625 [revision 62bbb113ae68a7e724255e17143520735bcb9ec9], GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.36.1.20210326-4) #1 SMP Thu Jul 15 03:36:02 UTC 2021 (89416ca) Jul 19 20:17:34 fortress kernel: NVRM: loading NVIDIA UNIX x86_64 Kernel Module 470.57.02 Tue Jul 13 16:14:05 UTC 2021 boot ID: -25 Jul 23 18:16:01 fortress kernel: Linux version 5.13.2-1-default (geeko@buildhost) (gcc (SUSE Linux) 11.1.1 20210625 [revision 62bbb113ae68a7e724255e17143520735bcb9ec9], GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.36.1.20210326-4) #1 SMP Thu Jul 15 03:36:02 UTC 2021 (89416ca) Jul 23 18:16:17 fortress kernel: iommu ivhd0: AMD-Vi: Event logged [INVALID_DEVICE_REQUEST device=0a:00.0 pasid=0x00000 address=0xfffffffdf8000000 flags=0x0a00] Jul 23 18:16:18 fortress kernel: NVRM: loading NVIDIA UNIX x86_64 Kernel Module 470.57.02 Tue Jul 13 16:14:05 UTC 2021 boot ID: -24 Jul 23 18:17:46 fortress kernel: Linux version 5.13.2-1-default (geeko@buildhost) (gcc (SUSE Linux) 11.1.1 20210625 [revision 62bbb113ae68a7e724255e17143520735bcb9ec9], GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.36.1.20210326-4) #1 SMP Thu Jul 15 03:36:02 UTC 2021 (89416ca) Jul 23 18:18:04 fortress kernel: NVRM: loading NVIDIA UNIX x86_64 Kernel Module 470.57.02 Tue Jul 13 16:14:05 UTC 2021 boot ID: -23 Jul 23 18:32:45 fortress kernel: Linux version 5.13.2-1-default (geeko@buildhost) (gcc (SUSE Linux) 11.1.1 20210625 [revision 62bbb113ae68a7e724255e17143520735bcb9ec9], GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.36.1.20210326-4) #1 SMP Thu Jul 15 03:36:02 UTC 2021 (89416ca) Jul 23 18:32:57 fortress kernel: NVRM: loading NVIDIA UNIX x86_64 Kernel Module 470.57.02 Tue Jul 13 16:14:05 UTC 2021 boot ID: -22 Jul 23 18:37:54 fortress kernel: Linux version 5.13.2-1-default (geeko@buildhost) (gcc (SUSE Linux) 11.1.1 20210625 [revision 62bbb113ae68a7e724255e17143520735bcb9ec9], GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.36.1.20210326-4) #1 SMP Thu Jul 15 03:36:02 UTC 2021 (89416ca) Jul 23 18:38:12 fortress kernel: NVRM: loading NVIDIA UNIX x86_64 Kernel Module 470.57.02 Tue Jul 13 16:14:05 UTC 2021 boot ID: -21 Jul 25 03:05:38 fortress kernel: Linux version 5.13.2-1-default (geeko@buildhost) (gcc (SUSE Linux) 11.1.1 20210625 [revision 62bbb113ae68a7e724255e17143520735bcb9ec9], GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.36.1.20210326-4) #1 SMP Thu Jul 15 03:36:02 UTC 2021 (89416ca) Jul 25 03:05:52 fortress kernel: iommu ivhd0: AMD-Vi: Event logged [INVALID_DEVICE_REQUEST device=0a:00.0 pasid=0x00000 address=0xfffffffdf8000000 flags=0x0a00] Jul 25 03:05:52 fortress kernel: NVRM: loading NVIDIA UNIX x86_64 Kernel Module 470.57.02 Tue Jul 13 16:14:05 UTC 2021 ``` the boot ID -25 was the reason why i thought it was a kernel regression in the first place. But further testing seems to show we have at least a partial problem at the nvidia driver. Do we have anyone from nvidia whom we could CC on the bug to get some input from them?