http://bugzilla.opensuse.org/show_bug.cgi?id=1188745 http://bugzilla.opensuse.org/show_bug.cgi?id=1188745#c2 --- Comment #2 from Marcus R�ckert <mrueckert@suse.com> --- 1. rebuilding didnt help 2. https://documentation.suse.com/sles/15-SP3/html/SLES-all/cha-tuning-kexec.ht... -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1188745 Stefan Dirsch <sndirsch@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |IN_PROGRESS CC| |mrueckert@suse.com Flags| |needinfo?(mrueckert@suse.co | |m) -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1188745 http://bugzilla.opensuse.org/show_bug.cgi?id=1188745#c5 --- Comment #5 from Stefan Dirsch <sndirsch@suse.com> --- Yeah, who knows ... -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1188745 http://bugzilla.opensuse.org/show_bug.cgi?id=1188745#c7 Stefan Dirsch <sndirsch@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jeffm@suse.com --- Comment #7 from Stefan Dirsch <sndirsch@suse.com> --- Jeff Mahoney? -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1188745 http://bugzilla.opensuse.org/show_bug.cgi?id=1188745#c10 --- Comment #10 from Joerg Roedel <jroedel@suse.com> --- (In reply to Marcus R�ckert from comment #0)

Jul 26 01:47:44 fortress kernel: iommu ivhd0: AMD-Vi: Event logged [INVALID_DEVICE_REQUEST device=0a:00.0 pasid=0x00000 address=0xfffffffdf8000000 flags=0x0a00]

This is an interrupt translation request while IRQ remapping is disabled for the device. Do the command line parameters differ between the first and the kexec kernel? There are no AMD IOMMU driver changes between 5.13.2 and 5.13.4, only ARM-SMMU and Intel VT-d fixes. So nothing changed on the AMD driver side.

I can provide full boot log if needed.

What hardware does this happen on? A full boot log might also help. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1188745 http://bugzilla.opensuse.org/show_bug.cgi?id=1188745#c13 --- Comment #13 from Joerg Roedel <jroedel@suse.com> --- I looked through the failure log file, but this didn't give any insights. What I found so far is that the device sends an interrupt request before any IRQs have been configured by the device driver. The IOMMU is configured to still block all interrupt requests from the device. This is the only situation where this can happen (in fact, when IV=1b and IntCtl=00b in the DTE for the device) As there are no AMD IOMMU changes in the range, my guess is that some other change between 5.14.2 and 5.14.4 influenced the behaviour of the Nvidia module, so that it now sends an IRQ when it really shouldn't. There are two options on how to proceed: * Bisect upstream stable kernels between 5.13.2 and 5.13.4 to find the commit which introduced the issue * Work around the problem by booting with 'intremap=off' on the kernel command line. Please let me know what you prefer. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1188745 http://bugzilla.opensuse.org/show_bug.cgi?id=1188745#c15 --- Comment #15 from Joerg Roedel <jroedel@suse.com> --- (In reply to Marcus R�ckert from comment #14)

bisect would mean rebuilding the kernel over and over and rebooting the machine for each kernel?

Yes, there are 610 commits between 5.13.2 and 5.13.4, so ~9-10 rounds of compiling/testing. I don't see another way to find the offending commit, sorry. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1188745 http://bugzilla.opensuse.org/show_bug.cgi?id=1188745#c17 --- Comment #17 from Marcus R�ckert <mrueckert@suse.com> --- 5.14.0~rc6 doesnt fix it either. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1188745 http://bugzilla.opensuse.org/show_bug.cgi?id=1188745#c19 --- Comment #19 from Stefan Dirsch <sndirsch@suse.com> --- Hmm. The issue already occurred with 5.13.4! Or did you make some backports from 5.14 to 5.13.4, which may explain this behaviour? I could ask nVidia if they can reproduce the issue on Kernel >= 5.13.4 or 5.14 (if needed). I'm sure if they do tests with kexec at all ... maybe they never supported it. -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1188745 http://bugzilla.opensuse.org/show_bug.cgi?id=1188745#c27 Marcus R�ckert <mrueckert@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(mrueckert@suse.co | |m) | --- Comment #27 from Marcus R�ckert <mrueckert@suse.com> --- ``` Feb 25 11:48:22 fortress kernel: nvidia-nvlink: Nvlink Core is being initialized, major device number 235 Feb 25 11:48:22 fortress kernel: Feb 25 11:48:22 fortress kernel: nvidia 0000:0a:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none Feb 25 11:48:22 fortress kernel: iommu ivhd0: AMD-Vi: Event logged [INVALID_DEVICE_REQUEST device=0a:00.0 pasid=0x00000 address=0xfffffffdf8000000 flags=0x0a00] Feb 25 11:48:22 fortress kernel: NVRM: loading NVIDIA UNIX x86_64 Kernel Module 510.54 Tue Feb 8 04:42:21 UTC 2022 Feb 25 11:48:23 fortress kernel: nvidia_uvm: module uses symbols from proprietary module nvidia, inheriting taint. Feb 25 11:48:23 fortress kernel: nvidia-uvm: Loaded the UVM driver, major device number 511. Feb 25 11:48:22 fortress kernel: nvidia-nvlink: Nvlink Core is being initialized, major device number 235 Feb 25 11:48:22 fortress kernel: nvidia 0000:0a:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none Feb 25 11:48:22 fortress kernel: iommu ivhd0: AMD-Vi: Event logged [INVALID_DEVICE_REQUEST device=0a:00.0 pasid=0x00000 address=0xfffffffdf8000000 flags=0x0a00] Feb 25 11:48:22 fortress kernel: NVRM: loading NVIDIA UNIX x86_64 Kernel Module 510.54 Tue Feb 8 04:42:21 UTC 2022 Feb 25 11:48:23 fortress kernel: nvidia_uvm: module uses symbols from proprietary module nvidia, inheriting taint. Feb 25 11:48:23 fortress kernel: nvidia-uvm: Loaded the UVM driver, major device number 511. Feb 25 11:48:23 fortress kernel: nvidia-modeset: Loading NVIDIA Kernel Mode Setting Driver for UNIX platforms 510.54 Tue Feb 8 04:34:06 UTC 2022 Feb 25 11:48:23 fortress kernel: [drm] [nvidia-drm] [GPU ID 0x00000a00] Loading driver Feb 25 11:48:27 fortress kernel: NVRM: GPU 0000:0a:00.0: RmInitAdapter failed! (0x23:0x65:1401) Feb 25 11:48:27 fortress kernel: NVRM: GPU 0000:0a:00.0: rm_init_adapter failed, device minor number 0 Feb 25 11:48:27 fortress kernel: [drm:nv_drm_load [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000a00] Failed to allocate NvKmsKapiDevice Feb 25 11:48:27 fortress kernel: [drm:nv_drm_probe_devices [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000a00] Failed to register device ``` so no -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1188745 http://bugzilla.opensuse.org/show_bug.cgi?id=1188745#c31 Peter S�tterlin <P.Suetterlin@royac.iac.es> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |P.Suetterlin@royac.iac.es --- Comment #31 from Peter S�tterlin <P.Suetterlin@royac.iac.es> --- Not sure if it is related in any way, but I do get the same error after upgrading to kernel 6.0 / TW 20221012: kernel: nvidia-modeset: Loading NVIDIA Kernel Mode Setting Driver for UNIX platforms 470.141.03 Thu Jun 30 18:34:41 UTC 2022 kernel: [drm] [nvidia-drm] [GPU ID 0x00000200] Loading driver kernel: NVRM: GPU 0000:02:00.0: Failed to copy vbios to system memory. kernel: NVRM: GPU 0000:02:00.0: RmInitAdapter failed! (0x30:0xffff:874) kernel: NVRM: GPU 0000:02:00.0: rm_init_adapter failed, device minor number 0 kernel: [drm:nv_drm_load [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000200] Failed to allocate NvKmsKapiDevice kernel: [drm:nv_drm_probe_devices [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000200] Failed to register device The card is 02:00.0 3D controller: NVIDIA Corporation GM108M [GeForce 940MX] (rev a2) (That's a Mobile / Optimus chip) -- You are receiving this mail because: You are on the CC list for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1188745 http://bugzilla.opensuse.org/show_bug.cgi?id=1188745#c35 --- Comment #35 from Peter S�tterlin <P.Suetterlin@royac.iac.es> --- (In reply to Stefan Dirsch from comment #34)

Hmm. Of course you could try with

options nvidia-drm modeset=0

in /etc/modprobe.d/50-nvidia-default.conf. But with that you can't use Wayland any longer. But since you're using suse-prime you're apparently still using X anyway.

Hmm, tried that, but it is still always loading nvidia-modeset!? Needless to say it still won't work: [drm:nv_drm_load [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000200] Failed to allocate NvKmsKapiDevice [drm:nv_drm_probe_devices [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000200] Failed to register device (With latest TW and kernel 6.0.3). Should I open a separate bug? I had only posted here because it's the same error message... -- You are receiving this mail because: You are on the CC list for the bug.