http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c2 --- Comment #2 from Graeme Murray --- Created attachment 857540 --> http://bugzilla.opensuse.org/attachment.cgi?id=857540&action=edit hwinfo for 5.17 sorry for delay, info as requested -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c4 --- Comment #4 from Graeme Murray --- Created attachment 857542 --> http://bugzilla.opensuse.org/attachment.cgi?id=857542&action=edit journal log for 5.17 -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c6 Graeme Murray changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(graeme.w.murray@g | |mail.com) | --- Comment #6 from Graeme Murray --- Created attachment 857545 --> http://bugzilla.opensuse.org/attachment.cgi?id=857545&action=edit no_console_suspend -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c8 --- Comment #8 from Graeme Murray --- Created attachment 857577 --> http://bugzilla.opensuse.org/attachment.cgi?id=857577&action=edit journal log for 5.17.1-2 Sorry, misreported, hibernate is working on 5.17 on my machine. It is only sleep which is not working. I have tried 5.17.1-2 and attached the journal log. Same issue with sleep. As before, looks like machine enters sleep correctly, power led blinks, as normal. After moving the mouse, power led does go solid but nothing else happens so only option is to reboot. I have had a look at the kernel configs for both 5.16 and 5.17 can't see anything which makes a difference. More than happy to try anything which might help. Thanks, -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c10 Graeme Murray changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(graeme.w.murray@g | |mail.com) | --- Comment #10 from Graeme Murray --- OK, tried kernel-vanilla from your repo, same result. Power led goes solid and fan starts, no display, it stays on standby. my keyboard (logi mx keys) does not have sysrq, (that I can find) so can't test sysrq +S or +B. I will go on a hunt for another keyboard and let you know if I can find one. -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c12 --- Comment #12 from Takashi Iwai --- And, let's check with nomodeset boot option. This will boot without the native graphics (amdgpu), driving with EFI framebuffer. It might be that it won't reach to the graphics environment, but it's fine with Linux console. There you can run manually like "systemctl suspend". At resume, you'll likely lose the graphics screen, but the system should still live. You can type blindly or reach via network there. e.g. run a command (e.g. "systemctl suspend" again) and see whether it works as expected. If the suspend/resume works without graphics, it's narrowed to be a problem of amdgpu. If it still hangs up, it's something else. -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c14 --- Comment #14 from Graeme Murray --- Ok same again, boot with nomodeset, got graphics desktop just a little big. Same result after sleep. -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c16 --- Comment #16 from Graeme Murray --- OK, managed to get Alt+prtscr+b working when not in sleep, so believe key seq is working. Tried it again when machine sleeping, no change, tried caps-lock, no led, then tried Alt+prtscr+o again no change. so looks like machine is not responding, (it's dead Jim). -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c18 --- Comment #18 from Takashi Iwai --- Hm, it might be some watchdog, but it doesn't explain why it happens with the newer kernel... Maybe we should try kdump for capturing the crash. Could you try to set up kdump (e.g. via YaST)? You likely need to give a higher RAM size than suggested there. After setting it up and reboot, try manually "alt-sysrq-c" combo. This will trigger a crash and it should result in a crash dump under /var/crash/*. If dmesg file is found there, it's good. Then we can try suspend/resume, and check whether we get either a dmesg file or a vmcore there. -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c20 --- Comment #20 from Takashi Iwai --- vmcore is too large to attach, please upload only dmesg. -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c22 --- Comment #22 from Graeme Murray --- trying to upload dmesg.txt but get "server internal error....., please contact opensuse support" ? is there someone I can report this too? -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c24 --- Comment #24 from Graeme Murray --- Thanks, email on it's way -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c26 Takashi Iwai changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |needinfo?(graeme.w.murray@g | |mail.com) --- Comment #26 from Takashi Iwai --- Thanks. The critical hit was: [ 251.498921] Enabling non-boot CPUs ... [ 251.498921] ------------[ cut here ]------------ [ 251.498921] kernel BUG at kernel/time/timer.c:1965! And it's the code: int timers_dead_cpu(unsigned int cpu) { struct timer_base *old_base; struct timer_base *new_base; int b, i; ==> BUG_ON(cpu_online(cpu)); It means that the given CPU is already online while it shouldn't be. But it's not clear why the teardown is called at this stage. Maybe another step failed and rolling back caused the problem. I'm building a test kernel with a debug patch in OBS home:tiwai:bsc1197558 repo. Once after the build finishes (takes an hour or so), it'll appear at http://download.opensuse.org/repositories/home:/tiwai:/bsc1197558/standard/ This replaced BUG_ON() with WARN_ON_ONCE(), so it shouldn't stop at the point. Please test with it and give the dmesg output after suspend/resume. -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c28 --- Comment #28 from Takashi Iwai --- BTW, another kernel WARNING before the BUG line is likely a different problem, but maybe a spurious issue. It seems hitting on many others, too: https://bugzilla.redhat.com/show_bug.cgi?id=2048838 -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c30 --- Comment #30 from Takashi Iwai --- (In reply to Graeme Murray from comment #29)

OK, thanks. This would affect VMs I guess?

Not sure, I guess it's a spurious issue that happens only at resume, but we need to check once when we get a working resume. -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c32 Takashi Iwai changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bpetkov@suse.com --- Comment #32 from Takashi Iwai --- Thanks. Hrm, it hits a failure at a different point now. The best we can do at this moment is to perform git-bisect, I guess. Can you try it in your side? Meanwhile, let's add Boris to Cc, as it might be his area. Boris, it's a regression between 5.16 and 5.17. Judging from two crash logs, thaw_secondary_cpus() at resume hits weird bugs, eventually leading to a BUG_ON() somewhere. It seems happening only with suspend-to-RAM, not with hibernation. -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c34 --- Comment #34 from Borislav Petkov --- That thing has a proprietary module in there: ... loaded Tainted: P OE 5.17.1-3.g58205bc-default... ... wl(POE) amdgpu r8168(OE)... [ 2.052104] wl: module license 'MIXED/Proprietary' taints kernel. [ 2.052108] Disabling lock debugging due to kernel taint And that thing warns first in the AMD iommu code and the warning adding commit a8d4a37d1bb9 ("iommu/amd: Restore GA log/tail pointer on host resume") talks about some guest vapic log getting a chance to resume. The timer BUG_ON() points to a CPU not coming online, presumably CPU1 or so. So, can you reproduce the same thing with that wl module completely removed from your system? -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c36 --- Comment #36 from Graeme Murray --- Created attachment 857705 --> http://bugzilla.opensuse.org/attachment.cgi?id=857705&action=edit crash dmesg.txt, r8168 removed ok, removed r8168, same test as before. -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c38 --- Comment #38 from Graeme Murray --- Sorry but I have no idea what the wl module is? -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c40 --- Comment #40 from Graeme Murray --- ok, so it's the broadcom wireless lan driver. Can i modprobe -r it or should I deinstall completely? I guess I will not have a network connection after. -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c42 --- Comment #42 from Borislav Petkov --- Ok, it still explodes even with the broadcom driver removed. Now, add "amd_iommu=off" to the kernel command line in /boot/grub/grub.cfg - this will be there temporarily until the next time you install a kernel but it should be enough to test - and do the exact same thing. Thx. -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c44 Borislav Petkov changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jroedel@suse.com Flags| |needinfo?(jroedel@suse.com) --- Comment #44 from Borislav Petkov --- Thanks, apparently, one can't simply disable the iommu. Ok, back to the previous situation. Joerg, pls have a look at comment #41, there's: WARNING: CPU: 0 PID: 3514 at drivers/iommu/amd/init.c:851 amd_iommu_enable_interrupts+0x336/0x410 Something about interrupt remapping and something GA_LOG timeoutting. Also, Graeme, I see you have BIOS from almost two years ago: BIOS AF5NV701.25 12/15/2020 You could try updating it to see whether the IOMMU issue is fixed. Might be related to the resume failure... Thx. -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c46 --- Comment #46 from Takashi Iwai --- (In reply to Takashi Iwai from comment #45)

Note that the iommu problem is a fairly oft-seen problem, see the commit 28.

I meant comment 28. -- You are receiving this mail because: You are the assignee for the bug.

http://bugzilla.opensuse.org/show_bug.cgi?id=1197558 http://bugzilla.opensuse.org/show_bug.cgi?id=1197558#c48 --- Comment #48 from Borislav Petkov --- Ah, interesting. So it looks like with the kernel getting new features constantly and using them, the BIOS needs to "catch up", so to speak. Unfortunately, one can't know what got changed to fix your problem. Oh well... -- You are receiving this mail because: You are the assignee for the bug.