[Bug 1212833] New: Linux 6.3.9: Kernel panic at boot needing hard power-off
https://bugzilla.suse.com/show_bug.cgi?id=1212833 Bug ID: 1212833 Summary: Linux 6.3.9: Kernel panic at boot needing hard power-off Classification: openSUSE Product: openSUSE Tumbleweed Version: Current Hardware: Other OS: Other Status: NEW Severity: Major Priority: P5 - None Component: Kernel Assignee: kernel-bugs@opensuse.org Reporter: badshah400@gmail.com QA Contact: qa-bugs@suse.de Target Milestone: --- Found By: --- Blocker: --- Created attachment 867873 --> https://bugzilla.suse.com/attachment.cgi?id=867873&action=edit Output from `journalctl -b -1 --no-hostname` At boot (usually after booting into Windows and rebooting), my ThinkPad P14s Gen2 laptop has started to freeze with the Caps Lock LED blinking on and off repeatedly indicative of a kernel panic. I have to eventually hard power off the machine and try booting again. This happens at boot often, but not always, since the update to Kernel 6.3.9 on Tumbleweed "20230626", and had never happened before. The following messages are logged to the journal (full journal attached): ----------------------------------------------------
Jun 29 16:08:54 kernel: mt7921e 0000:03:00.0: WM Firmware Version: ____010000, Build Time: 20230526130958 Jun 29 16:08:54 kernel: BUG: kernel NULL pointer dereference, address: 000000000000005a Jun 29 16:08:54 kernel: #PF: supervisor read access in kernel mode Jun 29 16:08:54 kernel: #PF: error_code(0x0000) - not-present page Jun 29 16:08:54 kernel: PGD 0 P4D 0 Jun 29 16:08:54 kernel: Oops: 0000 [#1] PREEMPT SMP NOPTI Jun 29 16:08:54 kernel: CPU: 11 PID: 1119 Comm: irqbalance Not tainted 6.3.9-1-default #1 openSUSE Tumbleweed 4b767630dbc263131e96e89ef291fd4fd2951892 Jun 29 16:08:54 kernel: Hardware name: LENOVO 21A0CTO1WW/21A0CTO1WW, BIOS R1MET54W (1.24 ) 05/15/2023 Jun 29 16:08:54 kernel: RIP: 0010:show_interrupts+0x24c/0x340 Jun 29 16:08:54 kernel: Code: 85 d2 74 0f 48 c7 c6 c1 1c fa b9 48 89 ef e8 db 0a 2c 00 49 8b 5c 24 70 48 85 db 74 29 48 8b 53 50 48 c7 c6 c7 1c fa b9 eb 0b <48> 8b 53 50 48 c7 c6 9c 49 fd b9 48 89 ef e8 b1 0a 2c 00 48 8b 5b Jun 29 16:08:54 kernel: RSP: 0018:ffffa5e801b07cd8 EFLAGS: 00010006 Jun 29 16:08:54 kernel: RAX: 0000000000000000 RBX: 000000000000000a RCX: ffff0a00ffffff04 Jun 29 16:08:54 kernel: RDX: 0000000000001000 RSI: 0000000000000005 RDI: 0000000047c8f0a3 Jun 29 16:08:54 kernel: RBP: ffff96fd07dd18e8 R08: 0000000000000005 R09: ffff96fc47c8f0a4 Jun 29 16:08:54 kernel: R10: ffffffffffffffff R11: 0000000000000000 R12: ffff96fb40185200 Jun 29 16:08:54 kernel: R13: 0000000000000246 R14: ffff96fb401852a4 R15: ffffa5e801b07e10 Jun 29 16:08:54 kernel: FS: 00007f4ab1feb780(0000) GS:ffff970152180000(0000) knlGS:0000000000000000 Jun 29 16:08:54 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jun 29 16:08:54 kernel: CR2: 000000000000005a CR3: 0000000136220000 CR4: 0000000000750ee0 Jun 29 16:08:54 kernel: PKRU: 55555554 Jun 29 16:08:54 kernel: Call Trace: Jun 29 16:08:54 kernel: <TASK> Jun 29 16:08:54 kernel: ? __die+0x23/0x70 Jun 29 16:08:54 kernel: ? page_fault_oops+0x14d/0x490 Jun 29 16:08:54 kernel: ? number+0x320/0x3b0 Jun 29 16:08:54 kernel: ? exc_page_fault+0x6e/0x150 Jun 29 16:08:54 kernel: ? asm_exc_page_fault+0x26/0x30 Jun 29 16:08:54 kernel: ? show_interrupts+0x24c/0x340 Jun 29 16:08:54 kernel: ? show_interrupts+0x25f/0x340 Jun 29 16:08:54 kernel: seq_read_iter+0x123/0x480 Jun 29 16:08:54 kernel: ? apparmor_file_permission+0x70/0x170 Jun 29 16:08:54 kernel: proc_reg_read_iter+0x51/0x90 Jun 29 16:08:54 kernel: vfs_read+0x1f8/0x2d0 Jun 29 16:08:54 kernel: ksys_read+0x67/0xe0 Jun 29 16:08:54 kernel: do_syscall_64+0x60/0x90 Jun 29 16:08:54 kernel: ? handle_mm_fault+0x11e/0x310 Jun 29 16:08:54 kernel: ? do_user_addr_fault+0x1e0/0x720 Jun 29 16:08:54 kernel: ? do_syscall_64+0x6c/0x90 Jun 29 16:08:54 kernel: ? exc_page_fault+0x6e/0x150 Jun 29 16:08:54 kernel: entry_SYSCALL_64_after_hwframe+0x72/0xdc Jun 29 16:08:54 kernel: RIP: 0033:0x7f4ab219a091 Jun 29 16:08:54 kernel: Code: 00 0f 1f 84 00 00 00 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 90 90 80 3d ed 2a 0f 00 00 74 13 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 57 c3 66 0f 1f 44 00 00 48 83 ec 28 48 89 54 Jun 29 16:08:54 kernel: RSP: 002b:00007ffd49167138 EFLAGS: 00000246 ORIG_RAX: 0000000000000000 Jun 29 16:08:54 kernel: RAX: ffffffffffffffda RBX: 0000558d6c5592a0 RCX: 00007f4ab219a091 Jun 29 16:08:54 kernel: RDX: 0000000000000400 RSI: 0000558d6c559500 RDI: 0000000000000003 Jun 29 16:08:54 kernel: RBP: 00007f4ab227b660 R08: 0000558d6c55e7b0 R09: 00007f4ab227ece0 Jun 29 16:08:54 kernel: R10: 0000000000000040 R11: 0000000000000246 R12: 0000000000000000 Jun 29 16:08:54 kernel: R13: 0000000000000a68 R14: 00007f4ab227ad60 R15: 0000000000000a68 Jun 29 16:08:54 kernel: </TASK> Jun 29 16:08:54 kernel: Modules linked in: uvcvideo videobuf2_vmalloc uvc videobuf2_memops videobuf2_v4l2 videodev videobuf2_common mc binfmt_misc nls_iso8859_1 nls_cp437 vfat fat mt7921e snd_acp3x_pdm_dma snd_soc_dmic snd_acp3x_rn mt7921_common snd_sof_amd_rembrandt mt76_connac_lib snd_sof_amd_renoir snd_sof_amd_acp mt76 snd_sof_pci snd_sof_xtensa_dsp snd_ctl_led snd_sof snd_hda_codec_realtek snd_sof_utils snd_hda_codec_generic snd_hda_codec_hdmi snd_soc_core btusb btrtl snd_hda_intel btbcm btintel snd_intel_dspcfg snd_compress snd_intel_sdw_acpi btmtk snd_pcm_dmaengine snd_hda_codec mac80211 snd_pci_ps snd_rpl_pci_acp6x snd_hda_core bluetooth intel_rapl_msr snd_acp_pci snd_hwdep intel_rapl_common thinkpad_acpi snd_pci_acp6x r8169 snd_pcm ledtrig_audio libarc4 snd_pci_acp5x realtek edac_mce_amd snd_timer snd_rn_pci_acp3x platform_profile think_lmi(+) mdio_devres efi_pstore pcspkr snd_acp_config xfs ecdh_generic snd_soc_acpi snd nxp_nci_i2c nxp_nci firmware_attributes_class cfg80211 wmi_bmof tiny_power_button i2c_piix4 Jun 29 16:08:54 kernel: snd_pci_acp3x k10temp libphy thermal nci soundcore ac nfc rfkill amd_pmc button i2c_scmi acpi_cpufreq joydev fuse configfs dmi_sysfs ip_tables x_tables mmc_block amdgpu crct10dif_pclmul crc32_pclmul polyval_clmulni polyval_generic gf128mul i2c_algo_bit drm_ttm_helper ttm iommu_v2 ghash_clmulni_intel drm_buddy xhci_pci gpu_sched rtsx_pci_sdmmc sha512_ssse3 xhci_pci_renesas mmc_core xhci_hcd drm_display_helper nvme aesni_intel crypto_simd usbcore cryptd cec ucsi_acpi rtsx_pci nvme_core typec_ucsi sp5100_tco rc_core roles video typec battery wmi serio_raw btrfs blake2b_generic xor raid6_pq libcrc32c crc32c_intel sg dm_multipath dm_mod scsi_dh_rdac scsi_dh_emc scsi_dh_alua msr kvm_amd ccp kvm irqbypass efivarfs Jun 29 16:08:54 kernel: CR2: 000000000000005a Jun 29 16:08:54 kernel: ---[ end trace 0000000000000000 ]--- Jun 29 16:08:54 kernel: RIP: 0010:show_interrupts+0x24c/0x340 Jun 29 16:08:54 kernel: Code: 85 d2 74 0f 48 c7 c6 c1 1c fa b9 48 89 ef e8 db 0a 2c 00 49 8b 5c 24 70 48 85 db 74 29 48 8b 53 50 48 c7 c6 c7 1c fa b9 eb 0b <48> 8b 53 50 48 c7 c6 9c 49 fd b9 48 89 ef e8 b1 0a 2c 00 48 8b 5b Jun 29 16:08:54 kernel: RSP: 0018:ffffa5e801b07cd8 EFLAGS: 00010006 Jun 29 16:08:54 kernel: RAX: 0000000000000000 RBX: 000000000000000a RCX: ffff0a00ffffff04 Jun 29 16:08:54 kernel: RDX: 0000000000001000 RSI: 0000000000000005 RDI: 0000000047c8f0a3 Jun 29 16:08:54 kernel: RBP: ffff96fd07dd18e8 R08: 0000000000000005 R09: ffff96fc47c8f0a4 Jun 29 16:08:54 kernel: R10: ffffffffffffffff R11: 0000000000000000 R12: ffff96fb40185200 Jun 29 16:08:54 kernel: R13: 0000000000000246 R14: ffff96fb401852a4 R15: ffffa5e801b07e10 Jun 29 16:08:54 kernel: FS: 00007f4ab1feb780(0000) GS:ffff970152180000(0000) knlGS:0000000000000000 Jun 29 16:08:54 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jun 29 16:08:54 kernel: CR2: 000000000000005a CR3: 0000000136220000 CR4: 0000000000750ee0 Jun 29 16:08:54 kernel: PKRU: 55555554 Jun 29 16:08:54 kernel: note: irqbalance[1119] exited with irqs disabled Jun 29 16:08:54 kernel: note: irqbalance[1119] exited with preempt_count 1 Jun 29 16:08:54 kernel: ------------[ cut here ]------------ Jun 29 16:08:54 kernel: Voluntary context switch within RCU read-side critical section! Jun 29 16:08:54 kernel: WARNING: CPU: 11 PID: 1119 at kernel/rcu/tree_plugin.h:318 rcu_note_context_switch+0x5e6/0x640 Jun 29 16:08:54 kernel: Modules linked in: uvcvideo videobuf2_vmalloc uvc videobuf2_memops videobuf2_v4l2 videodev videobuf2_common mc binfmt_misc nls_iso8859_1 nls_cp437 vfat fat mt7921e snd_acp3x_pdm_dma snd_soc_dmic snd_acp3x_rn mt7921_common snd_sof_amd_rembrandt mt76_connac_lib snd_sof_amd_renoir snd_sof_amd_acp mt76 snd_sof_pci snd_sof_xtensa_dsp snd_ctl_led snd_sof snd_hda_codec_realtek snd_sof_utils snd_hda_codec_generic snd_hda_codec_hdmi snd_soc_core btusb btrtl snd_hda_intel btbcm btintel snd_intel_dspcfg snd_compress snd_intel_sdw_acpi btmtk snd_pcm_dmaengine snd_hda_codec mac80211 snd_pci_ps snd_rpl_pci_acp6x snd_hda_core bluetooth intel_rapl_msr snd_acp_pci snd_hwdep intel_rapl_common thinkpad_acpi snd_pci_acp6x r8169 snd_pcm ledtrig_audio libarc4 snd_pci_acp5x realtek edac_mce_amd snd_timer snd_rn_pci_acp3x platform_profile think_lmi(+) mdio_devres efi_pstore pcspkr snd_acp_config xfs ecdh_generic snd_soc_acpi snd nxp_nci_i2c nxp_nci firmware_attributes_class cfg80211 wmi_bmof tiny_power_button i2c_piix4 Jun 29 16:08:54 kernel: snd_pci_acp3x k10temp libphy thermal nci soundcore ac nfc rfkill amd_pmc button i2c_scmi acpi_cpufreq joydev fuse configfs dmi_sysfs ip_tables x_tables mmc_block amdgpu crct10dif_pclmul crc32_pclmul polyval_clmulni polyval_generic gf128mul i2c_algo_bit drm_ttm_helper ttm iommu_v2 ghash_clmulni_intel drm_buddy xhci_pci gpu_sched rtsx_pci_sdmmc sha512_ssse3 xhci_pci_renesas mmc_core xhci_hcd drm_display_helper nvme aesni_intel crypto_simd usbcore cryptd cec ucsi_acpi rtsx_pci nvme_core typec_ucsi sp5100_tco rc_core roles video typec battery wmi serio_raw btrfs blake2b_generic xor raid6_pq libcrc32c crc32c_intel sg dm_multipath dm_mod scsi_dh_rdac scsi_dh_emc scsi_dh_alua msr kvm_amd ccp kvm irqbypass efivarfs Jun 29 16:08:54 kernel: CPU: 11 PID: 1119 Comm: irqbalance Tainted: G D 6.3.9-1-default #1 openSUSE Tumbleweed 4b767630dbc263131e96e89ef291fd4fd2951892 Jun 29 16:08:54 kernel: Hardware name: LENOVO 21A0CTO1WW/21A0CTO1WW, BIOS R1MET54W (1.24 ) 05/15/2023 Jun 29 16:08:54 kernel: RIP: 0010:rcu_note_context_switch+0x5e6/0x640 Jun 29 16:08:54 kernel: Code: 00 00 00 00 0f 85 31 fd ff ff 49 89 84 24 a0 00 00 00 e9 24 fd ff ff 48 c7 c7 88 86 01 ba c6 05 a3 ef f2 01 01 e8 3a af f4 ff <0f> 0b e9 6d fa ff ff c6 43 11 00 48 8b 73 20 ba 01 00 00 00 48 8b Jun 29 16:08:54 kernel: RSP: 0018:ffffa5e801b07c40 EFLAGS: 00010086 Jun 29 16:08:54 kernel: RAX: 0000000000000000 RBX: ffff9701521bab40 RCX: 0000000000000027 Jun 29 16:08:54 kernel: RDX: ffff9701521a74c8 RSI: 0000000000000001 RDI: ffff9701521a74c0 Jun 29 16:08:54 kernel: RBP: 0000000000000000 R08: 0000000000000000 R09: ffffa5e801b07ae8 Jun 29 16:08:54 kernel: R10: 0000000000000003 R11: ffff970151afffe8 R12: ffff9701521b9d00 Jun 29 16:08:54 kernel: R13: ffff96fb84762900 R14: ffffffffba9aec40 R15: ffff96fb43d69470 Jun 29 16:08:54 kernel: FS: 0000000000000000(0000) GS:ffff970152180000(0000) knlGS:0000000000000000 Jun 29 16:08:54 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jun 29 16:08:54 kernel: CR2: 000000000000005a CR3: 00000002c7636000 CR4: 0000000000750ee0 Jun 29 16:08:54 kernel: PKRU: 55555554 Jun 29 16:08:54 kernel: Call Trace: Jun 29 16:08:54 kernel: <TASK> Jun 29 16:08:54 kernel: ? rcu_note_context_switch+0x5e6/0x640 Jun 29 16:08:54 kernel: ? __warn+0x81/0x130 Jun 29 16:08:54 kernel: ? rcu_note_context_switch+0x5e6/0x640 Jun 29 16:08:54 kernel: ? report_bug+0x171/0x1a0 Jun 29 16:08:54 kernel: ? handle_bug+0x3c/0x80 Jun 29 16:08:54 kernel: ? exc_invalid_op+0x17/0x70 Jun 29 16:08:54 kernel: ? asm_exc_invalid_op+0x1a/0x20 Jun 29 16:08:54 kernel: ? rcu_note_context_switch+0x5e6/0x640 Jun 29 16:08:54 kernel: __schedule+0xb0/0x1490 Jun 29 16:08:54 kernel: ? __slab_free+0xc4/0x300 Jun 29 16:08:54 kernel: ? select_task_rq_fair+0x1ae/0x1a00 Jun 29 16:08:54 kernel: schedule+0x5e/0xd0 Jun 29 16:08:54 kernel: schedule_preempt_disabled+0x15/0x30 Jun 29 16:08:54 kernel: __mutex_lock.constprop.0+0x403/0x710 Jun 29 16:08:54 kernel: ? ttwu_queue_wakelist+0xef/0x110 Jun 29 16:08:54 kernel: synchronize_rcu_expedited+0x432/0x740 Jun 29 16:08:54 kernel: ? xas_load+0xe/0x50 Jun 29 16:08:54 kernel: ? wake_up_q+0x4e/0x90 Jun 29 16:08:54 kernel: ? rwsem_wake.isra.0+0x69/0x90 Jun 29 16:08:54 kernel: namespace_unlock+0xd2/0x1a0 Jun 29 16:08:54 kernel: put_mnt_ns+0x6d/0x90 Jun 29 16:08:54 kernel: free_nsproxy+0x1b/0x1b0 Jun 29 16:08:54 kernel: do_exit+0x334/0xa70 Jun 29 16:08:54 kernel: make_task_dead+0x81/0x170 Jun 29 16:08:54 kernel: rewind_stack_and_make_dead+0x17/0x20 Jun 29 16:08:54 kernel: RIP: 0033:0x7f4ab219a091 Jun 29 16:08:54 kernel: Code: Unable to access opcode bytes at 0x7f4ab219a067. Jun 29 16:08:54 kernel: RSP: 002b:00007ffd49167138 EFLAGS: 00000246 ORIG_RAX: 0000000000000000 Jun 29 16:08:54 kernel: RAX: ffffffffffffffda RBX: 0000558d6c5592a0 RCX: 00007f4ab219a091 Jun 29 16:08:54 kernel: RDX: 0000000000000400 RSI: 0000558d6c559500 RDI: 0000000000000003 Jun 29 16:08:54 kernel: RBP: 00007f4ab227b660 R08: 0000558d6c55e7b0 R09: 00007f4ab227ece0 Jun 29 16:08:54 kernel: R10: 0000000000000040 R11: 0000000000000246 R12: 0000000000000000 Jun 29 16:08:54 kernel: R13: 0000000000000a68 R14: 00007f4ab227ad60 R15: 0000000000000a68 Jun 29 16:08:54 kernel: </TASK> Jun 29 16:08:54 kernel: ---[ end trace 0000000000000000 ]--- Jun 29 16:08:54 polkitd[1128]: Finished loading, compiling and executing 6 rules Jun 29 16:08:54 polkitd[1128]: Acquired the name org.freedesktop.PolicyKit1 on the system bus Jun 29 16:08:54 (udev-worker)[702]: video2: Failed to create/update device symlink '/dev/video2', ignoring: File exists Jun 29 16:08:55 kernel: mt7921e 0000:03:00.0 wlp3s0: renamed from wlan0 Jun 29 16:08:57 kernel: Bluetooth: hci0: Device setup in 3177859 usecs Jun 29 16:08:57 kernel: Bluetooth: hci0: HCI Enhanced Setup Synchronous Connection command is advertised, but not supported. Jun 29 16:08:57 kernel: Bluetooth: hci0: AOSP extensions version v1.00 Jun 29 16:08:57 kernel: Bluetooth: hci0: AOSP quality report is supported Jun 29 16:09:12 nscd[1134]: 1134 checking for monitored file `/etc/services': No such file or directory Jun 29 16:09:40 kernel: sysrq: This sysrq operation is disabled. Jun 29 16:09:41 kernel: sysrq: Emergency Sync
At this point, I believe I try the magic keys reboot sequence: [Alt+SysRq] R S E I U B. Happy to provide any other info upon request. Thanks in advance. -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1212833 https://bugzilla.suse.com/show_bug.cgi?id=1212833#c3 Anthony Iliopoulos <ailiopoulos@suse.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ailiopoulos@suse.com --- Comment #3 from Anthony Iliopoulos <ailiopoulos@suse.com> --- looks like irqbalance reading /proc/interrupts is causing a null ptr deref at 000000000000005a (the rcu stall warnings is a side-effect, due to irqbalance crashing while being in a rcu reader critical section, and can be ignored). show_interrupts+0x24c corresponds kernel/irq/proc:529:
seq_printf(p, " %s", action->name);
and that implies that action->next was pointing to 0xa. Normally action->next is set when the irq line is shared, not sure what may be corrupting it here. You could perhaps add the following boot params:
kprobe_event="p request_threaded_irq irq=$arg1 handler=$arg2 thread_fn=$arg3 devname=+0($arg5):string dev_id=$arg6;p free_irq irq=$arg1 dev_id=$arg2;p __setup_irq irq=$arg1 desc=$arg2 new=$arg3 desc_action=+160($arg2)" ftrace_dump_on_oops trace_options=stacktrace
(append the above to GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub and run update-grub) after rebooting (and assuming the issue is reproduced), please attach the ftrace output (cat /sys/kernel/tracing/trace or from dmesg logs) -- You are receiving this mail because: You are on the CC list for the bug.
https://bugzilla.suse.com/show_bug.cgi?id=1212833 Stuart Rogers <stuart@stella-maris.org.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |stuart@stella-maris.org.uk -- You are receiving this mail because: You are on the CC list for the bug.
participants (1)
-
bugzilla_noreply＠suse.com