https://bugzilla.suse.com/show_bug.cgi?id=1202386 https://bugzilla.suse.com/show_bug.cgi?id=1202386#c1 --- Comment #1 from Andrea Manzini --- Created attachment 860781 --> https://bugzilla.suse.com/attachment.cgi?id=860781&action=edit stack trace of kernel fault add another stack trace from journalctl --system -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1202386 https://bugzilla.suse.com/show_bug.cgi?id=1202386#c3 --- Comment #3 from Andrea Manzini --- Created attachment 860834 --> https://bugzilla.suse.com/attachment.cgi?id=860834&action=edit hwinfo output -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1202386 https://bugzilla.suse.com/show_bug.cgi?id=1202386#c5 Takashi Iwai changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |malte@wcic.co.za --- Comment #5 from Takashi Iwai --- *** Bug 1202507 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1202386 https://bugzilla.suse.com/show_bug.cgi?id=1202386#c7 Jiri Slaby changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jslaby@suse.com --- Comment #7 from Jiri Slaby ---

BUG: kernel NULL pointer dereference, address: 0000000000000000 #PF: supervisor instruction fetch in kernel mode #PF: error_code(0x0010) - not-present page PGD 0 P4D 0 Oops: 0010 [#1] PREEMPT SMP NOPTI CPU: 8 PID: 899 Comm: systemd-logind Tainted: P OE 5.19.1-1-default #1 openSUSE Tumbleweed f3b44bfb672cdb9f235aff53b57724eba8b9411b Hardware name: LENOVO 20YQ001XZA/20YQ001XZA, BIOS N37ET39W (1.20 ) 04/15/2022 RIP: 0010:0x0 Code: Unable to access opcode bytes at RIP 0xffffffffffffffd6. RSP: 0018:ffffb1a741f0fc78 EFLAGS: 00010202 RAX: 0000000000000000 RBX: ffffffff95a5de28 RCX: ffffffff95a5de28 RDX: ffffb1a741f0fc80 RSI: 0000000000000004 RDI: ffff8a02404ec800 RBP: 0000000000000004 R08: ffff8a02404ec838 R09: ffff8a0203f1b500 R10: 0000000000000000 R11: 0000000000000001 R12: ffff8a0207cf0000 R13: ffff8a02404ec838 R14: ffffffff95a5de28 R15: ffff8a02404ec800 FS: 00007f6913f87b80(0000) GS:ffff8a094fe00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: ffffffffffffffd6 CR3: 0000000103e1a004 CR4: 0000000000770ee0 PKRU: 55555554 Call Trace: <TASK> power_supply_show_property+0xb3/0x230 dev_attr_show+0x15/0x40 sysfs_kf_seq_show+0xa0/0xe0 seq_read_iter+0x11f/0x450 new_sync_read+0xf7/0x180 vfs_read+0x144/0x190 ksys_read+0x63/0xe0 do_syscall_64+0x58/0x80 entry_SYSCALL_64_after_hwframe+0x63/0xcd RIP: 0033:0x7f691472042c Code: ec 28 48 89 54 24 18 48 89 74 24 10 89 7c 24 08 e8 27 31 f8 ff 48 8b 54 24 18 48 8b 74 24 10 41 89 c0 8b 7c 24 08 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 34 44 89 c7 48 89 44 24 08 e8 8d 31 f8 ff 48 RSP: 002b:00007ffc86fa8490 EFLAGS: 00000246 ORIG_RAX: 0000000000000000 RAX: ffffffffffffffda RBX: 0000000000001001 RCX: 00007f691472042c RDX: 0000000000001001 RSI: 000055e7440c0560 RDI: 0000000000000018 RBP: 000055e7440c0560 R08: 0000000000000000 R09: 00007f691481cb10 R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000018 R13: 0000000000001001 R14: ffffffffffffffff R15: 0000000000001000 </TASK> Modules linked in: snd_ctl_led snd_soc_skl_hda_dsp snd_soc_intel_hda_dsp_common snd_soc_hdac_hdmi snd_sof_probes snd_usb_audio snd_usbmidi_lib snd_rawmidi snd_seq_device hid_logitech_hidpp af_packet cmac algif_hash algif_skcipher af_alg nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 bnep nft_reject btusb btrtl btbcm btintel hid_logitech_dj btmtk nft_ct bluetooth uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common hid_generic videodev mc usbhid ecdh_generic nft_chain_nat nf_tables ebtable_nat ebtable_broute ip6table_nat ip6table_mangle nvidia_drm(POE) nvidia_modeset(POE) ip6table_raw ip6table_security iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c iptable_mangle iptable_raw nvidia_uvm(POE) iptable_security ip_set nfnetlink nvidia(POE) ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter bpfilter qrtr dmi_sysfs snd_soc_dmic snd_hda_codec_realtek snd_hda_codec_generic ee1004 iTCO_wdt snd_sof_pci_intel_tgl spi_nor snd_sof_intel_hda_common intel_pmc_bxt iTCO_vendor_support iwlmvm mtd soundwire_intel soundwire_generic_allocation soundwire_cadence snd_sof_intel_hda mei_wdt mei_hdcp mei_pxp intel_rapl_msr snd_sof_pci snd_sof_xtensa_dsp mac80211 snd_sof snd_sof_utils snd_soc_hdac_hda snd_hda_ext_core intel_tcc_cooling snd_soc_acpi_intel_match x86_pkg_temp_thermal intel_powerclamp snd_soc_acpi coretemp soundwire_bus libarc4 kvm_intel snd_soc_core snd_compress snd_pcm_dmaengine kvm snd_hda_intel irqbypass iwlwifi snd_intel_dspcfg snd_intel_sdw_acpi thinkpad_acpi snd_hda_codec iwlmei nls_iso8859_1 joydev ledtrig_audio efi_pstore nls_cp437 platform_profile snd_hda_core think_lmi i2c_i801 vfat spi_intel_pci int3403_thermal ac firmware_attributes_class cfg80211 wmi_bmof i2c_smbus spi_intel fat snd_hwdep rfkill snd_pcm mei_me thunderbolt igc snd_timer processor_thermal_device_pci_legacy mei processor_thermal_device processor_thermal_rfim snd processor_thermal_mbox processor_thermal_rapl soundcore intel_rapl_common int340x_thermal_zone thermal intel_soc_dts_iosf int3400_thermal intel_pmc_core intel_hid acpi_thermal_rel sparse_keymap acpi_pad acpi_tad tiny_power_button fuse configfs ip_tables x_tables ext4 mbcache jbd2 sdhci_pci xhci_pci xhci_pci_renesas cqhci xhci_hcd ucsi_acpi crct10dif_pclmul sdhci nvme crc32_pclmul typec_ucsi crc32c_intel ghash_clmulni_intel aesni_intel crypto_simd cryptd serio_raw roles mmc_core usbcore nvme_core typec wmi battery video pinctrl_tigerlake button l2tp_ppp l2tp_netlink l2tp_core ip6_udp_tunnel udp_tunnel pppox ppp_generic slhc sg dm_multipath dm_mod scsi_dh_rdac scsi_dh_emc scsi_dh_alua msr efivarfs Unloaded tainted modules: acpi_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 fjes():1 pcc_cpufreq():1 acpi_cpufreq():1 fjes():1 acpi_cpufreq():1 pcc_cpufreq():1 fjes():1 pcc_cpufreq():1 acpi_cpufreq():1 fjes():1 pcc_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 fjes():1 acpi_cpufreq():1 fjes():1 asus_ec_sensors():1 pcc_cpufreq():1 asus_ec_sensors():1 fjes():1 acpi_cpufreq():1 CR2: 0000000000000000

-- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1202386 https://bugzilla.suse.com/show_bug.cgi?id=1202386#c9 --- Comment #9 from Jiri Slaby --- (In reply to Jiri Slaby from comment #8)

Decoded call Trace:

...

power_supply_show_property (drivers/power/supply/power_supply_sysfs.c:284)

So psy->desc->get_property is apparently -ENOMSG when called.

No, CR2 (0xffffffffffffffd6) confused me. ->get_property is apparently NULL according to RAX and the second CR2. -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1202386 https://bugzilla.suse.com/show_bug.cgi?id=1202386#c11 Jiri Slaby changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |needinfo?(malte@wcic.co.za) | |, | |needinfo?(nicholas.yang@sus | |e.com) --- Comment #11 from Jiri Slaby --- (In reply to Takashi Iwai from comment #6)

Could you try to blacklist ucsi_acpi ?

This still holds ^^. Anyone? Common modules to the three reporters. aesni_intel battery button configfs crct10dif_pclmul crc32c_intel crc32_pclmul cryptd crypto_simd dm_mod dm_multipath efivarfs fuse ghash_clmulni_intel ip_tables libcrc32c mc mmc_core msr nvidia_drm(POE) nvidia_modeset(POE) nvidia(POE) nvidia_uvm(POE) nvme nvme_core pinctrl_tigerlake roles scsi_dh_alua scsi_dh_emc scsi_dh_rdac serio_raw sg typec typec_ucsi ucsi_acpi usbcore video videodev wmi xhci_hcd xhci_pci xhci_pci_renesas x_tables -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1202386 https://bugzilla.suse.com/show_bug.cgi?id=1202386#c13 Andrea Manzini changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(malte@wcic.co.za) | |, | |needinfo?(nicholas.yang@sus | |e.com) | --- Comment #13 from Andrea Manzini --- Created attachment 860900 --> https://bugzilla.suse.com/attachment.cgi?id=860900&action=edit new hwinfo after zypper update (kernel 5.19.1-1) -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1202386 https://bugzilla.suse.com/show_bug.cgi?id=1202386#c15 --- Comment #15 from Andrea Manzini --- the report on https://bugzilla.suse.com/show_bug.cgi?id=1202388 seems about 5.19.0 ; the reporter also mentions black screen, which could point us to a problem with NVIDIA drivers because I had to update also them and compile against the new kernel to have graphical mode working on my laptop. -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1202386 https://bugzilla.suse.com/show_bug.cgi?id=1202386#c17 --- Comment #17 from Takashi Iwai --- This reminded me of one basic question: could the bug be rather in Nvidia driver? i.e. Can anyone reproduce the bug without Nvidia driver at all? -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1202386 https://bugzilla.suse.com/show_bug.cgi?id=1202386#c19 --- Comment #19 from Malte von der Lancken Wakenitz --- (In reply to Takashi Iwai from comment #6)

bug 1202388 might be relevant, too.

Could you try to blacklist ucsi_acpi ?

I have blacklisted ucsi_acpi and subsequently booted twice (consecutively) successfully. I do not observe any null pointer dereferences in the logs. The system seems to be stable so far. -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1202386 https://bugzilla.suse.com/show_bug.cgi?id=1202386#c21 --- Comment #21 from Malte von der Lancken Wakenitz --- (In reply to Takashi Iwai from comment #20)

FWIW, I'm building a test kernel with a debug patch that does NULL checks and some debug prints around the power supply driver in OBS home:tiwai:bsc1202386. If it were a simple NULL ops, the crash could be avoided (and show the problematic device name).

Unfortunately, the package build gets stuck for hours, maybe because of the reduced build power in our build firm. In anyway, once after the build finishes, it'd be great if anyone can test it (in the similar condition you triggered the bug in the past) and give the dmesg output. The test kernel will appear at: http://download.opensuse.org/repositories/home:/tiwai:/bsc1202386/standard/

Note that it's an unofficial build, hence disable Secure Boot beforehand.

I have added the above repo to my normal repos. I see only i586 and i686 architecture kernels. But I have also never installed a "patched" debug kernel like this before. Am I missing something? If so could you perhaps point me towards some documentation that might help me understand what to do? -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1202386 https://bugzilla.suse.com/show_bug.cgi?id=1202386#c23 --- Comment #23 from Takashi Iwai --- Yes, now you can find x86-64 binaries, too. The build took just too long yesterday :-< I recommend *not* to add the repo to zypper, but just download kernel-default.rpm from the URL, and install it via zypper install. You might need to pass --oldpackage option, too. Also, another recommendation is to increase the number of installable kernels by editing /etc/zypp/zypp.conf beforehand. Add more entries in the line "multiversion.kernels = ..." line, e.g. multiversion.kernels = latest,latest-1,latest-2,latest-3,running -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1202386 https://bugzilla.suse.com/show_bug.cgi?id=1202386#c25 --- Comment #25 from Nicholas Yang --- I installed the test kernel-default and kernel-default-devel. Zypper complains lack of dependency `kernel-devel` and nvidia drivers failed to link against it. I boot the test kernel without nvidia drivers and kernel oops is still there. BTW, I dump another copy of dmesg and hwinfo when 5.19.1-1-default boots successfully without blacklisting ucsi_acpi. Do you need these log files? -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1202386 https://bugzilla.suse.com/show_bug.cgi?id=1202386#c27 --- Comment #27 from Malte von der Lancken Wakenitz --- (In reply to Takashi Iwai from comment #23)

Yes, now you can find x86-64 binaries, too. The build took just too long yesterday :-<

I recommend *not* to add the repo to zypper, but just download kernel-default.rpm from the URL, and install it via zypper install. You might need to pass --oldpackage option, too.

Also, another recommendation is to increase the number of installable kernels by editing /etc/zypp/zypp.conf beforehand. Add more entries in the line "multiversion.kernels = ..." line, e.g. multiversion.kernels = latest,latest-1,latest-2,latest-3,running

I successfully installed the debug kernel. Removed the module ucsi_acpi from the blacklist. Disabled secure boot. And tried to boot without success. It did get to a login screen (so nvidia driver was working), but on login the plymouth screen just started spinning it's busy image. I toggled to console (ctrl-alt-F1) and started seeing a stream of messages (presumably debug): localhost.localdomain kernel: power_supply ucsi-source-psy-USBC000:002: XXX psy->desc->get_property is NULL I could not log in on console. The system had become unresponsive. I then hard reset the machine. Added the module ucsi_acpi back into the blacklist and rebooted successfully. I have attached as much log information as I could get from the attempt. -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1202386 https://bugzilla.suse.com/show_bug.cgi?id=1202386#c29 --- Comment #29 from Takashi Iwai --- Thanks, that helped understanding what's going on there. I guess the problem is the upstream commit 87d0e2f41b8cc2018499be4e8003fa8c09b6f2fb usb: typec: ucsi: add a common function ucsi_unregister_connectors() This commit looks as if it were a harmless cleanup, but this failed in a subtle way. Namely, in the error scenario, the driver gets an error at ucsi_register_altmodes(), and goes to the error handling to release the resources. Through this refactoring, the release part was unified to a funciton ucsi_unregister_connectors(). And there, it has a NULL check of con->wq, and it bails out the loop if it's NULL. Meanwhile, ucsi_register_port() itself still calls destroy_workqueue() and clear con->wq at its error path. This ended up in the leftover power supply device with the uninitialized / cleared device. Now I'm building yet another kernel with the revert of the commit above in OBS home:tiwai:bsc1202386-2 repo. The test kernel will appear at http://download.opensuse.org/repositories/home:/tiwai:/bsc1202386-2/standard... Please give it a try later. -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1202386 https://bugzilla.suse.com/show_bug.cgi?id=1202386#c31 --- Comment #31 from Malte von der Lancken Wakenitz --- (In reply to Takashi Iwai from comment #29)

Thanks, that helped understanding what's going on there.

I guess the problem is the upstream commit 87d0e2f41b8cc2018499be4e8003fa8c09b6f2fb usb: typec: ucsi: add a common function ucsi_unregister_connectors()

This commit looks as if it were a harmless cleanup, but this failed in a subtle way. Namely, in the error scenario, the driver gets an error at ucsi_register_altmodes(), and goes to the error handling to release the resources. Through this refactoring, the release part was unified to a funciton ucsi_unregister_connectors(). And there, it has a NULL check of con->wq, and it bails out the loop if it's NULL. Meanwhile, ucsi_register_port() itself still calls destroy_workqueue() and clear con->wq at its error path. This ended up in the leftover power supply device with the uninitialized / cleared device.

Now I'm building yet another kernel with the revert of the commit above in OBS home:tiwai:bsc1202386-2 repo. The test kernel will appear at

http://download.opensuse.org/repositories/home:/tiwai:/bsc1202386-2/standard...

Please give it a try later.

I have successfully booted twice in consecutively. This error is still reported: ucsi_acpi USBC000:00: con2: failed to register alt modes, but without the subsequent null pointer dereference. I assume this fixes the issue. How long before one might expect this fix to be included in an official kernel build? I do have a question though. What is the significance of the error? I see other USCI related errors, but also without any noticeable consequence. Are these just errors caused by the general "flakyness" of USB-C type docking stations? I will attach the dmesg output from the last two boots. And thank you very much for your help. -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1202386 https://bugzilla.suse.com/show_bug.cgi?id=1202386#c33 --- Comment #33 from Malte von der Lancken Wakenitz --- Created attachment 860949 --> https://bugzilla.suse.com/attachment.cgi?id=860949&action=edit Successful second boot with USCI_ACPI patch -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1202386 https://bugzilla.suse.com/show_bug.cgi?id=1202386#c35 Takashi Iwai changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #35 from Takashi Iwai --- Upstream took the revert patch in the end as a fix. I refreshed the patch accordingly on stable branch. -- You are receiving this mail because: You are the assignee for the bug.

https://bugzilla.suse.com/show_bug.cgi?id=1202386 https://bugzilla.suse.com/show_bug.cgi?id=1202386#c37 --- Comment #37 from Takashi Iwai --- The error is still expected to appear. What we fixed is that a bug that was triggered by this error handling. But, it's still interesting whether this error appears newly in 5.19.x. Did it appear on the earlier, 5.18.x kernels, too? -- You are receiving this mail because: You are the assignee for the bug.