On Wednesday, 5 April 2023 11:55:11 PM ACST Rodney Baker wrote:
On Wednesday, 5 April 2023 11:42:02 PM ACST Andrei Borzenkov wrote:
On Wed, Apr 5, 2023 at 5:09 PM Rodney Baker <rodney.baker@iinet.net.au>
wrote:
On Wednesday, 5 April 2023 11:25:37 PM ACST Andrei Borzenkov wrote:
On Wed, Apr 5, 2023 at 4:47 PM Rodney Baker <rodney.baker@iinet.net.au>
wrote:
On Wednesday, 5 April 2023 7:19:11 PM ACST Manfred Hollstein wrote:
On Wed, 05 Apr 2023, 11:06:53 +0200, Manfred Hollstein wrote: > [...] > > <https://bugzilla.suse.com/show_bug.cgi?id=1210146>
This appears to be caused by having "retbleed=stuff" on the kernel command line. Up to 6.2.9 it never made any problems.
Cheers.
l8er manfred
I got a kernel panic on 6.2.9 even earlier than that, and I don't have retbleed parameters on my kernel command line.
My bootloader kernel command line is: idewait=10 plymouth.enable=0 splash quiet showopts nvidia-drm.modeset=1 elevator=cfq
I'm also not booting from NVMe but from a SATA SDD.
Crash details: ----- Kernel Panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,5)
This is partition 5 of sda. Kernel should not even attempt to mount it as root. Is initrd present and not corrupted? Any errors from grub about loading initrd?
No, not that I can see. /dev/sda5 is the root partition (dev/sda1 is /boot). I think this is still using a traditional partition table (/dev/sda5 is on an extended partition), not GPT.
I do see that dracut is complaining about not being able to find the zfs module for initrd when building it. I tried to get zfs working a while back but failed - but I can't remember where to tell dracut not to try to load that module.
It's not a problem on kernel 6.2.6-1-default though - only on 6.2.9-1-default, and I've run dracut -fM --kver <kernel version> for both kernels manually with the same result.
Just installed missing zfs-kmp-default and dracut completed successfully for 6.2.6-1-default but not for 6.2.9-1-default. It looks like the zfs module hasn't yet been build against 6.2.9-1-default.
Sorry - I hit "send" before I was ready. After trying the above I deleted everything associated with zfs and tried dracut again - this time it completed successfully for both 6.2.6-1-default and 6.2.9-1-default (without complaining about zfs), but still got exactly the same result - kernel panic with 6.2.9-1-default at the same point. Here's the output from dracut for both kernels: dracut -fM --kver 6.2.6-1-default dracut: Executing: /usr/bin/dracut -fM --kver 6.2.6-1-default dracut: dracut module 'systemd-coredump' will not be installed, because command 'coredumpctl' could not be found! dracut: dracut module 'systemd-coredump' will not be installed, because command '/usr/lib/systemd/systemd-coredump' could not be found! dracut: dracut module 'systemd-pcrphase' will not be installed, because command '/usr/lib/systemd/systemd-pcrphase' could not be found! dracut: dracut module 'systemd-portabled' will not be installed, because command 'portablectl' could not be found! dracut: dracut module 'systemd-portabled' will not be installed, because command '/usr/lib/systemd/systemd-portabled' could not be found! dracut: dracut module 'systemd-repart' will not be installed, because command 'systemd-repart' could not be found! dracut: dracut module 'dbus-broker' will not be installed, because command 'dbus-broker' could not be found! dracut: dracut module 'rngd' will not be installed, because command 'rngd' could not be found! dracut: dracut module 'connman' will not be installed, because command 'connmand' could not be found! dracut: dracut module 'connman' will not be installed, because command 'connmanctl' could not be found! dracut: dracut module 'connman' will not be installed, because command 'connmand-wait-online' could not be found! dracut: dracut module 'btrfs' will not be installed, because command 'btrfs' could not be found! dracut: dracut module 'tpm2-tss' will not be installed, because command 'tpm2' could not be found! dracut: dracut module 'nvmf' will not be installed, because command 'nvme' could not be found! dracut: dracut module 'biosdevname' will not be installed, because command 'biosdevname' could not be found! dracut: dracut module 'memstrack' will not be installed, because command 'memstrack' could not be found! dracut: memstrack is not available dracut: If you need to use rd.memdebug>=4, please install memstrack and procps-ng dracut: dracut module 'squash' will not be installed, because command 'mksquashfs' could not be found! dracut: dracut module 'squash' will not be installed, because command 'unsquashfs' could not be found! dracut: dracut module 'systemd-coredump' will not be installed, because command 'coredumpctl' could not be found! dracut: dracut module 'systemd-coredump' will not be installed, because command '/usr/lib/systemd/systemd-coredump' could not be found! dracut: dracut module 'systemd-pcrphase' will not be installed, because command '/usr/lib/systemd/systemd-pcrphase' could not be found! dracut: dracut module 'systemd-portabled' will not be installed, because command 'portablectl' could not be found! dracut: dracut module 'systemd-portabled' will not be installed, because command '/usr/lib/systemd/systemd-portabled' could not be found! dracut: dracut module 'systemd-repart' will not be installed, because command 'systemd-repart' could not be found! dracut: dracut module 'dbus-broker' will not be installed, because command 'dbus-broker' could not be found! dracut: dracut module 'rngd' will not be installed, because command 'rngd' could not be found! dracut: dracut module 'connman' will not be installed, because command 'connmand' could not be found! dracut: dracut module 'connman' will not be installed, because command 'connmanctl' could not be found! dracut: dracut module 'connman' will not be installed, because command 'connmand-wait-online' could not be found! dracut: dracut module 'btrfs' will not be installed, because command 'btrfs' could not be found! dracut: dracut module 'tpm2-tss' will not be installed, because command 'tpm2' could not be found! dracut: dracut module 'nvmf' will not be installed, because command 'nvme' could not be found! dracut: dracut module 'memstrack' will not be installed, because command 'memstrack' could not be found! dracut: memstrack is not available dracut: If you need to use rd.memdebug>=4, please install memstrack and procps-ng dracut: dracut module 'squash' will not be installed, because command 'mksquashfs' could not be found! dracut: dracut module 'squash' will not be installed, because command 'unsquashfs' could not be found! systemd systemd-initrd i18n drm plymouth kernel-modules kernel-modules-extra nvdimm rootfs-block suse-xfs terminfo udev-rules dracut: Skipping udev rule: 40-redhat.rules dracut: Skipping udev rule: 50-firmware.rules dracut: Skipping udev rule: 50-udev.rules dracut: Skipping udev rule: 91-permissions.rules dracut: Skipping udev rule: 80-drivers-modprobe.rules dracut: Skipping udev rule: 70-persistent-net.rules dracut-systemd haveged ostree usrmount base fs-lib shutdown suse suse-initrd dracut: *** Including modules done *** dracut: *** Installing kernel module dependencies *** dracut: *** Installing kernel module dependencies done *** dracut: *** Resolving executable dependencies *** dracut: *** Resolving executable dependencies done *** dracut: *** Hardlinking files *** dracut: Mode: real dracut: Method: sha256 dracut: Files: 1107 dracut: Linked: 2 files dracut: Compared: 0 xattrs dracut: Compared: 119 files dracut: Saved: 352.16 KiB dracut: Duration: 0.006722 seconds dracut: *** Hardlinking files done *** dracut: *** Generating early-microcode cpio image *** dracut: *** Constructing GenuineIntel.bin *** dracut: *** Store current command line parameters *** dracut: Stored kernel commandline: dracut: root=UUID=93cda319-ba05-4170-a509-8f43ad15dc44 rootfstype=ext4 rootflags=rw,noatime dracut: *** Stripping files *** dracut: *** Stripping files done *** dracut: *** Creating image file '/boot/initrd-6.2.6-1-default' *** dracut: *** Creating initramfs image file '/boot/initrd-6.2.6-1-default' done *** dracut -fM --kver 6.2.9-1-default dracut: Executing: /usr/bin/dracut -fM --kver 6.2.9-1-default dracut: dracut module 'systemd-coredump' will not be installed, because command 'coredumpctl' could not be found! dracut: dracut module 'systemd-coredump' will not be installed, because command '/usr/lib/systemd/systemd-coredump' could not be found! dracut: dracut module 'systemd-pcrphase' will not be installed, because command '/usr/lib/systemd/systemd-pcrphase' could not be found! dracut: dracut module 'systemd-portabled' will not be installed, because command 'portablectl' could not be found! dracut: dracut module 'systemd-portabled' will not be installed, because command '/usr/lib/systemd/systemd-portabled' could not be found! dracut: dracut module 'systemd-repart' will not be installed, because command 'systemd-repart' could not be found! dracut: dracut module 'dbus-broker' will not be installed, because command 'dbus-broker' could not be found! dracut: dracut module 'rngd' will not be installed, because command 'rngd' could not be found! dracut: dracut module 'connman' will not be installed, because command 'connmand' could not be found! dracut: dracut module 'connman' will not be installed, because command 'connmanctl' could not be found! dracut: dracut module 'connman' will not be installed, because command 'connmand-wait-online' could not be found! dracut: dracut module 'btrfs' will not be installed, because command 'btrfs' could not be found! dracut: dracut module 'tpm2-tss' will not be installed, because command 'tpm2' could not be found! dracut: dracut module 'nvmf' will not be installed, because command 'nvme' could not be found! dracut: dracut module 'biosdevname' will not be installed, because command 'biosdevname' could not be found! dracut: dracut module 'memstrack' will not be installed, because command 'memstrack' could not be found! dracut: memstrack is not available dracut: If you need to use rd.memdebug>=4, please install memstrack and procps-ng dracut: dracut module 'squash' will not be installed, because command 'mksquashfs' could not be found! dracut: dracut module 'squash' will not be installed, because command 'unsquashfs' could not be found! dracut: dracut module 'systemd-coredump' will not be installed, because command 'coredumpctl' could not be found! dracut: dracut module 'systemd-coredump' will not be installed, because command '/usr/lib/systemd/systemd-coredump' could not be found! dracut: dracut module 'systemd-pcrphase' will not be installed, because command '/usr/lib/systemd/systemd-pcrphase' could not be found! dracut: dracut module 'systemd-portabled' will not be installed, because command 'portablectl' could not be found! dracut: dracut module 'systemd-portabled' will not be installed, because command '/usr/lib/systemd/systemd-portabled' could not be found! dracut: dracut module 'systemd-repart' will not be installed, because command 'systemd-repart' could not be found! dracut: dracut module 'dbus-broker' will not be installed, because command 'dbus-broker' could not be found! dracut: dracut module 'rngd' will not be installed, because command 'rngd' could not be found! dracut: dracut module 'connman' will not be installed, because command 'connmand' could not be found! dracut: dracut module 'connman' will not be installed, because command 'connmanctl' could not be found! dracut: dracut module 'connman' will not be installed, because command 'connmand-wait-online' could not be found! dracut: dracut module 'btrfs' will not be installed, because command 'btrfs' could not be found! dracut: dracut module 'tpm2-tss' will not be installed, because command 'tpm2' could not be found! dracut: dracut module 'nvmf' will not be installed, because command 'nvme' could not be found! dracut: dracut module 'memstrack' will not be installed, because command 'memstrack' could not be found! dracut: memstrack is not available dracut: If you need to use rd.memdebug>=4, please install memstrack and procps-ng dracut: dracut module 'squash' will not be installed, because command 'mksquashfs' could not be found! dracut: dracut module 'squash' will not be installed, because command 'unsquashfs' could not be found! systemd systemd-initrd i18n drm plymouth kernel-modules kernel-modules-extra nvdimm rootfs-block suse-xfs terminfo udev-rules dracut: Skipping udev rule: 40-redhat.rules dracut: Skipping udev rule: 50-firmware.rules dracut: Skipping udev rule: 50-udev.rules dracut: Skipping udev rule: 91-permissions.rules dracut: Skipping udev rule: 80-drivers-modprobe.rules dracut: Skipping udev rule: 70-persistent-net.rules dracut-systemd haveged ostree usrmount base fs-lib shutdown suse suse-initrd dracut: *** Including modules done *** dracut: *** Installing kernel module dependencies *** dracut: *** Installing kernel module dependencies done *** dracut: *** Resolving executable dependencies *** dracut: *** Resolving executable dependencies done *** dracut: *** Hardlinking files *** dracut: Mode: real dracut: Method: sha256 dracut: Files: 1108 dracut: Linked: 2 files dracut: Compared: 0 xattrs dracut: Compared: 120 files dracut: Saved: 352.16 KiB dracut: Duration: 0.006601 seconds dracut: *** Hardlinking files done *** dracut: *** Generating early-microcode cpio image *** dracut: *** Constructing GenuineIntel.bin *** dracut: *** Store current command line parameters *** dracut: Stored kernel commandline: dracut: root=UUID=93cda319-ba05-4170-a509-8f43ad15dc44 rootfstype=ext4 rootflags=rw,noatime dracut: *** Stripping files *** dracut: *** Stripping files done *** dracut: *** Creating image file '/boot/initrd-6.2.9-1-default' *** dracut: *** Creating initramfs image file '/boot/initrd-6.2.9-1-default' done ***
Posting full output for both kernels may give some hints.
CPU:9 PID:1 Comm: Swapper/0 Not tainted 6.2.9-1-default #1 openSUSE Tumbleweed 72246ebc6bb73a9bec193bedaaaba1392d6fb332 Hardware name: System manufacturer System Product Name/PRIME X299-A, BIOS 3501 07/13/2021
Call Trace: <TASK> dump_stack_lvl+0x23/0x60 panic=0x346/0x350 mount_block_root=-x1f7/0x280 prepare_namespace=0xec/0x170 kernel_init_freeable= 0x411/0x450 ? __pfx_kernel_init=0x10/0x10 kernel_init=016/0x1c0 ret_from_fork=0x29/0x50 </TASK>
Kernel Offset: 0x31e00000 from 0xffffffff81000000 (relocation range 0xffffffff80000000-0xffffffffbfffffff) Rebooting in 90 seconds.. -----
The system boots fine with 6.2.6-1-default.\
Regards,
-- ==================================================================== == ==== ======================================== Rodney Baker rodney.baker@iinet.net.au ==================================================================== == ==== ========================================
-- ======================================================================== == ======================================== Rodney Baker rodney.baker@iinet.net.au ======================================================================== == ========================================
-- ================================================================================================================== Rodney Baker rodney.baker@iinet.net.au ==================================================================================================================