Yikes! Looks like an update has screwed up my laptop
I am running OpenSuSE 15.5 with KDE/Plasma on my laptop and I, this morning, gave it permission to install the latest set of updates that I was told were waiting. It included a kernel update and as usual those want me to reboot after being installed. So I did and now I can no longer get my desktop up and running. The visual log, seen by using the Esc key during the system startup process, gives the following clues. During a regular (init 5) startup, I see two "start jobs" that seem to run forever - (1 of 2) A start job is running for User Login Management (xxmin xxs / ymin) (2 of 2) A start job is running for Hold until boot process finishes up (xxmin xxsec / no limit) Start job (1 of 2) keeps upping the ymin limit for about 15 to 20 minutes and then the following messages are logged - FAILED] Failed to start User Login Management. See 'systemctl status systemd-logind services' for details. [ OK ] Stopped User Login Management. Starting Load Kernel Module drm... [ OK ] Finished Load Kernel Module drm. Starting User Login Management... Starting Cleanup of Temporary Directories... [ OK ] Finished Cleanup of Temporary Directories. and then this cycle repeats forever. I next tried to start up OpenSuSE to init 3 and only the following error message was simply reported - FAILED to start Command Scheduler I am then allowed to login as root but I don't know what to do or how to proceed. While logged in, I do get repeating message interrupts telling me - kernel:[ nnn.nnnnnnn C0] BUG:workqueue lockup - pool cpus=14 node=0 flags=0x0 nice=0 stuck for xxxs! So how do I get my laptop back to a happy state? Marc C 1 1 -- --... ...-- .----. ... -.. . .-- .- --... .--. -..- .-- -- .- .-. -.-. <b>Computers: the final frontier. These are the voyages of the user Marc.<br> His mission: to explore strange new hardware. To seek out new software and new applications.<br> To boldly go where no Marc has gone before!<br></b>
Marc Chamberlin composed on 2024-01-20 12:57 (UTC-0800):
I am running OpenSuSE 15.5 with KDE/Plasma on my laptop and I, this morning, gave it permission to install the latest set of updates that I was told were waiting. It included a kernel update and as usual those want me to reboot after being installed. So I did and now I can no longer get my desktop up and running. The visual log, seen by using the Esc key during the system startup process, gives the following clues.
During a regular (init 5) startup, I see two "start jobs" that seem to run forever -
(1 of 2) A start job is running for User Login Management (xxmin xxs / ymin)
(2 of 2) A start job is running for Hold until boot process finishes up (xxmin xxsec / no limit)
Start job (1 of 2) keeps upping the ymin limit for about 15 to 20 minutes and then the following messages are logged -
FAILED] Failed to start User Login Management. See 'systemctl status systemd-logind services' for details. [ OK ] Stopped User Login Management. Starting Load Kernel Module drm... [ OK ] Finished Load Kernel Module drm. Starting User Login Management... Starting Cleanup of Temporary Directories... [ OK ] Finished Cleanup of Temporary Directories.
and then this cycle repeats forever. I next tried to start up OpenSuSE to init 3 and only the following error message was simply reported -
FAILED to start Command Scheduler
I am then allowed to login as root but I don't know what to do or how to proceed. While logged in, I do get repeating message interrupts telling me -
kernel:[ nnn.nnnnnnn C0] BUG:workqueue lockup - pool cpus=14 node=0 flags=0x0 nice=0 stuck for xxxs!
So how do I get my laptop back to a happy state?
Marc C 1 1
Note in /boot/ whether the initrds all have today's timestamps, or only just the newest kernel. Reboot selecting the prior kernel from advanced options. What happens? If same problem, and the older kernel's initrd was rebuilt today, try restoring the initrd from your backup for prior kernel, then try booting using it. If none of that helps. Try init 3 again, then zypper ref, zypper up, to see if the earlier update may not have completed. I did multiple PCs' 15.5 updates with new kernels yesterday and Thursday, all smooth, working as expected. -- Evolution as taught in public schools is, like religion, based on faith, not based on science. Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata
On 1/20/24 13:08, Felix Miata wrote:
Marc Chamberlin composed on 2024-01-20 12:57 (UTC-0800):
Note in /boot/ whether the initrds all have today's timestamps, or only just the newest kernel. Reboot selecting the prior kernel from advanced options. What happens? If same problem, and the older kernel's initrd was rebuilt today, try restoring the initrd from your backup for prior kernel, then try booting using it.
The initrds have different timestamps - -rwxrwxrwx 1 root root 97142441 Jan 3 15:07 initrd-5.14.21-150500.55.39-default -rwxrwxrwx 1 root root 97182349 Jan 20 11:19 initrd-5.14.21-150500.55.44-default Dropping back to OpenSuSE 15.5 with Linux 5.14.21-150500.55.39-default works fine. OpenSuSE 15.5. with Linux 5.14.21-150500.55.44-default fails
If none of that helps. Try init 3 again, then zypper ref, zypper up, to see if the earlier update may not have completed. Umm, going to the 55.44-default version init 3 is no longer allowing me to log in as root for some reason. It asks me for the login name but when I type in "root" it just hangs and doesn't even ask for a password now. I will await for further advice before doing anything further I did multiple PCs' 15.5 updates with new kernels yesterday and Thursday, all smooth, working as expected. Yeah, story of my life LOL, no one else is having troubles when I do... sigh... Marc...
1 -- --... ...-- .----. ... -.. . .-- .- --... .--. -..- .-- -- .- .-. -.-. <b>Computers: the final frontier. These are the voyages of the user Marc.<br> His mission: to explore strange new hardware. To seek out new software and new applications.<br> To boldly go where no Marc has gone before!<br></b>
Marc Chamberlin composed on 2024-01-20 14:08 (UTC-0800):
Felix Miata wrote:
Marc Chamberlin composed on 2024-01-20 12:57 (UTC-0800):
Note in /boot/ whether the initrds all have today's timestamps, or only just the newest kernel. Reboot selecting the prior kernel from advanced options. What happens? If same problem, and the older kernel's initrd was rebuilt today, try restoring the initrd from your backup for prior kernel, then try booting using it.
The initrds have different timestamps -
-rwxrwxrwx 1 root root 97142441 Jan 3 15:07 initrd-5.14.21-150500.55.39-default
Be sure this one is preserved until solution is employed, either via backup, or immutability, or both.
-rwxrwxrwx 1 root root 97182349 Jan 20 11:19 initrd-5.14.21-150500.55.44-default
Dropping back to OpenSuSE 15.5 with Linux 5.14.21-150500.55.39-default works fine.
OpenSuSE 15.5. with Linux 5.14.21-150500.55.44-default fails
If none of that helps. Try init 3 again, then zypper ref, zypper up, to see if the earlier update may not have completed.
Umm, going to the 55.44-default version init 3 is no longer allowing me to log in as root for some reason. It asks me for the login name but when I type in "root" it just hangs and doesn't even ask for a password now. I will await for further advice before doing anything further
Use 55.39 boot to try zypper ref, zypper up. If anything is found to apply, boot 55.44. If still fubar, boot 55.39 to regenerate 55.44 and try it again.
I did multiple PCs' 15.5 updates with new kernels yesterday and Thursday, all smooth, working as expected.
Yeah, story of my life LOL, no one else is having troubles when I do... sigh... Marc...
You haven't indicated the hardware involved. If often makes a difference in reproducibility by others. All mine yesterday and the day before were with Intel CPUs and either AMD or Intel graphics. I have more yet to do, some with NVidia graphics, none with dual graphics, and a trivial few with AMD CPUs. -- Evolution as taught in public schools is, like religion, based on faith, not based on science. Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata
Marc Chamberlin composed on 2024-01-20 14:08 (UTC-0800):
Felix Miata wrote:
Marc Chamberlin composed on 2024-01-20 12:57 (UTC-0800): Note in /boot/ whether the initrds all have today's timestamps, or only just the newest kernel. Reboot selecting the prior kernel from advanced options. What happens? If same problem, and the older kernel's initrd was rebuilt today, try restoring the initrd from your backup for prior kernel, then try booting using it. The initrds have different timestamps - -rwxrwxrwx 1 root root 97142441 Jan 3 15:07 initrd-5.14.21-150500.55.39-default Be sure this one is preserved until solution is employed, either via backup, or immutability, or both.
-rwxrwxrwx 1 root root 97182349 Jan 20 11:19 initrd-5.14.21-150500.55.44-default Dropping back to OpenSuSE 15.5 with Linux 5.14.21-150500.55.39-default works fine. OpenSuSE 15.5. with Linux 5.14.21-150500.55.44-default fails
If none of that helps. Try init 3 again, then zypper ref, zypper up, to see if the earlier update may not have completed. Umm, going to the 55.44-default version init 3 is no longer allowing me to log in as root for some reason. It asks me for the login name but when I type in "root" it just hangs and doesn't even ask for a password now. I will await for further advice before doing anything further Use 55.39 boot to try zypper ref, zypper up. If anything is found to apply, boot 55.44. If still fubar, boot 55.39 to regenerate 55.44 and try it again. Thanks again Felix for helping me, but still no joy getting 55.44 to work. I booted 55.39 and ran zypper ref which reported all repositories were up to date. I then ran zypper up and it reported that there was nothing to do. Tried 55.44 and got the same problems that I reported before. So still fubared. I don't grok your second sentence however, how do I "regenerate" 55.44? Do I delete it from the /boot dir and then do
On 1/20/24 14:27, Felix Miata wrote: the zypper up command again?
You haven't indicated the hardware involved. If often makes a difference in reproducibility by others. All mine yesterday and the day before were with Intel CPUs and either AMD or Intel graphics. I have more yet to do, some with NVidia graphics, none with dual graphics, and a trivial few with AMD CPUs.
I am working on a Dell XPS 15 9530 laptop. hwinfo --short gives the following info, hths! wa7pxw:/home/marc/Documents # hwinfo --short cpu: 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz 13th Gen Intel(R) Core(TM) i9-13900H, 933 MHz 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz keyboard: /dev/input/event0 AT Translated Set 2 keyboard /dev/tty7 serial console mouse: /dev/input/mice Logitech Optical Wheel Mouse /dev/input/mice PS/2 Generic Mouse /dev/input/mice ELAN900C:00 04F3:2D25 /dev/input/mice VEN_04F3:00 04F3:32AA Mouse /dev/input/mice VEN_04F3:00 04F3:32AA Touchpad monitor: 1D20G 156XG01 LCD Monitor graphics card: nVidia 3D controller Intel VGA compatible controller sound: Intel Multimedia audio controller storage: Intel RAID bus controller Samsung Electronics NVMe SSD Controller PM9A1/PM9A3/980PRO network: wlan0 Intel WLAN controller eth4 ASIX Electronics AX88179 Gigabit Ethernet network interface: wlan0 WLAN network interface eth4 Ethernet network interface lo Loopback network interface disk: /dev/nvme0n1 Samsung Electronics NVMe SSD Controller PM9A1/PM9A3/980PRO partition: /dev/nvme0n1p1 Partition /dev/nvme0n1p2 Partition /dev/nvme0n1p3 Partition /dev/nvme0n1p4 Partition /dev/nvme0n1p5 Partition /dev/nvme0n1p6 Partition /dev/nvme0n1p7 Partition /dev/nvme0n1p8 Partition /dev/nvme0n1p9 Partition /dev/nvme0n1p10 Partition usb controller: Intel USB Controller Intel USB Controller Intel USB Controller bios: BIOS bridge: Intel PCI bridge Intel ISA bridge Intel PCI bridge Intel PCI bridge Intel PCI bridge Intel Host bridge Intel PCI bridge hub: Genesys Logic Hub VIA USB3.0 Hub Linux Foundation 2.0 root hub Genesys Logic USB3.1 Hub VIA USB2.0 Hub Linux Foundation 3.0 root hub Terminus USB 2.0 Hub Linux Foundation 2.0 root hub Linux Foundation 3.0 root hub memory: Main Memory bluetooth: Intel Bluetooth Device unknown: FPU DMA controller PIC Keyboard controller PS/2 Controller Intel System peripheral Intel Serial bus controller Intel Serial bus controller Intel Signal processing controller Realtek RTS5260 PCI Express Card Reader Intel Communication controller Intel Serial bus controller Intel Serial controller Intel Serial bus controller Intel System peripheral Intel RAM memory Intel SMBus Intel Signal processing controller Microdia Integrated_Webcam_HD Shenzhen Goodix Technology Co.,Ltd Goodix USB2.0 MISC VIA USB Billboard Device -- --... ...-- .----. ... -.. . .-- .- --... .--. -..- .-- -- .- .-. -.-. <b>Computers: the final frontier. These are the voyages of the user Marc.<br> His mission: to explore strange new hardware. To seek out new software and new applications.<br> To boldly go where no Marc has gone before!<br></b>
Marc Chamberlin composed on 2024-01-20 16:53 (UTC-0800):
Thanks again Felix for helping me, but still no joy getting 55.44 to work. I booted 55.39 and ran zypper ref which reported all repositories were up to date. I then ran zypper up and it reported that there was nothing to do. Tried 55.44 and got the same problems that I reported before. So still fubared. I don't grok your second sentence however, how do I "regenerate" 55.44? Do I delete it from the /boot dir and then do the zypper up command again?
Regenerate = make a new one to replace the old one. Commands mkinitrd and dracut can both do this. You'll have to visit the man page of your choice to get the applicable syntax for building one that is not currently booted or in process of being installed. <https://doc.opensuse.org/documentation/leap/reference/html/book-reference/cha-boot.html> provides a dracut procedure outline. Google didn't find me any SDB: covering this subject. :p The forums and mailing list archives should have copious examples if you find the man pages too terse.
I am working on a Dell XPS 15 9530 laptop. hwinfo --short gives the following info, hths!
wa7pxw:/home/marc/Documents # hwinfo --short cpu: 13th Gen Intel(R) Core(TM) i9-13900H, 3000 MHz ... graphics card: nVidia 3D controller Intel VGA compatible controller
The combination of CPU newness (Q1'23) and NVidia suggests you may be using an NVidia proprietary driver, in which case this may be a problem of a kernel change that requires an update to the proprietary driver that isn't yet available. Alternatively, that less than year old CPU may be exhibiting a new bug as a result of a kernel patch in the newer version of a kernel that originally predated your CPU measured in years. If initrd regeneration fails to solve the problem, I suggest you just keep your 55.39 kernel protected and report a bug on 55.44: https://en.opensuse.org/openSUSE:Bugreport_kernel https://en.opensuse.org/openSUSE:Submitting_bug_reports Inxi, which you may need to install, does a better job of providing concise summary of hardware and software configuration: basics: inxi -bz expanded basics: inxi -Fz extensive detail: inxi -Faz details of what's more possibly relevant according to your issue: inxi -CGMSnaz -- Evolution as taught in public schools is, like religion, based on faith, not based on science. Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata
On 2024-01-20 16:08, Marc Chamberlin via openSUSE Users wrote:
On 1/20/24 13:08, Felix Miata wrote:
Marc Chamberlin composed on 2024-01-20 12:57 (UTC-0800):
Note in /boot/ whether the initrds all have today's timestamps, or only just the newest kernel. Reboot selecting the prior kernel from advanced options. What happens? If same problem, and the older kernel's initrd was rebuilt today, try restoring the initrd from your backup for prior kernel, then try booting using it.
The initrds have different timestamps -
-rwxrwxrwx 1 root root 97142441 Jan 3 15:07 initrd-5.14.21-150500.55.39-default -rwxrwxrwx 1 root root 97182349 Jan 20 11:19 initrd-5.14.21-150500.55.44-default
Dropping back to OpenSuSE 15.5 with Linux 5.14.21-150500.55.39-default works fine.
OpenSuSE 15.5. with Linux 5.14.21-150500.55.44-default fails
When you boot into the previous kernel as Felix suggested, the root file system is mounted read-only. This means you cannot make permanent changes to the system at this point. To allow that you must first _rollback_ the system to the old (working) kernel. If your root partition is formatted btrfs, this step is simple: as root, just run snapper rollback and reboot If your root partition is formatted with some other file system, things become more complicated, and I will have to read things in more detail before offering any advice. Perhaps in the meanwhile some suse guru will come along and do the job for me :) However you are able to do it, once you have restored the old kernel as the default kernel, you may then do a system update. Since this will install the new kernel again, you will have to reboot. At this point, if the new kernel still will not boot, roll the system back again, and mark the kernel as "locked" with zypper, or in yast. Then report the problem in the opensuse bugzilla.
If none of that helps. Try init 3 again, then zypper ref, zypper up, to see if the earlier update may not have completed. Umm, going to the 55.44-default version init 3 is no longer allowing me to log in as root for some reason. It asks me for the login name but when I type in "root" it just hangs and doesn't even ask for a password now. I will await for further advice before doing anything further I did multiple PCs' 15.5 updates with new kernels yesterday and Thursday, all smooth, working as expected. Yeah, story of my life LOL, no one else is having troubles when I do... sigh... Marc...
1
Darryl Gregorash composed on 2024-01-21 17:42 (UTC-0600):
When you boot into the previous kernel as Felix suggested, the root file system is mounted read-only. This means you cannot make permanent changes to the system at this point. To allow that you must first _rollback_ the system to the old (working) kernel. If your root partition is formatted btrfs...
Did Marc state anywhere in this thread, or elsewhere you are privy to, what his root filesystem is? Without rollbacks enabled, booting prior kernel does not result in ro /, not here at least, where ext4 is used for /, and btrfs is not. -- Evolution as taught in public schools is, like religion, based on faith, not based on science. Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata
On 2024-01-21 18:08, Felix Miata wrote:
Darryl Gregorash composed on 2024-01-21 17:42 (UTC-0600):
When you boot into the previous kernel as Felix suggested, the root file system is mounted read-only. This means you cannot make permanent changes to the system at this point. To allow that you must first _rollback_ the system to the old (working) kernel. If your root partition is formatted btrfs...
Did Marc state anywhere in this thread, or elsewhere you are privy to, what his root filesystem is? Without rollbacks enabled, booting prior kernel does not result in ro /, not here at least, where ext4 is used for /, and btrfs is not. The opensuse default is now / formatted btrfs with rollbacks enabled. My message intended in part to get him to reveal this information to us. You also neglect to note the second part, where I said that things get more difficult, if his root system is not formatted btrfs, in which case I have to review a lot more to be able to provide assistance. Your responses have so far assumed that his system is NOT formatted btrfs, etc -- which is perhaps a rather big assumption. At least I do not assume anything about that, but rather state clearly that there are 2 possible routes to him to recover a bootable system, depending on how his system actually is formatted. The method you posted may work for you, but clearly it did not work for him -- after he boots into the old kernel, zypper up tells him there is no work to do.
On 22.01.2024 02:42, Darryl Gregorash wrote:
On 2024-01-20 16:08, Marc Chamberlin via openSUSE Users wrote:
On 1/20/24 13:08, Felix Miata wrote:
Marc Chamberlin composed on 2024-01-20 12:57 (UTC-0800):
Note in /boot/ whether the initrds all have today's timestamps, or only just the newest kernel. Reboot selecting the prior kernel from advanced options. What happens? If same problem, and the older kernel's initrd was rebuilt today, try restoring the initrd from your backup for prior kernel, then try booting using it.
The initrds have different timestamps -
-rwxrwxrwx 1 root root 97142441 Jan 3 15:07 initrd-5.14.21-150500.55.39-default -rwxrwxrwx 1 root root 97182349 Jan 20 11:19 initrd-5.14.21-150500.55.44-default
Dropping back to OpenSuSE 15.5 with Linux 5.14.21-150500.55.39-default works fine.
OpenSuSE 15.5. with Linux 5.14.21-150500.55.44-default fails
When you boot into the previous kernel as Felix suggested, the root file system is mounted read-only. This means you cannot make permanent changes to the system at this point. To allow that you must first _rollback_ the system to the old (working) kernel.
You confuse "boot previous kernel" with "boot from snapshot".
On 2024-01-21 22:06, Andrei Borzenkov wrote:
On 22.01.2024 02:42, Darryl Gregorash wrote:
On 2024-01-20 16:08, Marc Chamberlin via openSUSE Users wrote:
On 1/20/24 13:08, Felix Miata wrote:
Marc Chamberlin composed on 2024-01-20 12:57 (UTC-0800):
Note in /boot/ whether the initrds all have today's timestamps, or only just the newest kernel. Reboot selecting the prior kernel from advanced options. What happens? If same problem, and the older kernel's initrd was rebuilt today, try restoring the initrd from your backup for prior kernel, then try booting using it.
The initrds have different timestamps -
-rwxrwxrwx 1 root root 97142441 Jan 3 15:07 initrd-5.14.21-150500.55.39-default -rwxrwxrwx 1 root root 97182349 Jan 20 11:19 initrd-5.14.21-150500.55.44-default
Dropping back to OpenSuSE 15.5 with Linux 5.14.21-150500.55.39-default works fine.
OpenSuSE 15.5. with Linux 5.14.21-150500.55.44-default fails
When you boot into the previous kernel as Felix suggested, the root file system is mounted read-only. This means you cannot make permanent changes to the system at this point. To allow that you must first _rollback_ the system to the old (working) kernel.
You confuse "boot previous kernel" with "boot from snapshot".
I think that, if I boot from a snapshot that had a different (previous) kernel version, then I am booting a previous kernel, yes? ;)
Darryl Gregorash composed on 2024-01-21 22:56 (UTC-0600):
Andrei Borzenkov wrote:
You confuse "boot previous kernel" with "boot from snapshot".
I think that, if I boot from a snapshot that had a different (previous) kernel version, then I am booting a previous kernel, yes? ;)
I suggested to boot a previous kernel, not an entire previous operating system, which is what booting a snapshot amounts to; to change fewest practical potential failure points at a time, not many. Booting a snapshot facilitates getting back in business quickly well, but not so much isolating or solving a problem arising from updating. -- Evolution as taught in public schools is, like religion, based on faith, not based on science. Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata
On 2024-01-21 23:21, Felix Miata wrote:
Darryl Gregorash composed on 2024-01-21 22:56 (UTC-0600):
Andrei Borzenkov wrote:
You confuse "boot previous kernel" with "boot from snapshot".
I think that, if I boot from a snapshot that had a different (previous) kernel version, then I am booting a previous kernel, yes? ;)
I suggested to boot a previous kernel, not an entire previous operating system, which is what booting a snapshot amounts to; to change fewest practical potential failure points at a time, not many. Booting a snapshot facilitates getting back in business quickly well, but not so much isolating or solving a problem arising from updating. But what is there in anything Marc has posted so far that suggests simply recreating the initramfs file will solve the problem? If the kernel update somehow did not complete properly, then I doubt dracut/mkinitrd will resolve the issue. Assuming that Marc cannot simply do a system rollback, would it make sense to a) boot into the previous kernel b) do an unconditional update on the following packages: kernel-default kernel-default-extra kernel-default-optional c) reboot ?
Darryl Gregorash composed on 2024-01-22 00:49 (UTC-0500):
Felix Miata wrote:
Darryl Gregorash composed on 2024-01-21 22:56 (UTC-0600):
Andrei Borzenkov wrote:
You confuse "boot previous kernel" with "boot from snapshot".
I think that, if I boot from a snapshot that had a different (previous) kernel version, then I am booting a previous kernel, yes? ;)
I suggested to boot a previous kernel, not an entire previous operating system, which is what booting a snapshot amounts to; to change fewest practical potential failure points at a time, not many. Booting a snapshot facilitates getting back in business quickly well, but not so much isolating or solving a problem arising from updating.
But what is there in anything Marc has posted so far that suggests simply recreating the initramfs file will solve the problem?
He's the one with the problem. We're here to help him solve it. That includes possible troubleshooting steps, one of which is to try regenerating the initrd.
If the kernel update somehow did not complete properly, then I doubt dracut/mkinitrd will resolve the issue.
Maybe it would, maybe it wouldn't. It's a troubleshooting step. We don't yet know what the issue causing the failure is.
Assuming that Marc cannot simply do a system rollback, would it make sense to a) boot into the previous kernel b) do an unconditional update on the following packages: kernel-default kernel-default-extra kernel-default-optional c) reboot ?
Looks like a longer, more complicated series of steps to me, which I'm in no position to test. /If/ he's on an recent installation, or is otherwise using btrfs and snapshotting, whether or not he accepted the partitioning defaults or otherwise configured btrfs and snapshotting, /then/ it's a path he could try. What I suggested would work or not regardless his / filesystem, and whether or not snapshotting is available to him. Your way will require he reinstall all updates or wait for some of those to be installed again, plus new. My way he keeps all the updates unrelated to kernel and existing initrd. Both ways he's using the older kernel and doesn't know what went wrong until he does some troubleshooting. These describe options, not demands. I'm wild-guessing his problem is either kernel backporting introduced a problem with his quite youthful CPU that escaped QA, or more likely, a backport introduced a need for NVidia driver update that has yet to become available, if even recognized. -- Evolution as taught in public schools is, like religion, based on faith, not based on science. Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata
Darryl Gregorash composed on 2024-01-22 00:49 (UTC-0500):
If the kernel update somehow did not complete properly, then I doubt dracut/mkinitrd will resolve the issue.
Maybe it would, maybe it wouldn't. It's a troubleshooting step. We don't yet know what the issue causing the failure is. If the problem was just an error in creating the initramfs file, then dracut or mkinitrd alone should work. If there was an error somewhere else in the kernel update, running either of those would not succeed, and actually might make things worse.
Assuming that Marc cannot simply do a system rollback, would it make sense to a) boot into the previous kernel b) do an unconditional update on the following packages: kernel-default kernel-default-extra kernel-default-optional c) reboot ?
Looks like a longer, more complicated series of steps to me, which I'm in no position to test. /If/ he's on an recent installation, or is otherwise using btrfs and snapshotting, whether or not he accepted the partitioning defaults or otherwise configured btrfs and snapshotting, /then/ it's a path he could try. What What I just suggested there has nothing to do with the file system on /. Read the opening clause there, please. What I am suggesting is booting into the previous kernel, then doing an unconditional update -- nothing else -- on the new kernel packages. Then we can be assured that the kernel would be installed from beginning to end, including re-creating the initramfs (it's in the post-install
I suggested would work or not regardless his / filesystem, and whether or not snapshotting is available to him. Your way will require he reinstall all updates or wait for some of those to be installed again, plus new. My way he keeps all the updates unrelated to kernel and existing initrd. Both ways he's using the older kernel and doesn't know what went wrong until he does some troubleshooting. These describe options, not demands. What is "my way"? I suggested 2 ways, in fact -- one I know enough about to make a suggestion, _if_ the pre-conditions are satisfied (they're not, so we can forget about rollbacks now, yes?) The second method I had in mind actually was to roll back the kernel using the Yast snapper module. THAT I agree is a process that would take some work to get the job done. That is one reason I never went into it in more detail. Since then, a third method occurred to me -- as a result of your suggestion to use dracut or mkinitrd, in fact. As I have suggested, if
On 2024-01-22 00:37, Felix Miata wrote: scripts, after all). That would actually save a lot of time, as I rather imagine Marc would have quite a bit of work to do to learn how to use either dracut or mkinitrd in this situation. I know I would. Why not let the system do all that for you? there is a problem with the new kernel other than just an errant intitramfs, using either of those would be useless at best, problematic at worst. So why not force a refresh of those 3 kernel packages (they're the only ones involved in that update), and make sure it all gets done properly?
I'm wild-guessing his problem is either kernel backporting introduced a problem with his quite youthful CPU that escaped QA, or more likely, a backport introduced a need for NVidia driver update that has yet to become available, if even recognized.
If a kernel refresh/reboot fails to resolve the issue, then, yes, it becomes necessary to look beyond the kernel for problems.
Hi Guys, oh my quite a flurry of activity this afternoon while I was away! Thanks so much for all your input, and I will answer the burning question of the moment. My root / partition is formatted with Ext4, what I think was once called Reiserfs or something like that. I know little about all these different formats, just that I tried Btrfs once, got into some trouble with it, and skedaddled back to Ext4 and stuck with it ever since because it has never cause me any trouble that I know of. But like I said, my knowledge of the advantages versus the disadvantages of different formats isn't worth the paper this note is written on! FWIW the last thing I expected to be talking about, when I started this thread/query is the format of my partition! LOL Amazing where things can lead to... Marc... On 1/21/24 21:49, Darryl Gregorash wrote:
On 2024-01-21 23:21, Felix Miata wrote:
Darryl Gregorash composed on 2024-01-21 22:56 (UTC-0600):
Andrei Borzenkov wrote:
You confuse "boot previous kernel" with "boot from snapshot".
I think that, if I boot from a snapshot that had a different (previous) kernel version, then I am booting a previous kernel, yes? ;)
I suggested to boot a previous kernel, not an entire previous operating system, which is what booting a snapshot amounts to; to change fewest practical potential failure points at a time, not many. Booting a snapshot facilitates getting back in business quickly well, but not so much isolating or solving a problem arising from updating. But what is there in anything Marc has posted so far that suggests simply recreating the initramfs file will solve the problem? If the kernel update somehow did not complete properly, then I doubt dracut/mkinitrd will resolve the issue. Assuming that Marc cannot simply do a system rollback, would it make sense to a) boot into the previous kernel b) do an unconditional update on the following packages: kernel-default kernel-default-extra kernel-default-optional c) reboot ?
Hello? I am still not grokking what I should do next, seems like there was a lot of discussion but no resolution. Anyone up to the task of guiding me and baby steps please, I am NOT a guru and totally feel like a fish out of water here... I am getting more software update notifications, can I apply them to my OpenSuSE 15.5 with Linux 5.14.21-150500.55.39-default system or should I wait until I get this earlier upgrade resolved first? Marc... On 1/21/24 23:11, Marc Chamberlin via openSUSE Users wrote:
Hi Guys, oh my quite a flurry of activity this afternoon while I was away! Thanks so much for all your input, and I will answer the burning question of the moment. My root / partition is formatted with Ext4, what I think was once called Reiserfs or something like that. I know little about all these different formats, just that I tried Btrfs once, got into some trouble with it, and skedaddled back to Ext4 and stuck with it ever since because it has never cause me any trouble that I know of.
But like I said, my knowledge of the advantages versus the disadvantages of different formats isn't worth the paper this note is written on! FWIW the last thing I expected to be talking about, when I started this thread/query is the format of my partition! LOL Amazing where things can lead to...
Marc...
On 1/21/24 21:49, Darryl Gregorash wrote:
On 2024-01-21 23:21, Felix Miata wrote:
Darryl Gregorash composed on 2024-01-21 22:56 (UTC-0600):
Andrei Borzenkov wrote:
You confuse "boot previous kernel" with "boot from snapshot".
I think that, if I boot from a snapshot that had a different (previous) kernel version, then I am booting a previous kernel, yes? ;)
I suggested to boot a previous kernel, not an entire previous operating system, which is what booting a snapshot amounts to; to change fewest practical potential failure points at a time, not many. Booting a snapshot facilitates getting back in business quickly well, but not so much isolating or solving a problem arising from updating. But what is there in anything Marc has posted so far that suggests simply recreating the initramfs file will solve the problem? If the kernel update somehow did not complete properly, then I doubt dracut/mkinitrd will resolve the issue. Assuming that Marc cannot simply do a system rollback, would it make sense to a) boot into the previous kernel b) do an unconditional update on the following packages: kernel-default kernel-default-extra kernel-default-optional c) reboot ?
Marc Chamberlin composed on 2024-01-25 01:09 (UTC-0500):
I am getting more software update notifications, can I apply them to my OpenSuSE 15.5 with Linux 5.14.21-150500.55.39-default system or should I wait until I get this earlier upgrade resolved first?
I wouldn't hesitate to apply updates, as long as your current .55.39 initrd and kernel are protected from removal or replacement. If you haven't already, lock the kernel: zypper al kernel-desktop-5.14.21-150500.55.39.1 You could instead do: zypper al kernel-de* That would prevent all kernel-desktop installations and removals as long as it remains in place, forcing such changes to wait until some safety in doing so arises. Once an initrd is known working here, I make it immutable, preventing any change to it. chattr +i /boot/initrd-5.14.21-150500.55.39* -- Evolution as taught in public schools is, like religion, based on faith, not based on science. Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata
On 2024-01-22 05:56, Darryl Gregorash wrote:
On 2024-01-21 22:06, Andrei Borzenkov wrote:
On 22.01.2024 02:42, Darryl Gregorash wrote:
On 2024-01-20 16:08, Marc Chamberlin via openSUSE Users wrote:
On 1/20/24 13:08, Felix Miata wrote:
Marc Chamberlin composed on 2024-01-20 12:57 (UTC-0800):
Note in /boot/ whether the initrds all have today's timestamps, or only just the newest kernel. Reboot selecting the prior kernel from advanced options. What happens? If same problem, and the older kernel's initrd was rebuilt today, try restoring the initrd from your backup for prior kernel, then try booting using it.
The initrds have different timestamps -
-rwxrwxrwx 1 root root 97142441 Jan 3 15:07 initrd-5.14.21-150500.55.39-default -rwxrwxrwx 1 root root 97182349 Jan 20 11:19 initrd-5.14.21-150500.55.44-default
Dropping back to OpenSuSE 15.5 with Linux 5.14.21-150500.55.39-default works fine.
OpenSuSE 15.5. with Linux 5.14.21-150500.55.44-default fails
When you boot into the previous kernel as Felix suggested, the root file system is mounted read-only. This means you cannot make permanent changes to the system at this point. To allow that you must first _rollback_ the system to the old (working) kernel.
You confuse "boot previous kernel" with "boot from snapshot".
I think that, if I boot from a snapshot that had a different (previous) kernel version, then I am booting a previous kernel, yes? ;)
No. Yes, it will be a previous kernel if you choose the right snapshot, but the meaning in suseland of "booting a previous kernel" is boot, and in grub menu choose the previous kernel. Not a previous snapshot. The result is a system running the previous kernel and all is mounted normally, r/w. -- Cheers / Saludos, Carlos E. R. (from openSUSE 15.5 (Laicolasse))
participants (5)
-
Andrei Borzenkov -
Carlos E. R. -
Darryl Gregorash -
Felix Miata -
Marc Chamberlin