[opensuse] Recover 12.3 system - help needed
Hi, During routine system update (OS12.3, 64-bit) by yast a couple of weeks ago something happened either during kernel update (3.7.10.-1.36-desktop) or nvidia driver update and the system is not bootable anymore. I can boot two other systems on the same HD (12.2 and 13.1). artition of 12.3 seems OK when I mount it from another system. When I attempt to boot 12.3 (no matter what option), (I even tried adding debug options for kernel) I see that initrd "boot" scripts are executed (11-block, 82-resume.kernel and so on) but the following error message is displayed a number of times: udevadm: error while loading shared libraries: libz.so.1: cannot open shared object file: no such file or directory. After some time I see the message: Could not find /dev/root Want me to fall back to /dev/disk/by-id/xxxxx (Y/n) and at this point only "three finger salute" works I see that libz.so.1 is present in /usr/local/ilb in initrd filesystem. I even tried loading the rescue system and updated the kernel to 3.7.10-1.40, so initrd has been re-created, but the result is the same. Any advice on how to recover the system / investigate the failure? Regards -- Mark Goldstein -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Saturday, 2014-08-09 at 15:06 +0300, Mark Goldstein wrote:
I see that libz.so.1 is present in /usr/local/ilb in initrd filesystem.
Strange. In my 13.1 it is in /lib64 (I don't have a running 12.3 system available)
I even tried loading the rescue system and updated the kernel to 3.7.10-1.40, so initrd has been re-created, but the result is the same.
Where is that library in the real system? - -- Cheers, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iEYEARECAAYFAlPmG/0ACgkQtTMYHG2NR9XYDACfeTuJT+NdEScSTQz0qK3BeL89 lmUAnRKR0WvEuoS0tiFGZUZar7AeNTru =kZb5 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sat, Aug 9, 2014 at 4:02 PM, Carlos E. R. <robin.listas@telefonica.net> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Saturday, 2014-08-09 at 15:06 +0300, Mark Goldstein wrote:
I see that libz.so.1 is present in /usr/local/ilb in initrd filesystem.
Strange.
In my 13.1 it is in /lib64 (I don't have a running 12.3 system available)
Yes, I also that in 13.1 in is in normal place and thought something might be wrong with initrd. ...
Where is that library in the real system?
Hmm, there are versions in all 3 places in 12.3: /lib and /lib64 contain libz.so.1.2.7 while /usr/local/lib contains libz.so.1.2.8 The one in initrd in /usr/local/lib is 1.2.8... so maybe there still shall be a version in /lib64. I have another instance of 12.3 under VirtualBox in my laptop (though 32bit version). I'll check there. -- Mark Goldstein -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sat, Aug 9, 2014 at 4:40 PM, Mark Goldstein <goldstein.mark@gmail.com> wrote:
On Sat, Aug 9, 2014 at 4:02 PM, Carlos E. R. <robin.listas@telefonica.net> wrote:
Where is that library in the real system?
Hmm, there are versions in all 3 places in 12.3: /lib and /lib64 contain libz.so.1.2.7 while /usr/local/lib contains libz.so.1.2.8
The one in initrd in /usr/local/lib is 1.2.8... so maybe there still shall be a version in /lib64. I have another instance of 12.3 under VirtualBox in my laptop (though 32bit version). I'll check there.
In 32-bit version libz.so.1.2.7 is in /lib in initrd... Something's seems wrong with this new initrd. But... why old kernel could not boot also? I'll re-check, maybe I did not try old kernel and only failsafe mode. -- Mark Goldstein -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sat, Aug 9, 2014 at 5:27 PM, Mark Goldstein <goldstein.mark@gmail.com> wrote:
On Sat, Aug 9, 2014 at 4:40 PM, Mark Goldstein <goldstein.mark@gmail.com> wrote:
On Sat, Aug 9, 2014 at 4:02 PM, Carlos E. R. <robin.listas@telefonica.net> wrote:
Where is that library in the real system?
Hmm, there are versions in all 3 places in 12.3: /lib and /lib64 contain libz.so.1.2.7 while /usr/local/lib contains libz.so.1.2.8
The one in initrd in /usr/local/lib is 1.2.8... so maybe there still shall be a version in /lib64. I have another instance of 12.3 under VirtualBox in my laptop (though 32bit version). I'll check there.
In 32-bit version libz.so.1.2.7 is in /lib in initrd...
Something's seems wrong with this new initrd. But... why old kernel could not boot also? I'll re-check, maybe I did not try old kernel and only failsafe mode.
OK, I checked that all initrds (for old kernel as well) were re-created during update, so if someting's wrong there, old kernel will be influeced as well. Probably I'll follow Felix' advice to re-install libz package from rescue system and then mkinitrd as in example given by Carl. -- Mark Goldstein -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2014-08-09 19:20 (GMT+0300) Mark Goldstein composed:
OK, I checked that all initrds (for old kernel as well) were re-created during update, so if someting's wrong there, old kernel will be influeced as well.
Probably I'll follow Felix' advice to re-install libz package from rescue system and then mkinitrd as in example given by Carl.
Might be interesting to try a foreign initrd. This is from an Intel 945G installation generated a month ago: http://fm.no-ip.com/Tmp/SUSE/123/initrd-3.7.10-1.36-desktop Previously installed kernel version: http://fm.no-ip.com/Tmp/SUSE/123/initrd-3.7.10-1.28-desktop I get irritated that openSUSE regenerates initrds that worked, so am in the habit of either backing them up, and/or making them immutable until time to retire them arrives. -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-------------------------------------------- On Sat, 8/9/14, Felix Miata <mrmazda@earthlink.net> wrote: Subject: Re: [opensuse] Recover 12.3 system - help needed To: opensuse@opensuse.org Date: Saturday, August 9, 2014, 12:50 PM On 2014-08-09 19:20 (GMT+0300) Mark Goldstein composed:
OK, I checked that all initrds (for old kernel as well) were re-created during update, so if someting's wrong there, old kernel will be influeced as well.
Probably I'll follow Felix' advice to re-install libz package from rescue system and then mkinitrd as in example given by Carl
If you put the root partition number of 12.3 in fstab of your 13.1 installation (you have to call it something different from /) run the command df -h. I sometimes had a problem with the 12.3 root partition filling up so that I could not boot. I manually cleaned up some temp files and some log warnings, etc in order to boot again. This worked for me, I hope it does for you. Peter -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2014-08-09 15:06 (GMT+0300) Mark Goldstein composed:
During routine system update (OS12.3, 64-bit) by yast a couple of weeks ago something happened either during kernel update (3.7.10.-1.36-desktop) or nvidia driver update and the system is not bootable anymore.
I can boot two other systems on the same HD (12.2 and 13.1). artition of 12.3 seems OK when I mount it from another system.
When I attempt to boot 12.3 (no matter what option), (I even tried adding debug options for kernel) I see that initrd "boot" scripts are
Did you try the failsafe Grub menu selection? Did you try any prior kernel Grub menu selection?
executed (11-block, 82-resume.kernel and so on) but the following error message is displayed a number of times:
udevadm: error while loading shared libraries: libz.so.1: cannot open shared object file: no such file or directory.
After some time I see the message:
Could not find /dev/root Want me to fall back to /dev/disk/by-id/xxxxx (Y/n)
and at this point only "three finger salute" works
I see that libz.so.1 is present in /usr/local/ilb in initrd filesystem.
I even tried loading the rescue system and updated the kernel to 3.7.10-1.40, so initrd has been re-created, but the result is the same.
Any advice on how to recover the system / investigate the failure?
First thing I'd try is chrooting to 12.3 from 12.2 or 13.1, using file to discover the package containing libz.so.1, then force reinstalling that package using zypper. You can use lsinitrd to verify that the existing initrds contain that file or a reference to it. You can scan /var/log/zypp/history and /var/log/zypper.log for clues to any packages producing error messages during the install process. Maybe a 'zypper ref; zypper up' will fix something that broke during the last updates. Recall that Nvidia drivers require nomodeset on cmdline, just in case updating bootloader didn't take that into account. If nothing seems to help, share lspci output from 12.2 or 13.1 when continuing this thread. -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sat, Aug 9, 2014 at 4:13 PM, Felix Miata <mrmazda@earthlink.net> wrote:
Did you try the failsafe Grub menu selection? Did you try any prior kernel Grub menu selection?
Yes, same result for failsafe and for previous kernel version...
First thing I'd try is chrooting to 12.3 from 12.2 or 13.1, using file to discover the package containing libz.so.1, then force reinstalling that package using zypper. You can use lsinitrd to verify that the existing initrds contain that file or a reference to it. You can scan /var/log/zypp/history and /var/log/zypper.log for clues to any packages producing error messages during the install process. Maybe a 'zypper ref; zypper up' will fix something that broke during the last updates.
I chrooted from rescue system, but I've only updated the kernel since I thought the problem is in initrd. I'll try what you suggested.
Recall that Nvidia drivers require nomodeset on cmdline, just in case updating bootloader didn't take that into account.
I think it was true for some old cards/drivers. I'm not using it in any of my systems now. Thanks, -- Mark Goldstein -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Sat, 9 Aug 2014 15:06:22 +0300 Mark Goldstein wrote: <snipped>
Any advice on how to recover the system / investigate the failure?
Regards
Hi Mark, My system is running 12.3 x86_64 and it has twice experienced a somewhat comparable broken state following the last two kernel updates. I also have additional operating environments installed: Mint 14 and Vista. I presumed this behavior was an artifact of having just migrated the installed systems to an SSD. This could actually still be the case but I'm not certain. My solution has been to boot to rescue mode from my openSUSE *12.2* install DVD Side note: The 12.3 GM DVD requires additional boot parameters due to an ACPI related bug that I simply prefer to avoid. YMMV. Continuing... I mount the affected '/' partition to /mnt , then 'mount --bind /dev /mnt/dev' 'mount --bind /proc /mnt/proc' 'mount --bind /sys /mnt/sys' 'chroot /mnt' 'mkinitrd' I reinstall grub2: 'grub2-install /dev/sda' (select the desired device/partition here) 'grub2-mkconfig -o /boot/grub2/grub.cfg' When this is done, I 'exit', 'umount -a', 'eject /dev/sr0', remove the DVD and 'poweroff' (I want a cold start.) Notes and caveats: This failed the first time because I'd forgotten to adapt Mint's boot configuration to the SSD, which threw off detection and resulted in a broken boot configuration. To remedy this, I first treated Mint as though it were the 'primary' installed environment ... using Mint-specific grub2 installation commands, btw ... and the subsequent pass treating openSUSE 12.3 as 'primary' succeeded. I haven't needed to touch the Mint 14 configuration on the subsequent boot / initrd repairs. Another aspect of this: The same problem has been experienced during the same kernel updates on my daughter's computer. It also has openSUSE 12.3 and Vista installed (no Mint.) Finally, both of these systems have nVidia graphics adapters with the proprietary drivers installed. They, of course, need to be re-installed when the kernel is updated. Here are relevant excerpts of the error messages from the first incident on my system: - - - - - 8< - - - - - (4/6) Installing: nvidia-uvm-gfxG03-kmp-desktop-331.89_k3.7.10_1.1-28.1 ...............................................[done] Additional rpm output: /usr/src/kernel-modules/nvidia-uvm-331.89-desktop/rm / NVIDIA: calling KBUILD... make[1]: Entering directory `/usr/src/linux-3.7.10-1.36' make -C /usr/src/linux-obj/x86_64/desktop \ KBUILD_SRC=/usr/src/linux-3.7.10-1.36 \ KBUILD_EXTMOD="/usr/src/kernel-modules/nvidia-uvm-331.89-desktop/rm" -f /usr/src/linux-3.7.10-1.36/Makefile \ modules test -e include/generated/autoconf.h -a -e include/config/auto.conf || ( \ echo >&2; \ echo >&2 " ERROR: Kernel configuration is invalid."; \ echo >&2 " include/generated/autoconf.h or include/config/auto.conf are missing.";\ echo >&2 " Run 'make oldconfig && make prepare' on kernel src to fix it."; \ echo >&2 ; \ /bin/false) - - - - - 8< - - - - - and - - - - - 8< - - - - - Kernel image: /boot/vmlinuz-3.7.10-1.32-desktop Initrd image: /boot/initrd-3.7.10-1.32-desktop Root device: /dev/sda5 (mounted on / as ext4) Could not expand UUID=456a524c-9adf-4472-8fb0-88bbdbd62b53 to real device Kernel Modules: thermal_sys thermal processor fan scsi_dh scsi_dh_hp_sw scsi_dh_rdac scsi_dh_emc scsi_dh_alua xhci-hcd hid-logitech-dj Features: acpi plymouth block usb resume.userspace resume.kernel Kernel image: /boot/vmlinuz-3.7.10-1.36-desktop Initrd image: /boot/initrd-3.7.10-1.36-desktop Root device: /dev/sda5 (mounted on / as ext4) Could not expand UUID=456a524c-9adf-4472-8fb0-88bbdbd62b53 to real device Kernel Modules: thermal_sys thermal processor fan scsi_dh scsi_dh_hp_sw scsi_dh_rdac scsi_dh_emc scsi_dh_alua xhci-hcd hid-logitech-dj Features: acpi plymouth block usb resume.userspace resume.kernel Perl-Bootloader: 2014-07-22 18:21:43 <3> pbl-4616.2 Core::RunCommand.1642: Error: Command '/usr/sbin/grub2-install --target=i386-pc --force --skip-fs-probe "(hd0,4)" >/var/log/YaST2/y2log_bootloader 2>&1' failed with code 256 and output: /usr/sbin/grub2-bios-setup: error: disk `hd0,4' not found. There was an error generating the initrd (1) - - - - - 8< - - - - - I don't know for a fact that you're experiencing the identical problem, Mark, but there were enough similarities in your post that I thought it couldn't hurt to share this information with you. Good luck! Carl -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Mark Goldstein wrote:
Could not find /dev/root Want me to fall back to /dev/disk/by-id/xxxxx (Y/n)
---- Has your disk order changed? Can you boot to 'S' (i.e. @ boot prompt, add 'S' for runlevel). Is boot on same device as root partition? Can you mount the root partition from the rescue disk? I.e. say your boot dev is /sda5 and your root partition is on sda1. If you boot to rescue, and you presumably can mount /dev/sda5 (boot) can you also mount your root partition? Is there anything special about your root partition (encrypted, lvm member, software-raid member...etc?) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Mark Goldstein wrote:
Hi,
During routine system update (OS12.3, 64-bit) by yast a couple of weeks ago something happened either during kernel update (3.7.10.-1.36-desktop) or nvidia driver update and the system is not bootable anymore.
I can boot two other systems on the same HD (12.2 and 13.1). artition of 12.3 seems OK when I mount it from another system.
When I attempt to boot 12.3 (no matter what option), (I even tried adding debug options for kernel) I see that initrd "boot" scripts are executed (11-block, 82-resume.kernel and so on) but the following error message is displayed a number of times:
udevadm: error while loading shared libraries: libz.so.1: cannot open shared object file: no such file or directory.
After some time I see the message:
Could not find /dev/root Want me to fall back to /dev/disk/by-id/xxxxx (Y/n)
During initial configuration, ALWAYS ALWAYS ALWAYS use 'mount by label" It eliminates so many problems.
and at this point only "three finger salute" works
I see that libz.so.1 is present in /usr/local/ilb in initrd filesystem.
I even tried loading the rescue system and updated the kernel to 3.7.10-1.40, so initrd has been re-created, but the result is the same.
Any advice on how to recover the system / investigate the failure?
Regards
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (7)
-
Carl Hartung
-
Carlos E. R.
-
Dirk Gently
-
Felix Miata
-
Linda Walsh
-
Mark Goldstein
-
Peter Welles