15.4, how to repair a malfunctioning system - offline usb key re-install or something?

Support, some days ago, I probably have had power problems at a remote far away leap 15.4 system. I was kind of in the middle of zypper up fun things were that apparently there were some rpm db corruptions, which I rebuilt, but then more stuff, such as openssl_1_1_1 was installed twice on the system so it showed with a query, I never seen such thing before. also some further zypper ref or zypper up showed conflicts speaking about dependancies missing but stuff already been downloaded into the cache then again it didnt want to update or install some packages from the recent days updates. then i went into yast and its software management, and searched for openssl and it had those two lines openssl_1_1_1 one with the "i" same other package further below with "a-" which it wanted to remove? at the end now this system is still connected to by a remote ssh shell, but i can not execute rpm any more, it complains about libgrypt.so.20 missing is it still possible to fix this dang machine from far this way? i can not create additional ssh logins into this machine at the moment, only the still existing established connection I currently have. i have fetched the latest leap 15.4 x64 iso the updated one from get.opensuse.org or so I figure, from februar this year. can i easily fix this machine by booting into the usb key and the leap 15.4 installer and make it install over the somewhat messed up broken 15.4 machine? what would be a simple way to fix things? does the installer check and support kind of a repairing the already installed 15.4 system and so on? thanks for helpers in this matter.

Hi cagsm, On Tue, 28 Mar 2023, 18:09:36 +0200, cagsm wrote:
Support,
some days ago, I probably have had power problems at a remote far away leap 15.4 system. I was kind of in the middle of zypper up
fun things were that apparently there were some rpm db corruptions, which I rebuilt, but then more stuff, such as openssl_1_1_1 was installed twice on the system so it showed with a query, I never seen such thing before.
also some further zypper ref or zypper up showed conflicts speaking about dependancies missing but stuff already been downloaded into the cache then again it didnt want to update or install some packages from the recent days updates.
then i went into yast and its software management, and searched for openssl and it had those two lines openssl_1_1_1 one with the "i" same other package further below with "a-" which it wanted to remove?
at the end now this system is still connected to by a remote ssh shell, but i can not execute rpm any more, it complains about libgrypt.so.20 missing
is it still possible to fix this dang machine from far this way? i can not create additional ssh logins into this machine at the moment, only the still existing established connection I currently have.
you should copy the missing .rpm file to that machine, then install it just to the rpmdb: rpm -ivh --justdb libgcrypt20-1.9.4-150400.6.8.1.x86_64.rpm This will just update the RPM database, but not touch the filesystem.
i have fetched the latest leap 15.4 x64 iso the updated one from get.opensuse.org or so I figure, from februar this year.
can i easily fix this machine by booting into the usb key and the leap 15.4 installer and make it install over the somewhat messed up broken 15.4 machine?
what would be a simple way to fix things? does the installer check and support kind of a repairing the already installed 15.4 system and so on? thanks for helpers in this matter.
I'd recommend to try fixing the RPM database as far as possible. FWIW, I've done that before ;) HTH, cheers. l8er manfred

On Tue, 28 Mar 2023, 18:53:06 +0200, Manfred Hollstein wrote:
Hi cagsm,
On Tue, 28 Mar 2023, 18:09:36 +0200, cagsm wrote:
Support,
some days ago, I probably have had power problems at a remote far away leap 15.4 system. I was kind of in the middle of zypper up
fun things were that apparently there were some rpm db corruptions, which I rebuilt, but then more stuff, such as openssl_1_1_1 was installed twice on the system so it showed with a query, I never seen such thing before.
also some further zypper ref or zypper up showed conflicts speaking about dependancies missing but stuff already been downloaded into the cache then again it didnt want to update or install some packages from the recent days updates.
then i went into yast and its software management, and searched for openssl and it had those two lines openssl_1_1_1 one with the "i" same other package further below with "a-" which it wanted to remove?
at the end now this system is still connected to by a remote ssh shell, but i can not execute rpm any more, it complains about libgrypt.so.20 missing
is it still possible to fix this dang machine from far this way? i can not create additional ssh logins into this machine at the moment, only the still existing established connection I currently have.
you should copy the missing .rpm file to that machine, then install it just to the rpmdb:
rpm -ivh --justdb libgcrypt20-1.9.4-150400.6.8.1.x86_64.rpm
This will just update the RPM database, but not touch the filesystem.
i have fetched the latest leap 15.4 x64 iso the updated one from get.opensuse.org or so I figure, from februar this year.
can i easily fix this machine by booting into the usb key and the leap 15.4 installer and make it install over the somewhat messed up broken 15.4 machine?
what would be a simple way to fix things? does the installer check and support kind of a repairing the already installed 15.4 system and so on? thanks for helpers in this matter.
I'd recommend to try fixing the RPM database as far as possible. FWIW, I've done that before ;)
Forgot to mention, look at the output of "rpm -V -a" and check if the complaints are correct. HTH, cheers. l8er manfred

On Tue, Mar 28, 2023 at 6:54 PM Manfred Hollstein <mhollstein@t-online.de> wrote:
I'd recommend to try fixing the RPM database as far as possible. FWIW, I've done that before ;) Forgot to mention, look at the output of "rpm -V -a" and check if the complaints are correct.
unfortunately the rpm command does not work at all. it immediately brings up the libgcrypt.so.20 file not found or directory not found error message for all parameters i am trying. i guess even if i could transfer (uuencode? uudecode? copy paste via ssh) the rpm file, how would I be able to fix the libgcrypt stuff if rpm itself is already borked. ty.

On 28.03.2023 19:58, cagsm wrote:
On Tue, Mar 28, 2023 at 6:54 PM Manfred Hollstein <mhollstein@t-online.de> wrote:
I'd recommend to try fixing the RPM database as far as possible. FWIW, I've done that before ;) Forgot to mention, look at the output of "rpm -V -a" and check if the complaints are correct.
unfortunately the rpm command does not work at all. it immediately brings up the libgcrypt.so.20 file not found or directory not found error message for all parameters i am trying.
i guess even if i could transfer (uuencode? uudecode? copy paste via ssh) the rpm file, how would I be able to fix the libgcrypt stuff if rpm itself is already borked.
You can unpack/extract files from RPM package on another system (rpm2cpio if it is Linux, otherwise 7-Zip supports RPM "archives" on Windows) and copy this library to your system. /usr/lib64/libgcrypt.so.20 is provided by package libgcrypt20-1.9.4-150400.6.8.1.x86_64 (at least, on fully updated system). Can you run ssh/sshd on the remote system?
ty.

On Tue, 28 Mar 2023, 18:58:55 +0200, cagsm wrote:
On Tue, Mar 28, 2023 at 6:54 PM Manfred Hollstein <mhollstein@t-online.de> wrote:
I'd recommend to try fixing the RPM database as far as possible. FWIW, I've done that before ;) Forgot to mention, look at the output of "rpm -V -a" and check if the complaints are correct.
unfortunately the rpm command does not work at all. it immediately brings up the libgcrypt.so.20 file not found or directory not found error message for all parameters i am trying.
i guess even if i could transfer (uuencode? uudecode? copy paste via ssh) the rpm file, how would I be able to fix the libgcrypt stuff if rpm itself is already borked.
Boot from the USB .iso and choose Rescue system, then mount your root file system to /mnt, bind mount the usual /dev, /proc, /run, /sys, and /sys/firmware/efi/efivars directories: os=/mnt; mount --bind /dev $os/dev; mount --bind /proc $os/proc; mount --bind /run $os/run; mount --bind /sys $os/sys; mount --bind /sys/firmware/efi/efivars $os/sys/firmware/efi/efivars Copy the downloaded libgcrypt.rpm file to /mnt/root and do chroot /mnt; cd rpm -ivh --justdb libgcrypt.rpm You can then test it by running "rpm -qa". If everything is OK at that point, leave the chroot by typing "exit" and reboot into the 15.4 system to do further analysis by running "rpm -Va".
ty.
HTH, cheers. l8er manfred

On Tue, 28 Mar 2023, 19:39:06 +0200, Manfred Hollstein wrote:
On Tue, 28 Mar 2023, 18:58:55 +0200, cagsm wrote:
On Tue, Mar 28, 2023 at 6:54 PM Manfred Hollstein <mhollstein@t-online.de> wrote:
I'd recommend to try fixing the RPM database as far as possible. FWIW, I've done that before ;) Forgot to mention, look at the output of "rpm -V -a" and check if the complaints are correct.
unfortunately the rpm command does not work at all. it immediately brings up the libgcrypt.so.20 file not found or directory not found error message for all parameters i am trying.
i guess even if i could transfer (uuencode? uudecode? copy paste via ssh) the rpm file, how would I be able to fix the libgcrypt stuff if rpm itself is already borked.
Boot from the USB .iso and choose Rescue system, then mount your root file system to /mnt, bind mount the usual /dev, /proc, /run, /sys, and /sys/firmware/efi/efivars directories:
os=/mnt; mount --bind /dev $os/dev; mount --bind /proc $os/proc; mount --bind /run $os/run; mount --bind /sys $os/sys; mount --bind /sys/firmware/efi/efivars $os/sys/firmware/efi/efivars
Copy the downloaded libgcrypt.rpm file to /mnt/root and do
chroot /mnt; cd
rpm -ivh --justdb libgcrypt.rpm
WRONG, of course :( Instead of chroot'ing, you should use the *working* rpm command of the Rescue system to install the downloaded package to the broken system: rpm --root /mnt -ivh libgcrypt.rpm If it is actually missing from the broken system, just omit the "--justdb" from the command. Hope you get the idea.
You can then test it by running "rpm -qa". If everything is OK at that point, leave the chroot by typing "exit" and reboot into the 15.4 system to do further analysis by running "rpm -Va".
This can still be done after installation of the package.
ty.
HTH, cheers. l8er manfred

Am 28.03.23 um 18:58 schrieb cagsm:
On Tue, Mar 28, 2023 at 6:54 PM Manfred Hollstein <mhollstein@t-online.de> wrote:
I'd recommend to try fixing the RPM database as far as possible. FWIW, I've done that before ;) Forgot to mention, look at the output of "rpm -V -a" and check if the complaints are correct.
unfortunately the rpm command does not work at all. it immediately brings up the libgcrypt.so.20 file not found or directory not found error message for all parameters i am trying.
i guess even if i could transfer (uuencode? uudecode? copy paste via ssh) the rpm file, how would I be able to fix the libgcrypt stuff if rpm itself is already borked. ty.
Does wget or curl still work on the remote box? Then you could download the affected rpm's and extract it (as Andrei already suggested) with #> rpm2cpio "$rpmfile" | cpio -diumv --no-absolute-filenames Otherwise, you can transfer the rpm or the affected library files with xclip. Just tried it, it works. localbox#> xclip mylib.rpm remotebox#> xclip -o > mylib.rpm HTH, Manfred BTW, because of personal bad experiences: I always install a statically compiled busybox on all my boxes. So you can even delete /lib64/libc.so.6 and survive ...

thanks for all the helping hand in this thread. In the end I rebooted the machine on site physically, and booted the opensuse 15.4 iso usb key, and selected upgrade from its boot menu. it detected the already installed 15.4 on the hdd and updated it once again a few thousand? packages. in the end this brought back the machine alive and now it works again. thanks.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wednesday, 2023-03-29 at 01:25 +0200, cagsm wrote:
thanks for all the helping hand in this thread. In the end I rebooted the machine on site physically, and booted the opensuse 15.4 iso usb key, and selected upgrade from its boot menu. it detected the already installed 15.4 on the hdd and updated it once again a few thousand? packages. in the end this brought back the machine alive and now it works again.
As the database is nearly empty or old, it thinks it has to install everything. It doesn't matter, it overwrites everything with the same file if it exists, and the result is correct. - -- Cheers, Carlos E. R. (from openSUSE 15.4 x86_64 at Telcontar) -----BEGIN PGP SIGNATURE----- iHoEARECADoWIQQZEb51mJKK1KpcU/W1MxgcbY1H1QUCZCOWRRwccm9iaW4ubGlz dGFzQHRlbGVmb25pY2EubmV0AAoJELUzGBxtjUfV/rwAnArNliwP2eoBzm4jGQsF d2iURxl+AJoDz4+H1FexPu8qHDXdQ3Gp4BRH6Q== =WyPT -----END PGP SIGNATURE-----

On 2023-03-28 18:09, cagsm wrote: ...
can i easily fix this machine by booting into the usb key and the leap 15.4 installer and make it install over the somewhat messed up broken 15.4 machine?
what would be a simple way to fix things? does the installer check and support kind of a repairing the already installed 15.4 system and so on? thanks for helpers in this matter.
In the past, I used the iso choosing "upgrade". But I used this method yesterday and did not work. You know that there is a backup of the rpm database. In any case, make a second copy of that before it is rotated out of existence. /var/adm/backup/rpmdb -- Cheers / Saludos, Carlos E. R. (from Elesar, using openSUSE Leap 15.4)
participants (5)
-
Andrei Borzenkov
-
cagsm
-
Carlos E. R.
-
Manfred Hollstein
-
Manfred Schwarb