[opensuse] how to stop dracut rebuilding ALL initrds when updating system?
I have just hosed a customer system when I applied the latest updates for leap421. Not only was the new initrd not built correctly (dracut bug), but the old one was also rebuilt, for no reason whatsoever. Now I'm left with a non-booting system. Not happy. Some of this stuff just isn't being tested, at all. -- Per Jessen, Zürich (19.4°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Tue, 12 Apr 2016 18:12, Per Jessen wrote:
I have just hosed a customer system when I applied the latest updates for leap421. Not only was the new initrd not built correctly (dracut bug), but the old one was also rebuilt, for no reason whatsoever. Now I'm left with a non-booting system.
Not happy. Some of this stuff just isn't being tested, at all.
Not really a solution, but, to protect you other initrds, look at the "immutable" and "undeletable" extended attribute. "chattr +iu <file>" and "man chattr" for more info. I do that before installing a new kernel, by hand. Once bitten, twice as shy. Full rebuild only on libc / udev / systemd updates. And even then I do then sequentially. - Yamaban. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Yamaban wrote:
On Tue, 12 Apr 2016 18:12, Per Jessen wrote:
I have just hosed a customer system when I applied the latest updates for leap421. Not only was the new initrd not built correctly (dracut bug), but the old one was also rebuilt, for no reason whatsoever. Now I'm left with a non-booting system.
Not happy. Some of this stuff just isn't being tested, at all.
Not really a solution, but, to protect you other initrds, look at the "immutable" and "undeletable" extended attribute.
"chattr +iu <file>" and "man chattr" for more info.
I do that before installing a new kernel, by hand. Once bitten, twice as shy.
Thanks for the suggestion, I appreciate it. I'm busy trying to get the system running again. A tad difficult getting a working initrd when dracut is buggy .... I think I'll try going to copy over the faulty modules from 13.2. Unfortunately, I don't have a Leap421 test server. Mea culpa, I should have known better. -- Per Jessen, Zürich (18.9°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 12/04/2016 19:00, Per Jessen a écrit :
modules from 13.2. Unfortunately, I don't have a Leap421 test server. Mea culpa, I should have known better.
can't you boot on the last snapshot? BTRFS is said able to do this? (not a joke) anyway, the rescue menu should allow you to boot last kernel jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
jdd wrote:
Le 12/04/2016 19:00, Per Jessen a écrit :
modules from 13.2. Unfortunately, I don't have a Leap421 test server. Mea culpa, I should have known better.
can't you boot on the last snapshot? BTRFS is said able to do this? (not a joke)
anyway, the rescue menu should allow you to boot last kernel
This system doesn't use butterfs and although I could easily boot the last kernel, the last initrd which was overwritten by the overeager dracut ... -- Per Jessen, Zürich (18.3°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 12/04/2016 19:06, Per Jessen a écrit :
This system doesn't use butterfs
may be should have :-( and although I could easily boot the
last kernel, the last initrd which was overwritten by the overeager dracut ...
from memory, I have three kernels available, but I may be wrong you should give some hardware info, I never had such problem jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
jdd wrote:
and although I could easily boot the last kernel, the last initrd which was overwritten by the overeager dracut ...
from memory, I have three kernels available, but I may be wrong
Yes, the kernels are fine, but when the initrd doesn't work and both new ones and old ones are rebuilt when you upgrade a system, the system will no longer boot.
you should give some hardware info, I never had such problem
Not much point - it's not hardware related. The problem is the iscsi config in the initrd. -- Per Jessen, Zürich (15.6°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Le 12/04/2016 20:03, Per Jessen a écrit :
Not much point - it's not hardware related. The problem is the iscsi config in the initrd.
well...this is hardware :-)) jdd -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Per Jessen composed on 2016-04-12 19:00 (UTC+0200):
Yamaban wrote:
Per Jessen wrote:
I have just hosed a customer system when I applied the latest updates for leap421. Not only was the new initrd not built correctly (dracut bug), but the old one was also rebuilt, for no reason whatsoever. Now I'm left with a non-booting system.
Not happy. Some of this stuff just isn't being tested, at all.
Not really a solution, but, to protect you other initrds, look at the "immutable" and "undeletable" extended attribute.
"chattr +iu <file>" and "man chattr" for more info.
I do that before installing a new kernel, by hand. Once bitten, twice as shy.
I've been doing same nearly forever, with one difference, simply 'chattr +i'. Of what benefit is adding the "u"? The man page doesn't make it clear that u is not implied. In practice, immutable is probably a poor course of action on systems not your own. The automatic kernel retirement scripting likely will barf on it. As a matter of course each update I delete the marker file the kernel installation script creates for retiring older kernels, so don't have any idea of that impact.
Thanks for the suggestion, I appreciate it. I'm busy trying to get the system running again. A tad difficult getting a working initrd when dracut is buggy .... I think I'll try going to copy over the faulty modules from 13.2. Unfortunately, I don't have a Leap421 test server. Mea culpa, I should have known better.
Maybe the shortcut you want is to simply copy this into place: http://fm.no-ip.com/Tmp/SUSE/421/initrd-4.1.15-8-default It's from this B85 Haswell 42.1 installation (which includes RAID1). If you need one from something older with Intel CPU and chipset or with RAID absent, I can probably find you one that will work if this doesn't. I also have one AMD Athlon(tm) II X2 240 on NVIDIA® MCP61 chipset system to get one from. -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 04/12/2016 09:12 AM, Per Jessen wrote:
I have just hosed a customer system when I applied the latest updates for leap421. Not only was the new initrd not built correctly (dracut bug), but the old one was also rebuilt, for no reason whatsoever. Now I'm left with a non-booting system.
Not happy. Some of this stuff just isn't being tested, at all.
Is this reproduceable on any other system or just this specific system? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
sdm wrote:
On 04/12/2016 09:12 AM, Per Jessen wrote:
I have just hosed a customer system when I applied the latest updates for leap421. Not only was the new initrd not built correctly (dracut bug), but the old one was also rebuilt, for no reason whatsoever. Now I'm left with a non-booting system.
Not happy. Some of this stuff just isn't being tested, at all.
Is this reproduceable on any other system or just this specific system?
This would fail on every system that boots from iSCSI. -- Per Jessen, Zürich (15.2°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
12.04.2016 19:12, Per Jessen пишет:
I have just hosed a customer system when I applied the latest updates for leap421. Not only was the new initrd not built correctly (dracut bug), but the old one was also rebuilt, for no reason whatsoever. Now I'm left with a non-booting system.
Not happy. Some of this stuff just isn't being tested, at all.
It's not dracut. Packages that want initrd to be regenerated create flag; if they want to rebuild all initrds, they create /run/regenerate-initrd/all. So it is not dracut fault. E.g. udev does it to name a few. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Andrei Borzenkov wrote:
12.04.2016 19:12, Per Jessen пишет:
I have just hosed a customer system when I applied the latest updates for leap421. Not only was the new initrd not built correctly (dracut bug), but the old one was also rebuilt, for no reason whatsoever. Now I'm left with a non-booting system.
Not happy. Some of this stuff just isn't being tested, at all.
It's not dracut. Packages that want initrd to be regenerated create flag; if they want to rebuild all initrds, they create /run/regenerate-initrd/all. So it is not dracut fault. E.g. udev does it to name a few.
Well, dracut is buggy, and _something_ unnecessarily rebuilt my _working_ initrd and hosed my system. To some extent I can work with dracut being buggy, but I get upset when it overwrites my working setup. I think this is the bug I'm hitting: https://bugzilla.opensuse.org/show_bug.cgi?id=959365 I'm just now building another initrd with the correct iscsi parameters, I'll report back. -- Per Jessen, Zürich (14.1°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Per Jessen wrote:
Andrei Borzenkov wrote:
12.04.2016 19:12, Per Jessen пишет:
I have just hosed a customer system when I applied the latest updates for leap421. Not only was the new initrd not built correctly (dracut bug), but the old one was also rebuilt, for no reason whatsoever. Now I'm left with a non-booting system.
Not happy. Some of this stuff just isn't being tested, at all.
It's not dracut. Packages that want initrd to be regenerated create flag; if they want to rebuild all initrds, they create /run/regenerate-initrd/all. So it is not dracut fault. E.g. udev does it to name a few.
Well, dracut is buggy, and _something_ unnecessarily rebuilt my _working_ initrd and hosed my system. To some extent I can work with dracut being buggy, but I get upset when it overwrites my working setup.
I think this is the bug I'm hitting:
https://bugzilla.opensuse.org/show_bug.cgi?id=959365
I'm just now building another initrd with the correct iscsi parameters, I'll report back.
Yes, now that I've supplied the iSCSI target userid+password, the initrd works and the system boots. Only took 2.5hours and a very irate customer tearing me a new one ... To summarise - to make dracut work, I had to comment out a section of code in /usr/lib/dracut/modules.d/95iscsi/module-setup.sh - lines 63-72. Then I needed to manually rebuild the initrd specifying --kernel-cmdline "rd.iscsi.username=user rd.iscsi.password=pwd" After having done that chroot'ed from the rescue system, I have to rebuild the initrd once more to get the multipath setup added. I'll leave that for the weekend, I've already scheduled some downtime. I could have kicked myself for forgetting bug#959365, but this is our only Leap421 production system (at least at the customer's insistence). -- Per Jessen, Zürich (14.1°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Tue, 2016-04-12 at 21:00 +0300, Andrei Borzenkov wrote:
12.04.2016 19:12, Per Jessen пишет:
I have just hosed a customer system when I applied the latest updates for leap421. Not only was the new initrd not built correctly (dracut bug), but the old one was also rebuilt, for no reason whatsoever. Now I'm left with a non-booting system.
Not happy. Some of this stuff just isn't being tested, at all.
It's not dracut. Packages that want initrd to be regenerated create flag; if they want to rebuild all initrds, they create /run/regenerate-initrd/all. So it is not dracut fault. E.g. udev does it to name a few.
What possible reason is there for an update to rebuild anything, kernel or initrd -wise? Updates should be installing a new kernel and initrd and leaving everything that is already installed completely alone. If udev or, god forbid, systemd need it rebuilding then they should be part of the kernel multi-versioning system. Updates shouldn't break existing working systems under any circumstances. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-04-12 20:46, Dave Howorth wrote:
On Tue, 2016-04-12 at 21:00 +0300, Andrei Borzenkov wrote:
What possible reason is there for an update to rebuild anything, kernel or initrd -wise? Updates should be installing a new kernel and initrd and leaving everything that is already installed completely alone. If udev or, god forbid, systemd need it rebuilding then they should be part of the kernel multi-versioning system. Updates shouldn't break existing working systems under any circumstances.
The initrd image does not only contain kernel modules, but also configuration files (like /etc/fstab) and binaries needed for initial fsck and mount, even in the case that /usr is on a different partition. The only way I can see to protect from these mishaps is to keep backup copies of the initrd, made automatically somehow - perhaps a daily cronjob, storing them in /var/adm/backup/, for instance (only if modified, not really a daily copy). -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
Carlos E. R. wrote:
On 2016-04-12 20:46, Dave Howorth wrote:
On Tue, 2016-04-12 at 21:00 +0300, Andrei Borzenkov wrote:
What possible reason is there for an update to rebuild anything, kernel or initrd -wise? Updates should be installing a new kernel and initrd and leaving everything that is already installed completely alone. If udev or, god forbid, systemd need it rebuilding then they should be part of the kernel multi-versioning system. Updates shouldn't break existing working systems under any circumstances.
The initrd image does not only contain kernel modules, but also configuration files (like /etc/fstab) and binaries needed for initial fsck and mount, even in the case that /usr is on a different partition.
The only way I can see to protect from these mishaps is to keep backup copies of the initrd, made automatically somehow - perhaps a daily cronjob, storing them in /var/adm/backup/, for instance (only if modified, not really a daily copy).
Another way would be for the upgrade process to create a backup before writing a new one. If I ran dracut myself and forgot to back up the initrds, I would have only myself to blame, but with an otherwise innocent "zypper patch", I have to remember it before I hit enter, just in case. Sometimes our otherwise solid openSUSE system framework feels more like some rickety bamboo scaffolding. -- Per Jessen, Zürich (13.8°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Per Jessen composed on 2016-04-12 21:10 (UTC+0200):
Another way would be for the upgrade process to create a backup before writing a new one. If I ran dracut myself and forgot to back up the initrds, I would have only myself to blame, but with an otherwise innocent "zypper patch", I have to remember it before I hit enter, just in case. Sometimes our otherwise solid openSUSE system framework feels more like some rickety bamboo scaffolding.
Sometimes things are more reliably done manually: # lsattr /boot/*d-4.1* -------------e-- i0nitrd-4.1.12-1-default1 -------------e-- i0nitrd-4.1.12-1-default2 -------------e-- i0nitrd-4.1.13-5-default1 -------------e-- i0nitrd-4.1.15-8-default1 ----i--------e-- initrd-4.1.12-1-default ----i--------e-- initrd-4.1.13-5-default ----i--------e-- initrd-4.1.15-8-default # ll /boot/*rd* -rw-r--r-- 1 root root 6257280 Dec 26 19:55 i0nitrd-4.1.12-1-default1 -rw-rw-r-- 1 root root 5972064 Dec 27 04:47 i0nitrd-4.1.12-1-default2 -rw-r--r-- 1 root root 5930624 Dec 27 14:32 i0nitrd-4.1.13-5-default1 -rw-r--r-- 1 root root 5986488 Feb 7 18:07 i0nitrd-4.1.15-8-default1 lrwxrwxrwx 1 root root 23 Feb 7 18:13 initrd -> initrd-4.1.15-8-default -rw-rw-r-- 1 root root 5972064 Dec 27 04:47 initrd-4.1.12-1-default -rw-r--r-- 1 root root 5930624 Dec 27 14:32 initrd-4.1.13-5-default -rw-r--r-- 1 root root 5986488 Feb 7 18:07 initrd-4.1.15-8-default lrwxrwxrwx 1 root root 23 Feb 7 18:13 initrd-421 -> initrd-4.1.15-8-default lrwxrwxrwx 1 root root 23 Dec 27 14:33 initrd-421p -> initrd-4.1.13-5-default lrwxrwxrwx 1 root root 23 Dec 27 00:58 initrd-421p2 -> initrd-4.1.12-1-default lrwxrwxrwx 1 root root 23 Feb 7 18:07 initrd-cur -> initrd-4.1.15-8-default lrwxrwxrwx 1 root root 23 Dec 27 14:32 initrd-prv -> initrd-4.1.13-5-default lrwxrwxrwx 1 root root 23 Dec 26 19:48 initrd-prv2 -> initrd-4.1.12-1-default -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-04-12 21:10, Per Jessen wrote:
Carlos E. R. wrote:
The only way I can see to protect from these mishaps is to keep backup copies of the initrd, made automatically somehow - perhaps a daily cronjob, storing them in /var/adm/backup/, for instance (only if modified, not really a daily copy).
Another way would be for the upgrade process to create a backup before writing a new one. If I ran dracut myself and forgot to back up the initrds, I would have only myself to blame, but with an otherwise innocent "zypper patch", I have to remember it before I hit enter, just in case.
I am not sure if this has been corrected, but I have seen dracut running several times during the same yast/zypper sesion. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
Carlos E. R. wrote:
On 2016-04-12 21:10, Per Jessen wrote:
Carlos E. R. wrote:
The only way I can see to protect from these mishaps is to keep backup copies of the initrd, made automatically somehow - perhaps a daily cronjob, storing them in /var/adm/backup/, for instance (only if modified, not really a daily copy).
Another way would be for the upgrade process to create a backup before writing a new one. If I ran dracut myself and forgot to back up the initrds, I would have only myself to blame, but with an otherwise innocent "zypper patch", I have to remember it before I hit enter, just in case.
I am not sure if this has been corrected, but I have seen dracut running several times during the same yast/zypper sesion.
I think that issue was fixed. /Per -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-04-13 21:13, Per Jessen wrote:
Carlos E. R. wrote:
I am not sure if this has been corrected, but I have seen dracut running several times during the same yast/zypper sesion.
I think that issue was fixed.
I thought I read it mentioned. Tumbleweed, I think. But Leap? - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlcOnlUACgkQja8UbcUWM1w90AD/fxaMEp1TfNV8qNps6ayGboqP epy4+y3VRMVZnZNlp7IA+wbyVWbnyUvbIUGYF3UcbrkB1C19NVNFxqkiWy+Ly+7o =859p -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Carlos E. R. composed on 2016-04-13 21:30 (UTC+0200):
Per Jessen wrote:
Carlos E. R. wrote:
I am not sure if this has been corrected, but I have seen dracut running several times during the same yast/zypper sesion.
I think that issue was fixed.
I thought I read it mentioned. Tumbleweed, I think. But Leap?
https://bugzilla.opensuse.org/show_bug.cgi?id=786318 IIRC, 13.2 was first "release" with the fix. TW fix was 22 months ago, well over a year before 42.1 release. I'm not sure how comprehensive the fix was initially though. -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Felix Miata wrote:
Carlos E. R. composed on 2016-04-13 21:30 (UTC+0200):
Per Jessen wrote:
Carlos E. R. wrote:
I am not sure if this has been corrected, but I have seen dracut running several times during the same yast/zypper sesion.
I think that issue was fixed.
I thought I read it mentioned. Tumbleweed, I think. But Leap?
https://bugzilla.opensuse.org/show_bug.cgi?id=786318
IIRC, 13.2 was first "release" with the fix. TW fix was 22 months ago, well over a year before 42.1 release. I'm not sure how comprehensive the fix was initially though.
It's a separate issue too, not directly related to $SUBJ. /Per -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-04-13 21:54, Per Jessen wrote:
IIRC, 13.2 was first "release" with the fix. TW fix was 22 months ago, well over a year before 42.1 release. I'm not sure how comprehensive the fix was initially though.
It's a separate issue too, not directly related to $SUBJ.
It is to the trick of having a backup initrd of the previous version saved, so that you can recover from failure :-) Having the dracut run create the backup itself would be the best, unless it runs several times per update session. In that case, better create a backup with daily cron, same as the rpm database is done. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlcOpkoACgkQja8UbcUWM1w43wD+Nz0jJk+BOeaKyPQnBbe0gwwx X+kD/gquvVTA93cNLcgBAIfpRGQaiA+0Frz7Ud7+Vna1uAOgsK8sp95hqd4XXGYV =lZi6 -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Carlos E. R. wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 2016-04-13 21:54, Per Jessen wrote:
IIRC, 13.2 was first "release" with the fix. TW fix was 22 months ago, well over a year before 42.1 release. I'm not sure how comprehensive the fix was initially though.
It's a separate issue too, not directly related to $SUBJ.
It is to the trick of having a backup initrd of the previous version saved, so that you can recover from failure :-)
Hmmm. If I have a backup, it doesn't matter how often the initrd is rebuilt. If I don't have a backup, it also doesn't matter how often the initrd is rebuilt.
Having the dracut run create the backup itself would be the best, unless it runs several times per update session. In that case, better create a backup with daily cron, same as the rpm database is done.
There is little point in discussing solutions unless we're going to implement them ourselves. Personally, I don't think dracut should be involved in making any backups, that is a job for zypper or the update procedure when it invokes dracut automagically. /Per -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-04-13 22:29, Per Jessen wrote:
Carlos E. R. wrote:
It is to the trick of having a backup initrd of the previous version saved, so that you can recover from failure :-)
Hmmm. If I have a backup, it doesn't matter how often the initrd is rebuilt. If I don't have a backup, it also doesn't matter how often the initrd is rebuilt.
Having the dracut run create the backup itself would be the best, unless it runs several times per update session. In that case, better create a backup with daily cron, same as the rpm database is done.
There is little point in discussing solutions unless we're going to implement them ourselves. Personally, I don't think dracut should be involved in making any backups, that is a job for zypper or the update procedure when it invokes dracut automagically.
Yes, of course we have to implement it ourselves. If something official is done, it will be for the next release. There are two possibilities. The one I prefer is having it done by the daily cron job. For this one, it does not matter if zypper rebuilds initrd a dozen times. I might do it myself. It would be easy to package, for one that knows how to package. I don't, really. The other solution is insert the backup job in the dracut script. This is neat, but one needs that it is only called once per zypper/yast run, so this "detail" is important. If we keep, say, 3 copies, and zypper runs the script 3 times in a row, the backup would be useless. It would not work for me, having 13.1. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlcOxlMACgkQja8UbcUWM1xlEwEAh65biwpO/ktBjnQsUTHsEzss e7J1lIBHKG0prM+tKpUA/RSwfR2VbtMGCrbnQdiJIZudE+wJyL7gNPiPNhQ7tm9W =mMhZ -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Carlos E. R. wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 2016-04-13 22:29, Per Jessen wrote:
Carlos E. R. wrote:
It is to the trick of having a backup initrd of the previous version saved, so that you can recover from failure :-)
Hmmm. If I have a backup, it doesn't matter how often the initrd is rebuilt. If I don't have a backup, it also doesn't matter how often the initrd is rebuilt.
Having the dracut run create the backup itself would be the best, unless it runs several times per update session. In that case, better create a backup with daily cron, same as the rpm database is done.
There is little point in discussing solutions unless we're going to implement them ourselves. Personally, I don't think dracut should be involved in making any backups, that is a job for zypper or the update procedure when it invokes dracut automagically.
Yes, of course we have to implement it ourselves. If something official is done, it will be for the next release.
Oh I see. Well, it would be easy to put out as a patch to zypper or dracut. Always take a copy when overwriting, leave managing the backup copies to a daily cron job.
There are two possibilities. The one I prefer is having it done by the daily cron job. For this one, it does not matter if zypper rebuilds initrd a dozen times.
So always keep a daily copy named "initrd-a.b.c-whatever.yesterday" ? (copied when needed of course). Yes, that could work. Still wouldn't prevent a system becoming unbootable, but it would make it easier to fix.
The other solution is insert the backup job in the dracut script. This is neat, but one needs that it is only called once per zypper/yast run, so this "detail" is important. If we keep, say, 3 copies, and zypper runs the script 3 times in a row, the backup would be useless.
If dracut were to create a backup on every overwrite, it could just add a sequential number to each backup. -- Per Jessen, Zürich (8.6°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-04-14 08:29, Per Jessen wrote:
Carlos E. R. wrote:
So always keep a daily copy named "initrd-a.b.c-whatever.yesterday" ? (copied when needed of course). Yes, that could work. Still wouldn't prevent a system becoming unbootable, but it would make it easier to fix.
Yes, that's the idea. It would be possible to add entries in grub, or simply edit in grub the line, if the backups are kept in /boot. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2016-04-13 21:40, Felix Miata wrote:
Carlos E. R. composed on 2016-04-13 21:30 (UTC+0200):
Per Jessen wrote:
Carlos E. R. wrote:
I am not sure if this has been corrected, but I have seen dracut running several times during the same yast/zypper sesion.
I think that issue was fixed.
I thought I read it mentioned. Tumbleweed, I think. But Leap?
https://bugzilla.opensuse.org/show_bug.cgi?id=786318
IIRC, 13.2 was first "release" with the fix. TW fix was 22 months ago, well over a year before 42.1 release. I'm not sure how comprehensive the fix was initially though.
I don't know. I have most versions installed on virtual machines, but I don't remember which have this particular problem and which doesn't (I never use dup on them). 13.1 does not have dracut, but I think it does build initrd several times, on some updates. - -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" (Minas Tirith)) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlcOpMAACgkQja8UbcUWM1wU7AD+J8f7aYWQ6SYFTBXafXQDcyWx GybIAjB7nxektnR4V3cA/iC677EOFW9Rbihy9tAhwQIjmwH+hKvxlwNHHDxZ69HL =3rVl -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Carlos E. R. composed on 2016-04-13 21:57 (UTC+0200):
https://bugzilla.opensuse.org/show_bug.cgi?id=786318 ... 13.1 does not have dracut, but I think it does build initrd several times, on some updates.
Look when I filed it, right about the time 12.3 got cut off of Factory just before release. It was definitely happening in pre-releases of 13.1, same as in 12.3. https://bugzilla.opensuse.org/show_bug.cgi?id=786318#c6 indicates 13.2 as the most likely fix target of a private FATE. -- "The wise are known for their understanding, and pleasant words are persuasive." Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
13.04.2016 22:13, Per Jessen пишет:
Carlos E. R. wrote:
On 2016-04-12 21:10, Per Jessen wrote:
Carlos E. R. wrote:
The only way I can see to protect from these mishaps is to keep backup copies of the initrd, made automatically somehow - perhaps a daily cronjob, storing them in /var/adm/backup/, for instance (only if modified, not really a daily copy).
Another way would be for the upgrade process to create a backup before writing a new one. If I ran dracut myself and forgot to back up the initrds, I would have only myself to blame, but with an otherwise innocent "zypper patch", I have to remember it before I hit enter, just in case.
I am not sure if this has been corrected, but I have seen dracut running several times during the same yast/zypper sesion.
I think that issue was fixed.
This requires each package that rebuilds initrd to be modified so the question is "for which package"? Or do you mean all packages were adjusted? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Andrei Borzenkov wrote:
13.04.2016 22:13, Per Jessen пишет:
Carlos E. R. wrote:
On 2016-04-12 21:10, Per Jessen wrote:
Carlos E. R. wrote:
The only way I can see to protect from these mishaps is to keep backup copies of the initrd, made automatically somehow - perhaps a daily cronjob, storing them in /var/adm/backup/, for instance (only if modified, not really a daily copy).
Another way would be for the upgrade process to create a backup before writing a new one. If I ran dracut myself and forgot to back up the initrds, I would have only myself to blame, but with an otherwise innocent "zypper patch", I have to remember it before I hit enter, just in case.
I am not sure if this has been corrected, but I have seen dracut running several times during the same yast/zypper sesion.
I think that issue was fixed.
This requires each package that rebuilds initrd to be modified so the question is "for which package"? Or do you mean all packages were adjusted?
Andrei, to be honest, I'm not sure, but I thought packages that want/need/suggest rebuilding of the initrds were changed to do so by flagging it, which is then examined at the end of the run. -- Per Jessen, Zürich (8.7°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-04-14 08:16, Per Jessen wrote:
Andrei Borzenkov wrote:
13.04.2016 22:13, Per Jessen пишет:
Carlos E. R. wrote:
I am not sure if this has been corrected, but I have seen dracut running several times during the same yast/zypper sesion.
I think that issue was fixed.
This requires each package that rebuilds initrd to be modified so the question is "for which package"? Or do you mean all packages were adjusted?
Andrei, to be honest, I'm not sure, but I thought packages that want/need/suggest rebuilding of the initrds were changed to do so by flagging it, which is then examined at the end of the run.
I think that they don't run mkinitrd but flag that it needs to run, and it is run at the end, I suppose by zypper. But this is hearsay and guessing on my part. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
On 2016-04-12 18:12, Per Jessen wrote:
I have just hosed a customer system when I applied the latest updates for leap421. Not only was the new initrd not built correctly (dracut bug), but the old one was also rebuilt, for no reason whatsoever. Now I'm left with a non-booting system.
Well, not that it helps, but the pre-dracut method (the original mkinitrd) did the same: it built all kernels ram disks, instead of just the kernel that has being updated. However, when it wasn't the kernel that got updated/modified, it has to build all. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
Carlos E. R. wrote:
On 2016-04-12 18:12, Per Jessen wrote:
I have just hosed a customer system when I applied the latest updates for leap421. Not only was the new initrd not built correctly (dracut bug), but the old one was also rebuilt, for no reason whatsoever. Now I'm left with a non-booting system.
Well, not that it helps, but the pre-dracut method (the original mkinitrd) did the same: it built all kernels ram disks, instead of just the kernel that has being updated.
Yes, you're right - the difference is that mkinitrd was tried and tested. Thoroughly.
However, when it wasn't the kernel that got updated/modified, it has to build all.
I'm a little tired and more than a little upset, not a good combo, but AFAICS, creating a non-working initrd and also overwriting the working initrd completely screwed up the system. Honestly, when dracut throws syntax errors: *** Including module: iscsi *** cat: /sys/devices/platform/host2/flashnode_sess-*/is_boot_target: No such file or directory /usr/lib/dracut/modules.d/95iscsi/module-setup.sh: line 65: [: -eq: unary operator expected cat: /sys/devices/platform/host2/flashnode_sess-*/is_boot_target: No such file or directory /usr/lib/dracut/modules.d/95iscsi/module-setup.sh: line 65: [: -eq: unary operator expected cat: /sys/devices/platform/host2/flashnode_sess-*/is_boot_target: No such file or directory /usr/lib/dracut/modules.d/95iscsi/module-setup.sh: line 65: [: -eq: unary operator expected something was not tested very much, was it? -- Per Jessen, Zürich (13.8°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-04-12 21:01, Per Jessen wrote:
I'm a little tired and more than a little upset, not a good combo, but AFAICS, creating a non-working initrd and also overwriting the working initrd completely screwed up the system.
Honestly, when dracut throws syntax errors:
*** Including module: iscsi *** cat: /sys/devices/platform/host2/flashnode_sess-*/is_boot_target: No such file or directory /usr/lib/dracut/modules.d/95iscsi/module-setup.sh: line 65: [: -eq: unary operator expected cat: /sys/devices/platform/host2/flashnode_sess-*/is_boot_target: No such file or directory /usr/lib/dracut/modules.d/95iscsi/module-setup.sh: line 65: [: -eq: unary operator expected cat: /sys/devices/platform/host2/flashnode_sess-*/is_boot_target: No such file or directory /usr/lib/dracut/modules.d/95iscsi/module-setup.sh: line 65: [: -eq: unary operator expected
something was not tested very much, was it?
Unfortunately, dracut spews tons of texts. Many errors or warnings that have no consequence. It is difficult to see the grain with so much straw. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
Per Jessen wrote:
Honestly, when dracut throws syntax errors:
*** Including module: iscsi *** cat: /sys/devices/platform/host2/flashnode_sess-*/is_boot_target: No such file or directory /usr/lib/dracut/modules.d/95iscsi/module-setup.sh: line 65: [: -eq: unary operator expected cat: /sys/devices/platform/host2/flashnode_sess-*/is_boot_target: No such file or directory /usr/lib/dracut/modules.d/95iscsi/module-setup.sh: line 65: [: -eq: unary operator expected cat: /sys/devices/platform/host2/flashnode_sess-*/is_boot_target: No such file or directory /usr/lib/dracut/modules.d/95iscsi/module-setup.sh: line 65: [: -eq: unary operator expected
something was not tested very much, was it?
See https://bugzilla.opensuse.org/show_bug.cgi?id=975267 -- Per Jessen, Zürich (10.9°C) http://www.dns24.ch/ - your free DNS host, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (8)
-
Andrei Borzenkov -
Carlos E. R. -
Dave Howorth -
Felix Miata -
jdd -
Per Jessen -
sdm -
Yamaban