Start using transaction file triggers for ldconfig?
Hi list, now that boo#1041742 [1] appears to be resolved in Tumbleweed, is there anything stopping us from adding transaction file triggers into glibc to run ldconfig automatically? Then we could finally get rid of all the %post -n libnfoo%{libversion} -p /sbin/ldconfig %postun -n libfoo%{libversion} -p /sbin/ldconfig scripts in _every_ single spec file out there. Cheers, Dan Footnotes: [1] https://bugzilla.suse.com/show_bug.cgi?id=1041742 -- Dan Čermák <dcermak@suse.com> Software Engineer Development tools SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
Dan Čermák wrote:
I'm all for removing as many scriptlets as possible. So yes, please :-) The direct equivalent of those ldconfig calls would be normal file triggers though rather than the so far problematic transaction one. Depends on whether or not we want to update ld.so.cache after each package. The symlinks ldconfig also creates must be packaged anyway. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg)
Отправлено с iPhone
Corner case is when package B requires libraries from package A and expects them to be present during installation (e.g. due to scripts) and both are installed in the same transaction. Moving ldconfig to post trans may break it. Not sure if there is real life example (this needs libraries in non-standard places).
The direct equivalent of those ldconfig calls would be normal file triggers though rather than the so far problematic transaction one.
This has advantage of consolidating scripts in single trigger instance; trigger will still run for every package.
On Mon, 2021-08-30 at 09:59 +0200, Dan Čermák wrote:
I've read through this bug, and I have to say I fail to grok it. When exactly does a %transfiletrigger get executed if libzypp installs multiple packages at the same time? https://bugzilla.suse.com/show_bug.cgi?id=1041742#c19 makes it sound as if the %transfiletrigger was executed by rpm directly after the installation of a package, i.e. before libzypp calls rpm to install other packages: libzypp rpm --noposttrans install A %post of A %filetriggerin triggered by A %transfiletriggerin triggered by A (*) rpm --noposttrans install B %post of B -> %posttrans of A, B, ... (#) where (*), according to the BZ comments, would only be executed by rpm 4.16 and higher. But that would still be very different from what rpm itself does according to our late discussion on packaging (https://lists.opensuse.org/archives/list/packaging@lists.opensuse.org/thread...) where the %transfiletriggerin would be called after (#). Or am I misunderstanding? Martin
Footnotes: [1] https://bugzilla.suse.com/show_bug.cgi?id=1041742
On Mon, Aug 30, 2021 at 10:42 AM Andrei Borzenkov <arvidjaar@gmail.com> wrote:
The fundamental flaw of Zypper here is that it *doesn't* let RPM run the scriptlets in the first place. It also doesn't allow RPM to do the transaction itself. This is why Zypper struggles with some of the more advanced features of RPM: RPM expects that these things are run as part of one unified transaction with RPM *itself* ordering and executing things. Zypper collects the packages, installs the RPMs with rpm -Uvh --noscripts, and then turns around and scans for scripts and manually executes them. There was no effort to introduce delayed script execution to RPM, and no effort to properly integrate the higher level and lower level package managers properly. This is actually a fundamental difference between Zypper and *all* other RPM package managers. In particular, DNF executes transactions by collecting up the RPMs and passing it to librpm to apply the transaction. It then listens to a callback from librpm so that it can print out progress to the user. -- 真実はいつも一つ！/ Always, there's only one truth!
On Mon, 2021-08-30 at 10:51 -0400, Neal Gompa wrote:
Understood. I'm sure there are good (probably historic?) reasons why zypper does this the way it does. My question is what this means for %transfiletriggerin with zypper/libzypp. Can we do it at all, and if yes, how? What will be the semantic differences to plain rpm and/or dnf? Regards Martin
Martin Wilck <martin.wilck@suse.com> writes:
Yes we can do it on Tumbleweed and maybe on the next Leap if the changes to rpm and libzypp get backported.
What will be the semantic differences to plain rpm and/or dnf?
I guess that the order of the scripts will be different, but otherwise it should just work (yes I know, famous last words…). Cheers, Dan -- Dan Čermák <dcermak@suse.com> Software Engineer Development tools SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
Martin Wilck <martin.wilck@suse.com> writes:
According to https://bugzilla.opensuse.org/show_bug.cgi?id=1041742#c26 it's been added to rpm with 4.16.1.3, which looks like revision 575 in Base:system/rpm [1] The full change updating rpm to 4.16.1.3 is sr#884038 [2]. The Michael mentions in the bug that libzypp can use this as off 17.26.1, but unfortunately I cannot find this tag on github.
Unfortunately no idea. @Michael @Benjami? Cheers, Dan Footnotes: [1] https://build.opensuse.org/package/rdiff/Base:System/rpm?linkrev=base&rev=57... [2] https://build.opensuse.org/request/show/884038 -- Dan Čermák <dcermak@suse.com> Software Engineer Development tools SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
On Tuesday 31 August 2021 08:54:30 Dan Čermák wrote:
No, this will be available in libzypp-17.29 (iff used with rpm-16). (ETA by end of next week)
- libzypp-17.29 + rpm-16 will execte the %transfiletrigger at the end of the zypp-transaction, after all packages are installed. The scripts will be executed by rpm, so there shouldn't be too much difference... - libzypp-17.28.2 + ZYPP_SINGLE_RPMTRANS=1 zypper ... We're about to write the announcement for this. This (experimental) mode will use a single rpm transaction IFF download-in-advance is not disabled. The behavior should be similar to 'libzypp-17.29 + rpm-16', but for big zypper transactions (dup/migration) there might be a benefit in speed. -- cu, Michael Andres +------------------------------------------------------------------+ Key fingerprint = 2DFA 5D73 18B1 E7EF A862 27AC 3FB8 9E3A 27C6 B0E4 +------------------------------------------------------------------+ Michael Andres, E&I, ma@suse.com, Phone: ++49 (0)911 - 74 053-0 +------------------------------------------------------------------+ SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg Germany, (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer +------------------------------------------------------------------+
On Tue, 2021-08-31 at 12:58 +0200, Michael Andres wrote:
Well but I guess it'll take some time until we'll see this in Factory, let alone in Leap. Is there any way to achieve something similar with the current code in Leap? Use case: I want to be able to postpone initrd rebuilding in weak- modules2. But the kernel packages currently don't have %posttrans scripts, therefore I can't be sure that weak-modules2 will be run in %posttrans. I could add them to the kernel packages, but it'd be much more elegant to be able to do this without modifying the kernel packaging. Martin
On Tuesday 31 August 2021 21:24:54 Martin Wilck wrote:
Keep in mind that it needs 1 extra install to get the plugin installed. Be carefull if you want to execute long running jobs. You must respond back to libzypp within time (30sec.), otherwise the plugin gets discarded. A plugin is not allowed to eternally delay the commit (was a requirement at the time we introduced them). You can continue in the background, but zypp will end. 'zypper up && reboot' might be problematic then. Also reporting/error-reporting is limited (stderr will be logged while connected to libzypp, but no access to the users screen). -- cu, Michael Andres +------------------------------------------------------------------+ Key fingerprint = 2DFA 5D73 18B1 E7EF A862 27AC 3FB8 9E3A 27C6 B0E4 +------------------------------------------------------------------+ Michael Andres, E&I, ma@suse.com, Phone: ++49 (0)911 - 74 053-0 +------------------------------------------------------------------+ SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg Germany, (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer +------------------------------------------------------------------+
On Wed, 2021-09-01 at 10:11 +0200, Michael Andres wrote:
Yeah, I noted these. But the biggest problem I encountered is that my plugin would be called in parallel with the snapper plugin (in the COMMITEND hook), causing the latter to create the "post" snapshot before my work was finished, which is of course very unfortunate. So far I couldn't figure out how to avoid that. Martin
On Wednesday 01 September 2021 15:11:32 Martin Wilck wrote:
Let a package do the job, not a plugin. -- cu, Michael Andres +------------------------------------------------------------------+ Key fingerprint = 2DFA 5D73 18B1 E7EF A862 27AC 3FB8 9E3A 27C6 B0E4 +------------------------------------------------------------------+ Michael Andres, E&I, ma@suse.com, Phone: ++49 (0)911 - 74 053-0 +------------------------------------------------------------------+ SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg Germany, (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer +------------------------------------------------------------------+
On Tuesday 31 August 2021 20:41:09 Martin Wilck wrote:
- By upgrading to rpm-16 or if it's possible to backport the rpm code. The rpm-fix allows us to collect the triggers which would have been executed and to ask rprm at the end to execute those triggers only. - The poor man's way: Packages that want to trigger a delayed action, touch a flag file in their %post _and_ have a %posttrans that { performs the action; removes the flag file; } IFF the flag file exists (alt.: if the target is older than the flag file). So the 1st %posttrans executes and the others remain silent. -- cu, Michael Andres +------------------------------------------------------------------+ Key fingerprint = 2DFA 5D73 18B1 E7EF A862 27AC 3FB8 9E3A 27C6 B0E4 +------------------------------------------------------------------+ Michael Andres, E&I, ma@suse.com, Phone: ++49 (0)911 - 74 053-0 +------------------------------------------------------------------+ SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg Germany, (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer +------------------------------------------------------------------+
On Wed, Sep 1, 2021 at 10:30 AM Andrei Borzenkov <arvidjaar@gmail.com> wrote:
"rpm-16" is actually RPM 4.16. They dropped the "4." assuming everyone else knew what that meant. "rpm-fix" is the SUSE-specific patch to add options to inhibit and dump scripts so Zypper can run them manually: https://code.opensuse.org/package/rpm/blob/ef05feae29a8c5d9fd06b4afa072fbb41... -- 真実はいつも一つ！/ Always, there's only one truth!
On Monday 2021-08-30 18:12, Andrei Borzenkov wrote:
You can't have it both ways. # strace -fe execve /usr/bin/zypper up 2>&1 | grep execve 2>&1 | grep -v ENOENT [2 packages scheduled for installation: kernel-devel, kernel-macros] 27223 execve("/usr/bin/rpm", ["rpm", "--root", "/", "--dbpath", "/usr/lib/sysimage/rpm", "-U", "--percent", "--noglob", "--force", "--nodeps", "--", "/var/cache/zypp/packages/jeng/no"...], 0x5558798161e0 /* 87 vars */) = 0 27224 execve("/usr/bin/rpm", ["rpm", "--root", "/", "--dbpath", "/usr/lib/sysimage/rpm", "-i", "--percent", "--noglob", "--force", "--nodeps", "--", "/var/cache/zypp/packages/jeng/no"...], 0x5558798161e0 /* 87 vars */) = 0 27231 execve("/bin/sh", ["/bin/sh", "/var/tmp/rpm-tmp.GaR2LZ", "8"], 0x2241de0 /* 89 vars */) = 0 27235 execve("/usr/bin/rpmdb2solv", ["rpmdb2solv", "-r", "/", "-D", "/usr/lib/sysimage/rpm", "-X", "-p", "/etc/products.d", "/var/cache/zypp/solv/@System/sol"..., "-o", "/var/cache/zypp/solv/@System/sol"...], 0x55587fd75440 /* 86 vars */) = 0
On Mon, Aug 30, 2021 at 09:59:59AM +0200, Dan Čermák wrote:
I'm not sure that ldconfig is something for a transaction file trigger. I depends on why ldconfig needs to be run at all. If it is needed to make the package run correctly, it can't be done in a transaction trigger as other packages might depend on the package working in their post scriptlets. I think a normal file trigger is more correct, even if this means that ldconfig will be run quite often. Cheers, Michael. -- Michael Schroeder SUSE Software Solutions Germany GmbH mls@suse.de GF: Felix Imendoerffer HRB 36809, AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);}
On Tuesday 2021-08-31 10:24, Michael Schroeder wrote:
Perhaps https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... gives an answer to that. It also concludes:
But this will run at the end of the transaction, which is too late for other scriptlets
but also
Michael Schroeder <mls@suse.de> writes:
I have asked on the Fedora mailinglist about this and it looks like that Fedora is circumventing this by shipping symlinks to the shared libraries: ❯ ll /usr/lib64/libcrypto.so* lrwxrwxrwx. 1 root root 19 Aug 26 17:56 /usr/lib64/libcrypto.so -> libcrypto.so.1.1.1l lrwxrwxrwx. 1 root root 19 Aug 26 17:56 /usr/lib64/libcrypto.so.1.1 -> libcrypto.so.1.1.1l -rwxr-xr-x. 1 root root 3.0M Aug 26 17:56 /usr/lib64/libcrypto.so.1.1.1l Then running ldconfig is just a pure optimization.[1] I'd have to check if we already have the necessary post build scripts in place to setup all the necessary symlinks, but besides that, I don't see why we couldn't move this to %transfiletriggerin/-un once zypper can handle these. Cheers, Dan Footnotes: [1] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... -- Dan Čermák <dcermak@suse.com> Software Engineer Development tools SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nuremberg Germany (HRB 36809, AG Nürnberg) Managing Director: Felix Imendörffer
On 03.09.2021 18:05, Dan Čermák wrote:
What about libraries installed in non-standard location? bor@tw:~> cat /etc/ld.so.conf.d/graphviz.conf /usr/lib64/graphviz /usr/lib64/graphviz/sharp /usr/lib64/graphviz/java /usr/lib64/graphviz/perl /usr/lib64/graphviz/php /usr/lib64/graphviz/ocaml /usr/lib64/graphviz/python /usr/lib64/graphviz/lua /usr/lib64/graphviz/tcl /usr/lib64/graphviz/guile /usr/lib64/graphviz/ruby bor@tw:~>
On Friday 2021-09-03 17:05, Dan Čermák wrote:
openSUSE ships them too. Not so much for reasons involving ldconfig, but simply because they get created on `make install` by many, if not all, build systems in use. In fact, if there was, hypothetically, a build system that would only install libcrypto.so.1.1.1l, then, because %post-p-/sbin/ldconfig still exists in most .spec files, the symlink will be created during post-build-check, and because it's not owned by the rpm, will be flagged by p-b-c. As there are no build failures presently to that end, it's safe to say all our packages make the proper symlinks in %install already.
Then running ldconfig is just a pure optimization.[1]
For openSUSE, it is. The only remaining reason to use ldconfig is because of ld.so.conf.d paths. For example, glibc.i686 adds libc.so.6 to /lib/i686 or so, and that may have implications with regard to ld.so.cache. Like, when you remove glibc.i686, then /lib/i686/libc.so.6 is gone, but ld.so.cache may still say that libc.so.6 is to be found in (and only in) /lib/i686.
participants (8)
-
Andrei Borzenkov -
Dan Čermák -
Jan Engelhardt -
Ludwig Nussel -
Martin Wilck -
Michael Andres -
Michael Schroeder -
Neal Gompa