Migrate from update-alternatives to libalternatives
Hello folks, As some of you may know, there are some limitations regarding update-alternatives that prevent it from working perfectly when using transactional updates. Basically, /var isn't part of the snapshot's data. This means that all files created/manipulated during package installation in /var/lib/alternatives won't be taken into consideration when snapshots are rolled back. More information is available here: https://kubic.opensuse.org/documentation/transactional-update-guide/tu-setup... This fact could lead to broken symlinks: Suppose I install "go1.20"(snapshot A) and "go1.21" (snapshot B). I then boot from snapshot A, but "/var/lib/alternatives/go" still tells "update-alternatives" that "go1.21" would be a valid choice, since this information lives outside of the snapshot's directories, in "/var". If "update-alternatives" were to accept that choice, then "/etc/alternatives/go" would become a broken link, our last line of defence here being how carefully maintained "update-alternatives" is. So that's not a system-level design solution, but rather very application-specific. What we're planning: Replace update-alternatives with libalternatives: https://github.com/openSUSE/libalternatives This change should be done gradually on Factory, and ideally it would require no manual intervention from the users. We plan on releasing this change in batches: i.e.: all Go versions we provide on Factory would be migrated to libalternatives in a single-pass. Then another set of closely-related packages, such as the OpenJDK. Then another, so on and so forth. What we need: The goal of this thread is to establish a line of discussion around this change, which should be a gradual one on Factory. All opinions, comments, ideas and criticism are valid, and will be taken into consideration. Status: There are currently 14909 spec files on openSUSE:Factory. 964 (6.46%) of these depend on update-alternatives. Currently 52 packages use libalternatives, such as vim and nodejs-common. Basically, the cores of both MicroOS and SLE Micro are already using libalternatives. Things that seem to be clear in this particular context: 1 - libalternatives should work really well with transactional updates: As the state of alternatives kept by libalternatives is maintained in /usr/share/libalternatives/<binary>/<pref>.conf, this means that packages only need to provide a single file to provide their alternative. This file will be part of the package's regular files (declared in %files), and no state is modified by any hooks (preun, postun, prerm, postrm) during the updates. Rolling back a snapshot will revert back to a state where that alternative didn't exist (as it wasn't installed). 2 - libalternatives does not manipulate symlinks: /usr/bin/alts (the main binary for libalternatives) executes binaries directly (by calling exec()). The actually binary (for example, /usr/bin/vi) is a symlink to /usr/bin/alts, which then reads its configuration files and determines which binary to execute, based on the highest priority. 3 - libalternatives supports user overrides: non-root users can override the alternatives by creating their own local preference file in $HOME/.config/libalternatives.conf. This may be done by invoking "alts" directly, as it serves both purposes. 4 - Even if it doesn't manipulate symlinks, groups and manpages are still supported: libalternatives has a "group" directive for manipulating groups of binaries together, and it has some verifications to ensure the binaries in these groups are indeed declared in tandem with one another, essentially managing them as a collection. It also has a "man" directive for man pages, so even if it's not doing symlink manipulation, look up man pages still works. In fact, our "man" package was patched to work with libalternatives support: https://build.opensuse.org/package/view_file/Base:System/man/man-db-2.9.4-al... 5 - No RPM macros are required: This is really cool. We're basically just adding a file to %files, to use libalternatives. So no RPM macros are even required here. But we could create an RPM macro that automatically writes the preference file, so we could easily manipulate its contents inside the RPM spec. 6 - How libalternatives can actually be used So there are actually at least TWO ways to use libalternatives: we either create a symlink to /usr/bin/alts, or use libalternatives as a shared object and write a tiny wrapper program, such as this one: https://build.opensuse.org/package/view_file/devel:languages:nodejs/nodejs-c... The advantage of the first method is that we don't need a special – albeit short – C program to act as a dispatcher for libalternatives's magic sauce. The advantage of the second method is that we don't need packages to depend on the "alts" package (which contains /usr/bin/alts), but only on "libalternatives1" instead (which provides /usr/lib64/libalternatives.so.1). I appreciate if you can provide any insights here, they're more than welcome. Thank you.
On 9/29/23 15:11, Martin Schreiner via openSUSE Factory wrote:
Hello folks,
As some of you may know, there are some limitations regarding update-alternatives that prevent it from working perfectly when using transactional updates. Basically, /var isn't part of the snapshot's data. This means that all files created/manipulated during package installation in /var/lib/alternatives won't be taken into consideration when snapshots are rolled back. More information is available here: https://kubic.opensuse.org/documentation/transactional-update-guide/tu-setup...
Does this change mean that TW / Factory may be switching to using transactional-update sometime in the future ? I don't think it does but wanted to clarify. -- Regards, Joe
On Fri, Sep 29, Joe Salmeri wrote:
On 9/29/23 15:11, Martin Schreiner via openSUSE Factory wrote:
Hello folks,
As some of you may know, there are some limitations regarding update-alternatives that prevent it from working perfectly when using transactional updates. Basically, /var isn't part of the snapshot's data. This means that all files created/manipulated during package installation in /var/lib/alternatives won't be taken into consideration when snapshots are rolled back. More information is available here: https://kubic.opensuse.org/documentation/transactional-update-guide/tu-setup...
Does this change mean that TW / Factory may be switching to using transactional-update sometime in the future ? I don't think it does but wanted to clarify.
Factory did switch to transactional-update already many years ago if you haven't noticed, the Factory variants using it exclusively are called MicroOS, Aeon and Kalpa. And for the other openSUSE variants we have the "transactional server" role as install time option. So if you are afraid that TW may switch to transactional update, you are far too late, it did already years ago. Beside that update-alternatives is not only a problem for transactional-update, but also hermetic-usr and all kinds of image based updates. So getting the mis-design of update-alternatives solved is in interested of many people. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
On 10/1/23 14:35, Thorsten Kukuk wrote:
On Fri, Sep 29, Joe Salmeri wrote:
Does this change mean that TW / Factory may be switching to using transactional-update sometime in the future ? I don't think it does but wanted to clarify. Factory did switch to transactional-update already many years ago if you haven't noticed, the Factory variants using it exclusively are called MicroOS, Aeon and Kalpa. And for the other openSUSE variants we have the "transactional server" role as install time option. So if you are afraid that TW may switch to transactional update, you are far too late, it did already years ago.
Beside that update-alternatives is not only a problem for transactional-update, but also hermetic-usr and all kinds of image based updates.
So getting the mis-design of update-alternatives solved is in interested of many people.
Thorsten
I am aware of transactional-update being used with MicroOS, Aeon, Kalpa as well as the transactional server role in TW, however, last I checked ( its been a month or two since I spun up a new TW installation ) the standard TW install using KDE or GNome at least does NOT use transactional updates. So to better phrase my question, are there plans for TW KDE or GNome new installastions to be switching to the transactional-update mode ? If YES, then will there be a migration process for older TW installations that were not using transactional-update or will a new install be required ? -- Regards, Joe
On Sun, Oct 01, Joe Salmeri wrote:
I am aware of transactional-update being used with MicroOS, Aeon, Kalpa as well as the transactional server role in TW, however, last I checked ( its been a month or two since I spun up a new TW installation ) the standard TW install using KDE or GNome at least does NOT use transactional updates.
So to better phrase my question, are there plans for TW KDE or GNome new installastions to be switching to the transactional-update mode ?
If YES, then will there be a migration process for older TW installations that were not using transactional-update or will a new install be required ?
You have the choice if you want to have a transactional or a non-transactional system, why should we remove that and force people to use a transactional system? Today there are no such plans as they wouldn't make any sense. But in any case: if we would do that, there will be no migration process, as this would require a repartitioning of most users harddisks. -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
On 01/10/23 20:35, Thorsten Kukuk wrote:
So if you are afraid that TW may switch to transactional update, you are far too late, it did already years ago.
I think this statement is a bit misleading. Tumbleweed itself does not need to be transactional, nor does it state that a transactional system would be the default - Nor the installer nor on the Tumbleweed page in the wiki [1] would say anything like that. So, while it's certainly true that you can have TW as transactional server/desktop, the statement that Tumbleweed as distribution switched to being a transactional system might be a bit exaggerating, right? In short: MicroOS as of now is transactional-only, but in Tumbleweed you can still choose between a transactional and non-transactional system and there are no plans to kill the latter, or am I wrong? [1] https://en.opensuse.org/Portal:Tumbleweed Best, phoenix
On 29.09.2023 22:11, Martin Schreiner via openSUSE Factory wrote:
Hello folks,
As some of you may know, there are some limitations regarding update-alternatives that prevent it from working perfectly when using transactional updates. Basically, /var isn't part of the snapshot's data. This means that all files created/manipulated during package installation in /var/lib/alternatives won't be taken into consideration when snapshots are rolled back.
What prevents moving this information into / so it is included in snapshot? It was done for RPM itself.
More information is available here: https://kubic.opensuse.org/documentation/transactional-update-guide/tu-setup...
This fact could lead to broken symlinks: Suppose I install "go1.20"(snapshot A) and "go1.21" (snapshot B). I then boot from snapshot A, but "/var/lib/alternatives/go" still tells "update-alternatives" that "go1.21" would be a valid choice, since this information lives outside of the snapshot's directories, in "/var". If "update-alternatives" were to accept that choice, then "/etc/alternatives/go" would become a broken link, our last line of defence here being how carefully maintained "update-alternatives" is. So that's not a system-level design solution, but rather very application-specific.
What we're planning: Replace update-alternatives with libalternatives: https://github.com/openSUSE/libalternatives This change should be done gradually on Factory, and ideally it would require no manual intervention from the users. We plan on releasing this change in batches: i.e.: all Go versions we provide on Factory would be migrated to libalternatives in a single-pass. Then another set of closely-related packages, such as the OpenJDK. Then another, so on and so forth.
What we need: The goal of this thread is to establish a line of discussion around this change, which should be a gradual one on Factory. All opinions, comments, ideas and criticism are valid, and will be taken into consideration.
Status: There are currently 14909 spec files on openSUSE:Factory. 964 (6.46%) of these depend on update-alternatives.
Currently 52 packages use libalternatives, such as vim and nodejs-common. Basically, the cores of both MicroOS and SLE Micro are already using libalternatives.
Things that seem to be clear in this particular context:
1 - libalternatives should work really well with transactional updates: As the state of alternatives kept by libalternatives is maintained in /usr/share/libalternatives/<binary>/<pref>.conf,
This means every time program is started it needs to read and parse additional files (and actually more than one). It adds overhead. It adds complications using AppArmor/SELinux.
this means that packages only need to provide a single file to provide their alternative. This file will be part of the package's regular files (declared in %files), and no state is modified by any hooks (preun, postun, prerm, postrm) during the updates. Rolling back a snapshot will revert back to a state where that alternative didn't exist (as it wasn't installed).
2 - libalternatives does not manipulate symlinks: /usr/bin/alts (the main binary for libalternatives) executes binaries directly (by calling exec()). The actually binary (for example, /usr/bin/vi) is a symlink to /usr/bin/alts,
You immediately get problems with AppArmor and SELinux. You cannot assign more than one label to one file.
which then reads its configuration files and determines which binary to execute, based on the highest priority.
3 - libalternatives supports user overrides: non-root users can override the alternatives by creating their own local preference file in $HOME/.config/libalternatives.conf. This may be done by invoking "alts" directly, as it serves both purposes.
This will turn into support nightmare. It need a tool to query and display what a given program actually is showing where this decision comes from (similar to systemd-cat).
4 - Even if it doesn't manipulate symlinks, groups and manpages are still supported: libalternatives has a "group" directive for manipulating groups of binaries together, and it has some verifications to ensure the binaries in these groups are indeed declared in tandem with one another, essentially managing them as a collection. It also has a "man" directive for man pages, so even if it's not doing symlink manipulation, look up man pages still works. In fact, our "man" package was patched to work with libalternatives support: https://build.opensuse.org/package/view_file/Base:System/man/man-db-2.9.4-al...
5 - No RPM macros are required: This is really cool. We're basically just adding a file to %files, to use libalternatives. So no RPM macros are even required here. But we could create an RPM macro that automatically writes the preference file, so we could easily manipulate its contents inside the RPM spec.
6 - How libalternatives can actually be used So there are actually at least TWO ways to use libalternatives: we either create a symlink to /usr/bin/alts, or use libalternatives as a shared object and write a tiny wrapper program, such as this one: https://build.opensuse.org/package/view_file/devel:languages:nodejs/nodejs-c...
The advantage of the first method is that we don't need a special – albeit short – C program to act as a dispatcher for libalternatives's magic sauce.
The advantage of the second method is that we don't need packages to depend on the "alts" package (which contains /usr/bin/alts), but only on "libalternatives1" instead (which provides /usr/lib64/libalternatives.so.1).
I appreciate if you can provide any insights here, they're more than welcome. Thank you.
On Sat, Sep 30, Andrei Borzenkov wrote:
On 29.09.2023 22:11, Martin Schreiner via openSUSE Factory wrote:
Hello folks,
As some of you may know, there are some limitations regarding update-alternatives that prevent it from working perfectly when using transactional updates. Basically, /var isn't part of the snapshot's data. This means that all files created/manipulated during package installation in /var/lib/alternatives won't be taken into consideration when snapshots are rolled back.
What prevents moving this information into / so it is included in snapshot? It was done for RPM itself.
That the informations update-alternatives stores in /etc nor /var are static and thus cannot be put in the read-only part of the system.
You immediately get problems with AppArmor and SELinux. You cannot assign more than one label to one file.
Since how man years are we now using libalternatives with AppArmor and SELinux? I haven't heard from any problems.
3 - libalternatives supports user overrides: non-root users can override the alternatives by creating their own local preference file in $HOME/.config/libalternatives.conf. This may be done by invoking "alts" directly, as it serves both purposes.
This will turn into support nightmare. It need a tool to query and display what a given program actually is showing where this decision comes from (similar to systemd-cat).
Since how many years are we doing this now already? I'm not aware of any support nightmares, not even of support incidents. It's always interesting to see how skeptical people are about change, even though they've been using it for years without noticing it. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
Hello, Am Sonntag, 1. Oktober 2023, 20:43:19 CEST schrieb Thorsten Kukuk:
On Sat, Sep 30, Andrei Borzenkov wrote:
On 29.09.2023 22:11, Martin Schreiner via openSUSE Factory wrote:
You immediately get problems with AppArmor and SELinux. You cannot assign more than one label to one file.
Since how man years are we now using libalternatives with AppArmor and SELinux? I haven't heard from any problems.
I don't remember a bugreport with AppArmor and libalternatives. However, my guess is that the main reason is that so far the programs using libalternatives don't have an AppArmor profile. (Actually I have a few profiles that include read access to /usr/share/libalternatives/, for example the firewalld profile from the apparmor.d project [1].) What could happen is: Programs using libalternatives need to be able to read the config files in /usr/share/libalternatives/, /etc/libalternatives (_guessed_, doesn't exist on my system) and ~/.config/libalternatives.conf If the profile doesn't allow reading files in /usr/share/libalternatives/, I'd guess that things might break in funny ways (I admit that I didn't check the actual libalternatives/alts behaviour in this case). If the profile "just" doesn't allow to read the override in /etc/ or ~/.config/libalternatives.conf, then I'd guess that "just" the system/ user-specific config is ignored. In addition to that, programs that get started using the "alts" binary need an additional exec rule for /usr/bin/alts if the calling process is confined with a profile. So if a program comes with an AppArmor profile and later gets changed to use libalternatives, the profile will need these additions. For completeness: AppArmor rules get checked after resolving symlinks, which means: with update-alternatives symlinks, only permissions for the "real" file are needed.
3 - libalternatives supports user overrides: non-root users can override the alternatives by creating their own local preference file in $HOME/.config/libalternatives.conf. This may be done by invoking "alts" directly, as it serves both purposes.> This will turn into support nightmare. It need a tool to query and display what a given program actually is showing where this decision comes from (similar to systemd-cat).
Such a tool would indeed be helpful for debugging. Maybe as a verbose version of alts -t, additionally saying something like "found in $file"?
Since how many years are we doing this now already? I'm not aware of any support nightmares, not even of support incidents.
I'm not surprised, but let me ask some evil question: - how many users know that they can define their own preferences? - out of those, how many really did so? - out of those, how many didn't remember it when running into problems? I'd expect that the answer to the second question is a quite small number, and for the third question it's close to zero. To really find out, I created a small poll: https://dud-poll.inf.tu-dresden.de/libalternatives/ [2] Please vote - especially if you have a ~/.config/libalternatives.conf (The "what's that file?" and "I know it, but don't have one" options are there for completeness, but the numbers for these are probably not too interesting.) As a side note: I checked the content of the libalternatives1 and alts package, and miss documentation for - the file format in /usr/share/libalternatives/ - what the corresponding path in /etc/ is - the file format of ~/.config/libalternatives.conf (and BTW: why a single file instead of a directory structure like in /usr/?) Can you please add this to the README and/or the alts manpage? All these things are probably not too hard to find out, but I still prefer having some clear documentation ;-)
It's always interesting to see how skeptical people are about change, even though they've been using it for years without noticing it.
Indeed. I'd say that knowing about changes is fine, but often (not always ;-) they are less scary than they sound. Regards, Christian Boltz [1] The apparmor.d project provides a set of ~1500 profiles. As a random sample: From the processes running on my laptop, about 70% are confined - including most of KDE. If someone is interested in testing this profile set - it's available in OBS home:cboltz. (Let me warn you that they are currently still in complain mode. I'll switch them to enforce mode sooner or later without a warning on this ML ;-) [2] That service is similar to doodle, but privacy-friendly (even entering your name is optional) -- If we get Flemish, we should get also Plat-Duutsch, Beirisch and Östereichish. (The last two are the same, but don't tell them that, because the inhabitants think they are not, because they are better then the other. :) [houghi in opensuse]
On Fri, 29 Sep 2023, Martin Schreiner via openSUSE Factory wrote:
Hello folks,
As some of you may know, there are some limitations regarding update-alternatives that prevent it from working perfectly when using transactional updates. Basically, /var isn't part of the snapshot's data. This means that all files created/manipulated during package installation in /var/lib/alternatives won't be taken into consideration when snapshots are rolled back. More information is available here: https://kubic.opensuse.org/documentation/transactional-update-guide/tu-setup...
This fact could lead to broken symlinks: Suppose I install "go1.20"(snapshot A) and "go1.21" (snapshot B). I then boot from snapshot A, but "/var/lib/alternatives/go" still tells "update-alternatives" that "go1.21" would be a valid choice, since this information lives outside of the snapshot's directories, in "/var". If "update-alternatives" were to accept that choice, then "/etc/alternatives/go" would become a broken link, our last line of defence here being how carefully maintained "update-alternatives" is. So that's not a system-level design solution, but rather very application-specific.
What we're planning: Replace update-alternatives with libalternatives: https://github.com/openSUSE/libalternatives This change should be done gradually on Factory, and ideally it would require no manual intervention from the users. We plan on releasing this change in batches: i.e.: all Go versions we provide on Factory would be migrated to libalternatives in a single-pass. Then another set of closely-related packages, such as the OpenJDK. Then another, so on and so forth.
What we need: The goal of this thread is to establish a line of discussion around this change, which should be a gradual one on Factory. All opinions, comments, ideas and criticism are valid, and will be taken into consideration.
Status: There are currently 14909 spec files on openSUSE:Factory. 964 (6.46%) of these depend on update-alternatives.
Currently 52 packages use libalternatives, such as vim and nodejs-common. Basically, the cores of both MicroOS and SLE Micro are already using libalternatives.
Things that seem to be clear in this particular context: ... 5 - No RPM macros are required: This is really cool. We're basically just adding a file to %files, to use libalternatives. So no RPM macros are even required here. But we could create an RPM macro that automatically writes the preference file, so we could easily manipulate its contents inside the RPM spec.
6 - How libalternatives can actually be used So there are actually at least TWO ways to use libalternatives: we either create a symlink to /usr/bin/alts, or use libalternatives as a shared object and write a tiny wrapper program, such as this one: https://build.opensuse.org/package/view_file/devel:languages:nodejs/nodejs-c...
The advantage of the first method is that we don't need a special ? albeit short ? C program to act as a dispatcher for libalternatives's magic sauce.
The advantage of the second method is that we don't need packages to depend on the "alts" package (which contains /usr/bin/alts), but only on "libalternatives1" instead (which provides /usr/lib64/libalternatives.so.1).
How does this work when maintaining packages that need to go 1:1 from Factory to old products like SLES 12 SP5 and SLES 15 SPx? Do I need to keep both the "old" update-alternatives piceces in the spec file and the "new" libalternatives one with %if %{suse_version}? Note I also see in binutils # disable libalternatives for now until it's changed to not # introduce cmake/cunit-tests into the bootstrap cycle %if 0 && 0%{?suse_version} > 1500 %bcond_without libalternatives %else %bcond_with libalternatives %endif how do we address this? Richard. -- Richard Biener <rguenther@suse.de> SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg, Germany; GF: Ivo Totev, Andrew McDonald, Werner Knoblich; (HRB 36809, AG Nuernberg)
On Wed, 4 Oct 2023, Richard Biener wrote:
On Fri, 29 Sep 2023, Martin Schreiner via openSUSE Factory wrote:
Hello folks,
As some of you may know, there are some limitations regarding update-alternatives that prevent it from working perfectly when using transactional updates. Basically, /var isn't part of the snapshot's data. This means that all files created/manipulated during package installation in /var/lib/alternatives won't be taken into consideration when snapshots are rolled back. More information is available here: https://kubic.opensuse.org/documentation/transactional-update-guide/tu-setup...
This fact could lead to broken symlinks: Suppose I install "go1.20"(snapshot A) and "go1.21" (snapshot B). I then boot from snapshot A, but "/var/lib/alternatives/go" still tells "update-alternatives" that "go1.21" would be a valid choice, since this information lives outside of the snapshot's directories, in "/var". If "update-alternatives" were to accept that choice, then "/etc/alternatives/go" would become a broken link, our last line of defence here being how carefully maintained "update-alternatives" is. So that's not a system-level design solution, but rather very application-specific.
What we're planning: Replace update-alternatives with libalternatives: https://github.com/openSUSE/libalternatives This change should be done gradually on Factory, and ideally it would require no manual intervention from the users. We plan on releasing this change in batches: i.e.: all Go versions we provide on Factory would be migrated to libalternatives in a single-pass. Then another set of closely-related packages, such as the OpenJDK. Then another, so on and so forth.
What we need: The goal of this thread is to establish a line of discussion around this change, which should be a gradual one on Factory. All opinions, comments, ideas and criticism are valid, and will be taken into consideration.
Status: There are currently 14909 spec files on openSUSE:Factory. 964 (6.46%) of these depend on update-alternatives.
Currently 52 packages use libalternatives, such as vim and nodejs-common. Basically, the cores of both MicroOS and SLE Micro are already using libalternatives.
Things that seem to be clear in this particular context: ... 5 - No RPM macros are required: This is really cool. We're basically just adding a file to %files, to use libalternatives. So no RPM macros are even required here. But we could create an RPM macro that automatically writes the preference file, so we could easily manipulate its contents inside the RPM spec.
6 - How libalternatives can actually be used So there are actually at least TWO ways to use libalternatives: we either create a symlink to /usr/bin/alts, or use libalternatives as a shared object and write a tiny wrapper program, such as this one: https://build.opensuse.org/package/view_file/devel:languages:nodejs/nodejs-c...
The advantage of the first method is that we don't need a special ? albeit short ? C program to act as a dispatcher for libalternatives's magic sauce.
The advantage of the second method is that we don't need packages to depend on the "alts" package (which contains /usr/bin/alts), but only on "libalternatives1" instead (which provides /usr/lib64/libalternatives.so.1).
How does this work when maintaining packages that need to go 1:1 from Factory to old products like SLES 12 SP5 and SLES 15 SPx? Do I need to keep both the "old" update-alternatives piceces in the spec file and the "new" libalternatives one with %if %{suse_version}? Note I also see in binutils
# disable libalternatives for now until it's changed to not # introduce cmake/cunit-tests into the bootstrap cycle %if 0 && 0%{?suse_version} > 1500 %bcond_without libalternatives %else %bcond_with libalternatives %endif
how do we address this?
Were other alternatives to solve the transactional update issue considered, like placing /var/lib/alternatives in /run/ and populating that via some (systemd?) service at boot time from configuration? How does libalternatives solve the issue that its configuration is by definition not restored by rollback either? It only works to roll back when there's no configuration change inbetween? Richard. -- Richard Biener <rguenther@suse.de> SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg, Germany; GF: Ivo Totev, Andrew McDonald, Werner Knoblich; (HRB 36809, AG Nuernberg)
On Wed, Oct 04, Richard Biener wrote:
Were other alternatives to solve the transactional update issue considered, like placing /var/lib/alternatives in /run/ and populating that via some (systemd?) service at boot time from configuration?
This files are not owned by a RPM and need to survive a reboot. Beside that, /run/ would not solve any of the current problems with update-alternatives.
How does libalternatives solve the issue that its configuration is by definition not restored by rollback either? It only works to roll back when there's no configuration change inbetween?
??? Since libalternatives uses the standard configuration locations and has a clean split between distributon provided configurations and admin made changes, the configuration is of course restored during a rollback. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
On Wed, 4 Oct 2023, Thorsten Kukuk wrote:
On Wed, Oct 04, Richard Biener wrote:
Were other alternatives to solve the transactional update issue considered, like placing /var/lib/alternatives in /run/ and populating that via some (systemd?) service at boot time from configuration?
This files are not owned by a RPM and need to survive a reboot. Beside that, /run/ would not solve any of the current problems with update-alternatives.
Since /run isn't persistent it solves the transactional revert restoring the reverted distribution provided config like libalternatives does.
How does libalternatives solve the issue that its configuration is by definition not restored by rollback either? It only works to roll back when there's no configuration change inbetween?
??? Since libalternatives uses the standard configuration locations and has a clean split between distributon provided configurations and admin made changes, the configuration is of course restored during a rollback.
It rolls back the admin made changes?! Richard. -- Richard Biener <rguenther@suse.de> SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg, Germany; GF: Ivo Totev, Andrew McDonald, Werner Knoblich; (HRB 36809, AG Nuernberg)
On Wed, Oct 04, Richard Biener wrote:
On Wed, 4 Oct 2023, Thorsten Kukuk wrote:
On Wed, Oct 04, Richard Biener wrote:
Were other alternatives to solve the transactional update issue considered, like placing /var/lib/alternatives in /run/ and populating that via some (systemd?) service at boot time from configuration?
This files are not owned by a RPM and need to survive a reboot. Beside that, /run/ would not solve any of the current problems with update-alternatives.
Since /run isn't persistent it solves the transactional revert restoring the reverted distribution provided config like libalternatives does.
/var/lib/alternatives contains data update-alternatives creates during installation of packages using update-alternatives. So if you reboot, this informations are lost, but never rolled back.
How does libalternatives solve the issue that its configuration is by definition not restored by rollback either? It only works to roll back when there's no configuration change inbetween?
??? Since libalternatives uses the standard configuration locations and has a clean split between distributon provided configurations and admin made changes, the configuration is of course restored during a rollback.
It rolls back the admin made changes?!
Of course. If you do a rollback, you want a consistent, known working system again. Which means, admin made config changes, which may not fit anymore and may disrupt the system, are not wanted and needs to be rolled back, too. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
On Wed, 4 Oct 2023, Thorsten Kukuk wrote:
On Wed, Oct 04, Richard Biener wrote:
On Wed, 4 Oct 2023, Thorsten Kukuk wrote:
On Wed, Oct 04, Richard Biener wrote:
Were other alternatives to solve the transactional update issue considered, like placing /var/lib/alternatives in /run/ and populating that via some (systemd?) service at boot time from configuration?
This files are not owned by a RPM and need to survive a reboot. Beside that, /run/ would not solve any of the current problems with update-alternatives.
Since /run isn't persistent it solves the transactional revert restoring the reverted distribution provided config like libalternatives does.
/var/lib/alternatives contains data update-alternatives creates during installation of packages using update-alternatives. So if you reboot, this informations are lost, but never rolled back.
Yes, but if you avoid the persistent /var/lib/alternatives data and instead materialize it in /run at boot time from config data similar to that of libalternatives this is avoided.
How does libalternatives solve the issue that its configuration is by definition not restored by rollback either? It only works to roll back when there's no configuration change inbetween?
??? Since libalternatives uses the standard configuration locations and has a clean split between distributon provided configurations and admin made changes, the configuration is of course restored during a rollback.
It rolls back the admin made changes?!
Of course. If you do a rollback, you want a consistent, known working system again. Which means, admin made config changes, which may not fit anymore and may disrupt the system, are not wanted and needs to be rolled back, too.
Interesting. Then move the /var/lib/alternatives symlinks to the same place as the admin config and have the problem solved as well? Richard. -- Richard Biener <rguenther@suse.de> SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg, Germany; GF: Ivo Totev, Andrew McDonald, Werner Knoblich; (HRB 36809, AG Nuernberg)
To clarify and enhance my original post (and I may use this reply to various sub-threads here in this thread, since there's a lot going on). The issue isn't just that currently, information is stored in /var/lib/alternatives. Suppose we were to move this information elsewhere, to a directory that is part of the snapshotting system, say, /usr/lib/alternatives. This would ensure that yes, we can rollback safely, and the problem I described using 'go1.20' and 'go1.21' as examples would be SOLVED. That is true. Correct me if I'm wrong here, Thorsten. Here's a recap: 1. User installs go1.20 (snapshot A). 2. User installs go1.21 (snapshot B). 3. User rolls back to snapshot A. Both go1.21 and its 'alternatives' information are completely gone, since they were never part of snapshot A, and this information no longer resides in a directory that is NOT part of the snapshot system. At least, that is true if we trust the original documentation for transactional updates: https://kubic.opensuse.org/documentation/transactional-update-guide/tu-howit... There, we can see clearly that, for the duration of 'zypper dup' running, the filesystem is indeed mounted 'rw', which is how all file-manipulating hooks (post-install, pre-install, etc) can write to the filesystem directly. Still, another issue remains: we want users to be able to override distribution-provided defaults. That's the whole point of having an alternatives system: I can switch the system from go1.20 to go1.21 on a whim, so long as I have both versions installed. This is why update-alternatives has the concept of 'auto' and 'manual'. A decision may have been made by the system itself (auto), or by a user explicitly setting a version (manual). Here's what update-alternatives doesn't provide, *even if we somehow move its settings to a snapshotted directory*: ->>> A way for users to override distribution-specific defaults that doesn't need to modify the read-only filesystem. Basically, an overlay. This is what libalternatives does provide, as users may set their own alternatives in their home directories. This overlay is important because the distribution-specific defaults will be snapshotted, and mounted read-only. Therefore the user cannot interfere there. If the snapshot is rolled back, the user's settings are gone. And because the directory is read-only, they cannot be written there in the first place. Thus, the home directory is a good place to do it. Should users pick 'go1.21' as their local override, then revert back to snapshot A (which doesn't have go1.21 installed), then libalternatives can choose its highest-priority distribution-default instead. Or warn the user 'hey, your alternative is broken, I'm using go1.20 instead'. We have an option here. Now, we could patch update-alternatives to support this. This begs the question, however: as we know there's both a Fedora and a Debian implementation of update-alternatives, do we stick with the Debian one? Do we move to the Fedora one? Which one of these two groups is more likely to make these changes upstream? Do they even want to? Do we even want to have these changes be a part of upstream? I know we have to improve libalternative's documentation, and maybe add a flag to improve its instrospectability. But I think it'd be easier to make these improvements on libalternatives, than changing how update-alternatives works from the ground up, to accomodate our needs.
On Thu, 12 Oct 2023, Martin Schreiner via openSUSE Factory wrote:
To clarify and enhance my original post (and I may use this reply to various sub-threads here in this thread, since there's a lot going on).
The issue isn't just that currently, information is stored in /var/lib/alternatives. Suppose we were to move this information elsewhere, to a directory that is part of the snapshotting system, say, /usr/lib/alternatives. This would ensure that yes, we can rollback safely, and the problem I described using 'go1.20' and 'go1.21' as examples would be SOLVED. That is true.
Correct me if I'm wrong here, Thorsten.
Here's a recap: 1. User installs go1.20 (snapshot A). 2. User installs go1.21 (snapshot B). 3. User rolls back to snapshot A. Both go1.21 and its 'alternatives' information are completely gone, since they were never part of snapshot A, and this information no longer resides in a directory that is NOT part of the snapshot system.
At least, that is true if we trust the original documentation for transactional updates: https://kubic.opensuse.org/documentation/transactional-update-guide/tu-howit... There, we can see clearly that, for the duration of 'zypper dup' running, the filesystem is indeed mounted 'rw', which is how all file-manipulating hooks (post-install, pre-install, etc) can write to the filesystem directly.
Still, another issue remains: we want users to be able to override distribution-provided defaults. That's the whole point of having an alternatives system: I can switch the system from go1.20 to go1.21 on a whim, so long as I have both versions installed. This is why update-alternatives has the concept of 'auto' and 'manual'. A decision may have been made by the system itself (auto), or by a user explicitly setting a version (manual).
Here's what update-alternatives doesn't provide, *even if we somehow move its settings to a snapshotted directory*:
->>> A way for users to override distribution-specific defaults that ->>> doesn't need to modify the read-only filesystem. Basically, an ->>> overlay.
But your original mail framed it as a solution for a snapshot issue. But there isn't any snapshot issue, is there?
This is what libalternatives does provide, as users may set their own alternatives in their home directories. This overlay is important because the distribution-specific defaults will be snapshotted, and mounted read-only. Therefore the user cannot interfere there. If the snapshot is rolled back, the user's settings are gone. And because the directory is read-only, they cannot be written there in the first place. Thus, the home directory is a good place to do it. Should users pick 'go1.21' as their local override, then revert back to snapshot A (which doesn't have go1.21 installed), then libalternatives can choose its highest-priority distribution-default instead. Or warn the user 'hey, your alternative is broken, I'm using go1.20 instead'. We have an option here.
Now, we could patch update-alternatives to support this. This begs the question, however: as we know there's both a Fedora and a Debian implementation of update-alternatives, do we stick with the Debian one? Do we move to the Fedora one? Which one of these two groups is more likely to make these changes upstream? Do they even want to? Do we even want to have these changes be a part of upstream?
I know we have to improve libalternative's documentation, and maybe add a flag to improve its instrospectability. But I think it'd be easier to make these improvements on libalternatives, than changing how update-alternatives works from the ground up, to accomodate our needs.
How's doing a NIH 3rd tool going to make the world better? What's wrong with handling user alternatives via PATH? update-alternatives currently works as root only, if invoked as user it could look for a $HOME/.alternatives and maintain a second symlink farm there with user overrides (and suggest putting $HOME/.alternatives to the users PATH if it is not there). A shared lib is really really very ugly. Richard. -- Richard Biener <rguenther@suse.de> SUSE Software Solutions Germany GmbH, Frankenstrasse 146, 90461 Nuernberg, Germany; GF: Ivo Totev, Andrew McDonald, Werner Knoblich; (HRB 36809, AG Nuernberg)
Hi, maybe time again to explain why "update-alternatives" and the usage of /etc and /var/lib during installation and update is a problem, and what the use cases are we need to solve: There are three "ways" to deploy/install/update a system: * the traditional way like Tumbleweed is doing, one fat, read-write root partition and the whole system is accessible. * transactional-update with a limited root filesystem in a subvolume not including /var (because of rollback of static and volatile data). Means during update you have access to /etc and /usr and not much more, no /var (with exceptions, but most of the data there is not accessible after a reboot by the user!). * image based, normally means only /usr, no /etc, no /var available during update. transactional-update and image based updates are very similar from the requirements, except that image based has some additional requirements: * no /etc * no RPM pre/post install scripts during installation and update In the systemd or UAPI group world this is called "hermetic /usr/". update-alternatives makes changes in /etc and /var during installation and update. With transactional-update, /var is a real problem and /etc is not helpful. With an image based approach, update-alternatives does not work at all, because: * there is no /etc * there is no /var * you cannot call it from a pre/post install script For our transactional-update and image based products, somebody did implement libalternatives years ago and we converted already the core base system. So in the minimal default installation for e.g. a container host OS, update-alternatives is no longer in use. But if users want to install additional packages, which are still using update-alternatives, we have a problem. Every solution for the update-alternatives problem need to follow: https://github.com/uapi-group/specifications/blob/main/specs/configuration_f... Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
On 10/16/23 11:04, Richard Biener wrote:
A shared lib is really really very ugly.
Is the sticking point because of the shared library? :-) The reason for shared library is to be able to do equivalent of exec() call -- libalts_exec_default(). That's the point - to get access to the API for this. The rest is just convenience stuff. /usr/bin/{node/npm/npx} actually use this API for its own wonderful reason. :-) Of course this is not a direct replacement for symlinks. But I think the intentions of designer of update-alternatives was not to have the mess it created today. Like, who actually changed update-alternative defaults on regular basis? I think I only did this a handful of times in 2 decades. (Aside: from what I remember, if my memory doesn't disappoint me, origin of u-a was about who owns some binary name and super-long threads that happened in Debian around it -- u-a to the rescue) - Adam
Hey Richard,
But your original mail framed it as a solution for a snapshot issue. But there isn't any snapshot issue, is there?
Yes, there is. As was exemplified in my original message, if we keep everything in /var, then the information is never part of any snapshots. Settings in /etc/alternatives would be, settings in /var/lib/alternatives wouldn't be. If a user reverts to an older snapshot, then /var/lib/alternatives may not be in-sync with that, and a user can potentially make a choice that leads to broken symlinks. Some of this information plugs into what Thorsten said here. Yes, some parts of what he said are new information to me. If we move stuff to /usr/lib/alternatives, the following implication arises: 1: For transactional-updates, then the state would be synchronised. The snapshots would always be consistent with the alternatives system. Users wouldn't be able to break symlinks, because they wouldn't be given the choice to pick an alternative that's not currently installed. HOWEVER, there's still an issue: the filesystem is mounted read-only, therefore users can't make any changes without an overlay system. This issue is true regardless of where information is stored: 2: update-alternatives uses RPM pre/post install hooks to modify files in /var/lib/alternatives (or wherever we choose), so it's unsuitable for an image-based approach. libalternatives ships bare files which are part of the RPM package itself, doesn't use any hooks, and would solve this issue. Note that there may be other ways to handle this, I'm not denying that, just organising our requirements list. 3: We could use the overlay system you suggested: local symlinks inside $HOME, and changing $HOME. Some thoughts: 3.1: What happens if a user boots from an older snapshot, and all his symlinks are broken? Sure, we can say users will just have to deal with that. But I think that needs to be a conscious decision. libalternatives, being a library/executable, allows us to tell the user their current choice is invalid ahead of calling exec(), should they ever revert to an older snapshot that removes binaries the user has chosen in his local overlay. 3.2: How do you suggest we handle the scenario regarding shebangs/default interpreters, if we were to use symlinks in $HOME? Editing $PATH isn't system-wide. Because of the read-only nature of image-based systems, the overlays would always be per-user, even when using libalternatives. But at the very least, a user can always say "hey, I want /usr/bin/ruby to always be this specific version". All scripts pointing to /usr/bin/ruby will respect the user's choice. With symlinks, we actually have two versions: the system-wide one, and the user's local symlink. And the user has no way to set a global option that's respected seamlessly by the whole system. Again, this feature is provided by both update-alternatives and libalternatives, but a $PATH-based one raises the question.
How's doing a NIH 3rd tool going to make the world better?
It might be an opportunity to solve these issues gracefully, if we explore these issues in detail and do a good job, I'd say that's a win, even if it is a third tool. So much in open source is just a gradual, slightly better or different version of an older idea.
How does this work when maintaining packages that need to go 1:1 from Factory to old products like SLES 12 SP5 and SLES 15 SPx? Do I need to keep both the "old" update-alternatives piceces in the spec file and the "new" libalternatives one with %if %{suse_version}?
Yes, in this case we need to maintain compatibility with both codestreams with a single spec file, so that'd be the approach.
Note I also see in binutils # disable libalternatives for now until it's changed to not # introduce cmake/cunit-tests into the bootstrap cycle %if 0 && 0%{?suse_version} > 1500 %bcond_without libalternatives %else %bcond_with libalternatives %endif how do we address this? Richard.
This, in particular, is easily solved. I just did a build last night of libalternatives that doesn't require cunit-tests. CMake, yes. cunit-tests, no. With RPM 4.19 migrating to CMake, as Neal mentioned further down this ML, it might become part of the bootstrap dependencies for Factory/Tumbleweed/SLE16. So we can remove this cunit-tests dependency from upstream libalternatives (at least for a minimal build), and I'd say it's solved.
On Fri, 2023-09-29 at 19:11 +0000, Martin Schreiner via openSUSE Factory wrote:
What we need: The goal of this thread is to establish a line of discussion around this change, which should be a gradual one on Factory. All opinions, comments, ideas and criticism are valid, and will be taken into consideration.
Status: There are currently 14909 spec files on openSUSE:Factory. 964 (6.46%) of these depend on update-alternatives.
One major concern I have is the build system it uses. Currently it relies on cmake, which is not part of Ring0. So anything in ring0 has a hard time using libalternatives. If we can either port this to meson (or autoconf, if it must be) we could win that battle easily though. We try to keep the distro bootstrap as small as possible (and thus veto everything new as much as we can) Cheers, Dominique
On Wednesday 2023-10-04 09:24, Dominique Leuenberger wrote:
One major concern I have is the build system it uses. Currently it relies on cmake, which is not part of Ring0.
So anything in ring0 has a hard time using libalternatives.
If we can either port this to meson (or autoconf, if it must be) we could win that battle easily though.
There are plain Makefiles in libalternatives. But since autoconf takes just as many lines as the custom Makefiles, sure why not, say no more - https://github.com/jengelh/libalternatives This also adds the testing target and I think coverage flags, so if one were to ditch the cmake files now, there would be even more of a LOC win.
On 10/4/23 10:58, Jan Engelhardt wrote:
There are plain Makefiles in libalternatives. But since autoconf takes just as many lines as the custom Makefiles, sure why not, say no more - https://github.com/jengelh/libalternatives This also adds the testing target and I think coverage flags, so if one were to ditch the cmake files now, there would be even more of a LOC win.
Or keep both, cmake and autotools, as it's simple enough. Please do a PR. - Adam
On Wed, Oct 4, 2023 at 3:25 AM Dominique Leuenberger <dimstar@opensuse.org> wrote:
On Fri, 2023-09-29 at 19:11 +0000, Martin Schreiner via openSUSE Factory wrote:
What we need: The goal of this thread is to establish a line of discussion around this change, which should be a gradual one on Factory. All opinions, comments, ideas and criticism are valid, and will be taken into consideration.
Status: There are currently 14909 spec files on openSUSE:Factory. 964 (6.46%) of these depend on update-alternatives.
One major concern I have is the build system it uses. Currently it relies on cmake, which is not part of Ring0.
So anything in ring0 has a hard time using libalternatives.
If we can either port this to meson (or autoconf, if it must be) we could win that battle easily though.
We try to keep the distro bootstrap as small as possible (and thus veto everything new as much as we can)
Too late, RPM converted to CMake in RPM 4.19. -- 真実はいつも一つ！/ Always, there's only one truth!
Seeing as RPM 4.19 now depends on CMake, as Neal pointed out, what's our strategy going forward here?
On Thursday 2023-10-12 17:26, Martin Schreiner via openSUSE Factory wrote:
Seeing as RPM 4.19 now depends on CMake, as Neal pointed out, what's our strategy going forward here?
Adding back a somewhat minimal Makefile.am or something just for our case? (Or call $CC directly with the source file list we desire.) No conditionals, just hard-select e.g. the crypto lib to use. rpm's chanleog """When libsolv, dnf and all the related stack is powered by cmake build, it'd be just foolish to go with anything else. This way people working on the rpm stack have only one build system to learn""" seems to have missed https://xkcd.com/927/ .
On Fri, Oct 13, 2023 at 6:17 PM Jan Engelhardt <jengelh@inai.de> wrote:
On Thursday 2023-10-12 17:26, Martin Schreiner via openSUSE Factory wrote:
Seeing as RPM 4.19 now depends on CMake, as Neal pointed out, what's our strategy going forward here?
Adding back a somewhat minimal Makefile.am or something just for our case? (Or call $CC directly with the source file list we desire.) No conditionals, just hard-select e.g. the crypto lib to use.
rpm's chanleog """When libsolv, dnf and all the related stack is powered by cmake build, it'd be just foolish to go with anything else. This way people working on the rpm stack have only one build system to learn""" seems to have missed https://xkcd.com/927/ .
Also, all of us in RPM upstream hate autotools. Meson is too unstable to use and the rest of the RPM stack stuff is all CMake anyway. It was a logical choice. The XKCD reference is not appropriate for this situation, as we are reducing the number of things used, not increasing them. -- 真実はいつも一つ！/ Always, there's only one truth!
On Saturday 2023-10-14 00:21, Neal Gompa wrote:
Seeing as RPM 4.19 now depends on CMake, as Neal pointed out, what's our strategy going forward here?
rpm [...] """have only one build system to learn""" [...] seems to have missed https://xkcd.com/927/ .
The XKCD reference is not appropriate for this situation, as we are reducing the number of things used, not increasing them.
See, the comic is not just about making aware of the one-more-standard problem. A more subtext-y aspect is that the depicted party (panel 2) is an overly devout believer (defender of their argument). Another aspect is that there is a poorly-communicated limitation of scope, which is to say that: the development of the universal standard will /indeed/ lead to the desired end result, but /only/ in the sphere of influence of the party. See, cmake may (indeed) be the universal standard -- for rpm-the-upstream-project. But it also cannot be denied that cmake /is/ poised to become the "15th" item in ring-0, because autoconf is not going away in ring-0, bringing us to panel 3 of the comic.
On Sat, Oct 14, Jan Engelhardt wrote:
See, cmake may (indeed) be the universal standard -- for rpm-the-upstream-project. But it also cannot be denied that cmake /is/ poised to become the "15th" item in ring-0, because autoconf is not going away in ring-0, bringing us to panel 3 of the comic.
autoconf and meson are in ring-0, and none of them will go away. That meson made it already into ring-0 and CMake after all the dekades still not, should show what the better tool is ;) -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
On Fri, Oct 13, Neal Gompa wrote:
Also, all of us in RPM upstream hate autotools. Meson is too unstable to use and the rest of the RPM stack stuff is all CMake anyway. It was a logical choice.
??? There are big projects using meson like systemd, but I'm not aware of any big project using CMake without twists and hazzle. And compared to sysemd RPM is really, really small. I would claim that meson is stable and much better suited than CMake. Don't know why you think Meson is too unstable. All my projects use meanwhile meson and I did never run in any problems. What I cannot say about the projects using CMake where I'm from time to time involved. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect, Future Technologies SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany Managing Director: Ivo Totev, Andrew McDonald, Werner Knoblich (HRB 36809, AG Nürnberg)
Am 29.09.23 um 21:11 schrieb Martin Schreiner via openSUSE Factory:
2 - libalternatives does not manipulate symlinks: /usr/bin/alts (the main binary for libalternatives) executes binaries directly (by calling exec()). The actually binary (for example, /usr/bin/vi) is a symlink to /usr/bin/alts, which then reads its configuration files and determines which binary to execute, based on the highest priority.
I would prefer if we could keep symlinks. I don't care where they live, but symlinks are much more restricted and transparent than `exec`: $ readlink -f $(which clang) /usr/bin/clang-17.0.2
3 - libalternatives supports user overrides: non-root users can override the alternatives by creating their own local preference file in $HOME/.config/libalternatives.conf. This may be done by invoking "alts" directly, as it serves both purposes.
This alone doesn't prohibit symlinks: local overrides could be done via symlinks in ~/bin, and it would still be transparent.
4 - Even if it doesn't manipulate symlinks, groups and manpages are still supported: libalternatives has a "group" directive for manipulating groups of binaries together, and it has some verifications to ensure the binaries in these groups are indeed declared in tandem with one another, essentially managing them as a collection.
This could be an issue in LLVM, because new versions tend to add/remove binaries or manpages. But I should probably migrate that anyway from update-alternatives to GCC-style hardcoded symlinks in the metapackage. Aaron
On Mon, Oct 16, 2023 at 10:16 PM Aaron Puchert <aaronpuchert@alice-dsl.net> wrote:
Am 29.09.23 um 21:11 schrieb Martin Schreiner via openSUSE Factory:
2 - libalternatives does not manipulate symlinks: /usr/bin/alts (the main binary for libalternatives) executes binaries directly (by calling exec()). The actually binary (for example, /usr/bin/vi) is a symlink to /usr/bin/alts, which then reads its configuration files and determines which binary to execute, based on the highest priority.
I would prefer if we could keep symlinks. I don't care where they live, but symlinks are much more restricted and transparent than `exec`:
$ readlink -f $(which clang) /usr/bin/clang-17.0.2
3 - libalternatives supports user overrides: non-root users can override the alternatives by creating their own local preference file in $HOME/.config/libalternatives.conf. This may be done by invoking "alts" directly, as it serves both purposes.
This alone doesn't prohibit symlinks: local overrides could be done via symlinks in ~/bin, and it would still be transparent.
4 - Even if it doesn't manipulate symlinks, groups and manpages are still supported: libalternatives has a "group" directive for manipulating groups of binaries together, and it has some verifications to ensure the binaries in these groups are indeed declared in tandem with one another, essentially managing them as a collection.
This could be an issue in LLVM, because new versions tend to add/remove binaries or manpages. But I should probably migrate that anyway from update-alternatives to GCC-style hardcoded symlinks in the metapackage.
Instead of going straight to libalternatives, maybe we should consider evaluating the usage of alternatives in the first place. "Defaults" metapackages and swappable packages (packages using Provides+Conflicts to ensure one-and-only-one provider installed at a time) are both better options for managing system-wide pointers that RPM supports today. -- 真実はいつも一つ！/ Always, there's only one truth!
On 10/17/23 12:49, Neal Gompa wrote:
On Mon, Oct 16, 2023 at 10:16 PM Aaron Puchert <aaronpuchert@alice-dsl.net> wrote:
Am 29.09.23 um 21:11 schrieb Martin Schreiner via openSUSE Factory:
2 - libalternatives does not manipulate symlinks: /usr/bin/alts (the main binary for libalternatives) executes binaries directly (by calling exec()). The actually binary (for example, /usr/bin/vi) is a symlink to /usr/bin/alts, which then reads its configuration files and determines which binary to execute, based on the highest priority.
I would prefer if we could keep symlinks. I don't care where they live, but symlinks are much more restricted and transparent than `exec`:
$ readlink -f $(which clang) /usr/bin/clang-17.0.2
3 - libalternatives supports user overrides: non-root users can override the alternatives by creating their own local preference file in $HOME/.config/libalternatives.conf. This may be done by invoking "alts" directly, as it serves both purposes.
This alone doesn't prohibit symlinks: local overrides could be done via symlinks in ~/bin, and it would still be transparent.
4 - Even if it doesn't manipulate symlinks, groups and manpages are still supported: libalternatives has a "group" directive for manipulating groups of binaries together, and it has some verifications to ensure the binaries in these groups are indeed declared in tandem with one another, essentially managing them as a collection.
This could be an issue in LLVM, because new versions tend to add/remove binaries or manpages. But I should probably migrate that anyway from update-alternatives to GCC-style hardcoded symlinks in the metapackage.
Instead of going straight to libalternatives, maybe we should consider evaluating the usage of alternatives in the first place. "Defaults" metapackages and swappable packages (packages using Provides+Conflicts to ensure one-and-only-one provider installed at a time) are both better options for managing system-wide pointers that RPM supports today.
This is probably a good opportunity to re-asses which packages are using alternatives and whether they still need to. But my gut feeling is it probably doesn't make sense to drop it everywhere. For example I generally have lightdm and gdm both installed on my machines and some day's I end up switching between a few times as part of testing and i'd prefer not to do this via the package manager. From a packaging perspective we were also just looking at ImageMagick where alternatives is used for config files and is probably a cleaner solution then using _multibuild to create subpackages with the config in the default location. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
On Tue, Oct 17, 2023 at 9:36 AM Simon Lees <sflees@suse.de> wrote:
On 10/17/23 12:49, Neal Gompa wrote:
On Mon, Oct 16, 2023 at 10:16 PM Aaron Puchert <aaronpuchert@alice-dsl.net> wrote:
Am 29.09.23 um 21:11 schrieb Martin Schreiner via openSUSE Factory:
2 - libalternatives does not manipulate symlinks: /usr/bin/alts (the main binary for libalternatives) executes binaries directly (by calling exec()). The actually binary (for example, /usr/bin/vi) is a symlink to /usr/bin/alts, which then reads its configuration files and determines which binary to execute, based on the highest priority.
I would prefer if we could keep symlinks. I don't care where they live, but symlinks are much more restricted and transparent than `exec`:
$ readlink -f $(which clang) /usr/bin/clang-17.0.2
3 - libalternatives supports user overrides: non-root users can override the alternatives by creating their own local preference file in $HOME/.config/libalternatives.conf. This may be done by invoking "alts" directly, as it serves both purposes.
This alone doesn't prohibit symlinks: local overrides could be done via symlinks in ~/bin, and it would still be transparent.
4 - Even if it doesn't manipulate symlinks, groups and manpages are still supported: libalternatives has a "group" directive for manipulating groups of binaries together, and it has some verifications to ensure the binaries in these groups are indeed declared in tandem with one another, essentially managing them as a collection.
This could be an issue in LLVM, because new versions tend to add/remove binaries or manpages. But I should probably migrate that anyway from update-alternatives to GCC-style hardcoded symlinks in the metapackage.
Instead of going straight to libalternatives, maybe we should consider evaluating the usage of alternatives in the first place. "Defaults" metapackages and swappable packages (packages using Provides+Conflicts to ensure one-and-only-one provider installed at a time) are both better options for managing system-wide pointers that RPM supports today.
This is probably a good opportunity to re-asses which packages are using alternatives and whether they still need to. But my gut feeling is it probably doesn't make sense to drop it everywhere.
For example I generally have lightdm and gdm both installed on my machines and some day's I end up switching between a few times as part of testing and i'd prefer not to do this via the package manager.
From a packaging perspective we were also just looking at ImageMagick where alternatives is used for config files and is probably a cleaner solution then using _multibuild to create subpackages with the config in the default location.
The mechanism we use in openSUSE should be ripped out and switched for the systemd unit mechanism. We don't need to make display managers mutually exclusive to avoid the usage of alternatives there. Fedora did this ten years ago when they ripped out prefdm for directly using the display-manager.service unit: https://fedoraproject.org/wiki/Features/DisplayManagerRework -- 真実はいつも一つ！/ Always, there's only one truth!
On 10/18/23 00:17, Neal Gompa wrote:
On Tue, Oct 17, 2023 at 9:36 AM Simon Lees <sflees@suse.de> wrote:
On 10/17/23 12:49, Neal Gompa wrote:
On Mon, Oct 16, 2023 at 10:16 PM Aaron Puchert <aaronpuchert@alice-dsl.net> wrote:
Am 29.09.23 um 21:11 schrieb Martin Schreiner via openSUSE Factory:
2 - libalternatives does not manipulate symlinks: /usr/bin/alts (the main binary for libalternatives) executes binaries directly (by calling exec()). The actually binary (for example, /usr/bin/vi) is a symlink to /usr/bin/alts, which then reads its configuration files and determines which binary to execute, based on the highest priority.
I would prefer if we could keep symlinks. I don't care where they live, but symlinks are much more restricted and transparent than `exec`:
$ readlink -f $(which clang) /usr/bin/clang-17.0.2
3 - libalternatives supports user overrides: non-root users can override the alternatives by creating their own local preference file in $HOME/.config/libalternatives.conf. This may be done by invoking "alts" directly, as it serves both purposes.
This alone doesn't prohibit symlinks: local overrides could be done via symlinks in ~/bin, and it would still be transparent.
4 - Even if it doesn't manipulate symlinks, groups and manpages are still supported: libalternatives has a "group" directive for manipulating groups of binaries together, and it has some verifications to ensure the binaries in these groups are indeed declared in tandem with one another, essentially managing them as a collection.
This could be an issue in LLVM, because new versions tend to add/remove binaries or manpages. But I should probably migrate that anyway from update-alternatives to GCC-style hardcoded symlinks in the metapackage.
Instead of going straight to libalternatives, maybe we should consider evaluating the usage of alternatives in the first place. "Defaults" metapackages and swappable packages (packages using Provides+Conflicts to ensure one-and-only-one provider installed at a time) are both better options for managing system-wide pointers that RPM supports today.
This is probably a good opportunity to re-asses which packages are using alternatives and whether they still need to. But my gut feeling is it probably doesn't make sense to drop it everywhere.
For example I generally have lightdm and gdm both installed on my machines and some day's I end up switching between a few times as part of testing and i'd prefer not to do this via the package manager.
From a packaging perspective we were also just looking at ImageMagick where alternatives is used for config files and is probably a cleaner solution then using _multibuild to create subpackages with the config in the default location.
The mechanism we use in openSUSE should be ripped out and switched for the systemd unit mechanism. We don't need to make display managers mutually exclusive to avoid the usage of alternatives there.
Fedora did this ten years ago when they ripped out prefdm for directly using the display-manager.service unit: https://fedoraproject.org/wiki/Features/DisplayManagerRework
Well still, I am sure there are other cases where using alternatives provides a better user experience then having to uninstall / reinstall packages everytime. For those cases its worth keeping something for other cases where almost no one touches it post the first install then doing it with packaging probably does make more sense. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
Neal Gompa wrote:
[...]
Instead of going straight to libalternatives, maybe we should consider evaluating the usage of alternatives in the first place. "Defaults" metapackages and swappable packages (packages using Provides+Conflicts to ensure one-and-only-one provider installed at a time) are both better options for managing system-wide pointers that RPM supports today.
Yes, please! See bash-sh for an example. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH; GF: Ivo Totev, Andrew McDonald, Werner Knoblich; HRB 36809 (AG Nürnberg)
On Tue, Oct 17, 2023 at 12:40:20AM +0200, Aaron Puchert wrote:
Am 29.09.23 um 21:11 schrieb Martin Schreiner via openSUSE Factory:
2 - libalternatives does not manipulate symlinks: /usr/bin/alts (the main binary for libalternatives) executes binaries directly (by calling exec()). The actually binary (for example, /usr/bin/vi) is a symlink to /usr/bin/alts, which then reads its configuration files and determines which binary to execute, based on the highest priority.
I would prefer if we could keep symlinks. I don't care where they live, but symlinks are much more restricted and transparent than `exec`:
$ readlink -f $(which clang) /usr/bin/clang-17.0.2
3 - libalternatives supports user overrides: non-root users can override the alternatives by creating their own local preference file in $HOME/.config/libalternatives.conf. This may be done by invoking "alts" directly, as it serves both purposes.
This alone doesn't prohibit symlinks: local overrides could be done via symlinks in ~/bin, and it would still be transparent.
4 - Even if it doesn't manipulate symlinks, groups and manpages are still supported: libalternatives has a "group" directive for manipulating groups of binaries together, and it has some verifications to ensure the binaries in these groups are indeed declared in tandem with one another, essentially managing them as a collection.
This could be an issue in LLVM, because new versions tend to add/remove binaries or manpages. But I should probably migrate that anyway from update-alternatives to GCC-style hardcoded symlinks in the metapackage.
Please don't. The hardcoded links are the worst, especially in the cases when binaries are often added/removed. Thanks Michal
Am 17.10.23 um 08:38 schrieb Michal Suchánek:
On Tue, Oct 17, 2023 at 12:40:20AM +0200, Aaron Puchert wrote:
[...] But I should probably migrate [LLVM] from update-alternatives to GCC-style hardcoded symlinks in the metapackage. Please don't.
The hardcoded links are the worst, especially in the cases when binaries are often added/removed.
Could you elaborate what the issue would be? What I'm thinking of is having the metapackage contain symlinks for all files in /usr/bin of the corresponding versioned package. So if llvm17 has /usr/bin/opt-17 (let's say we go with the major version only), then llvm = 17 would have /usr/bin/opt -> opt-17. Quite similar to GCC actually, except with more binaries (and man pages). Aaron
On Wed, Oct 18, 2023 at 12:59:05AM +0200, Aaron Puchert wrote:
Am 17.10.23 um 08:38 schrieb Michal Suchánek:
On Tue, Oct 17, 2023 at 12:40:20AM +0200, Aaron Puchert wrote:
[...] But I should probably migrate [LLVM] from update-alternatives to GCC-style hardcoded symlinks in the metapackage. Please don't.
The hardcoded links are the worst, especially in the cases when binaries are often added/removed.
Could you elaborate what the issue would be? What I'm thinking of is having the metapackage contain symlinks for all files in /usr/bin of the corresponding versioned package. So if llvm17 has /usr/bin/opt-17 (let's say we go with the major version only), then llvm = 17 would have /usr/bin/opt -> opt-17. Quite similar to GCC actually, except with more binaries (and man pages).
And changing what the gcc links point to requires you .. to rebuild a package. Pin it to specific revision that points to the version you want, if you find one. And if you don't adapt the current one to point to a different version .. except that fails with ever-changing list of binaries. And have one project per gcc version you want to build with, too. Thanks Michal
Am 18.10.23 um 19:25 schrieb Michal Suchánek:
On Tue, Oct 17, 2023 at 12:40:20AM +0200, Aaron Puchert wrote:
[...] But I should probably migrate [LLVM] from update-alternatives to GCC-style hardcoded symlinks in the metapackage. Please don't.
The hardcoded links are the worst, especially in the cases when binaries are often added/removed. Could you elaborate what the issue would be? What I'm thinking of is having
Am 17.10.23 um 08:38 schrieb Michal Suchánek: the metapackage contain symlinks for all files in /usr/bin of the corresponding versioned package. So if llvm17 has /usr/bin/opt-17 (let's say we go with the major version only), then llvm = 17 would have /usr/bin/opt -> opt-17. Quite similar to GCC actually, except with more binaries (and man pages). And changing what the gcc links point to requires you .. to rebuild a
On Wed, Oct 18, 2023 at 12:59:05AM +0200, Aaron Puchert wrote: package. Pin it to specific revision that points to the version you want, if you find one.
And if you don't adapt the current one to point to a different version .. except that fails with ever-changing list of binaries.
And have one project per gcc version you want to build with, too.
So you're talking about usage in OBS, not in deployment? This is actually what worries me about the update-alternatives mechanism. Some packages want to build against older Clang versions, but use unversioned commands. This works as long as no other package (possibly transitively) requires a newer version for some reason. We've mostly been lucky until now, and the cases that I've seen were the other way around [1], but it's definitely fragile to require a package not to be there. (Just to be clear: we want to allow different compiler versions to coexist, so conflicts are not acceptable.) In such cases I think the proper way is to tell the build system to use the versioned compiler, e.g. via CC=clang-V CXX=clang++-V (or similar for GCC), and not to twist the symlink. The symlinks should in my view point to the distribution default. In deployment, you can of course override the distribution default in /usr/local/bin, but I wouldn't recommend it. (For a short time I used ld -> ld.lld, but that broke stuff because lld at that time didn't know GCC's -flto=auto, which is in the default flags baked into some packages.) Aaron [1] <https://build.opensuse.org/request/show/1114550>
On Thu, Oct 19, 2023 at 12:53:50AM +0200, Aaron Puchert wrote:
Am 18.10.23 um 19:25 schrieb Michal Suchánek:
On Tue, Oct 17, 2023 at 12:40:20AM +0200, Aaron Puchert wrote:
[...] But I should probably migrate [LLVM] from update-alternatives to GCC-style hardcoded symlinks in the metapackage. Please don't.
The hardcoded links are the worst, especially in the cases when binaries are often added/removed. Could you elaborate what the issue would be? What I'm thinking of is having
Am 17.10.23 um 08:38 schrieb Michal Suchánek: the metapackage contain symlinks for all files in /usr/bin of the corresponding versioned package. So if llvm17 has /usr/bin/opt-17 (let's say we go with the major version only), then llvm = 17 would have /usr/bin/opt -> opt-17. Quite similar to GCC actually, except with more binaries (and man pages). And changing what the gcc links point to requires you .. to rebuild a
On Wed, Oct 18, 2023 at 12:59:05AM +0200, Aaron Puchert wrote: package. Pin it to specific revision that points to the version you want, if you find one.
And if you don't adapt the current one to point to a different version .. except that fails with ever-changing list of binaries.
And have one project per gcc version you want to build with, too.
So you're talking about usage in OBS, not in deployment? This is actually what worries me about the update-alternatives mechanism. Some packages want to build against older Clang versions, but use unversioned commands. This works as long as no other package (possibly transitively) requires a newer version for some reason. We've mostly been lucky until now, and the cases that I've seen were the other way around [1], but it's definitely fragile to require a package not to be there. (Just to be clear: we want to allow different compiler versions to coexist, so conflicts are not acceptable.)
In such cases I think the proper way is to tell the build system to use the versioned compiler, e.g. via CC=clang-V CXX=clang++-V (or similar for GCC), and not to twist the symlink. The symlinks should in my view point to the distribution default.
Which some build systems accept, and some don't. Also you can build a package that links all binaries that clang-N provides into /usr/bin/clang-N or whereever and add this to your path. Nonetheless, the alternatives tend to work reasonably well. Thanks Michal
On Mon, Oct 23, 2023 at 1:23 AM Michal Suchánek <msuchanek@suse.de> wrote:
On Thu, Oct 19, 2023 at 12:53:50AM +0200, Aaron Puchert wrote:
Am 18.10.23 um 19:25 schrieb Michal Suchánek:
On Tue, Oct 17, 2023 at 12:40:20AM +0200, Aaron Puchert wrote:
[...] But I should probably migrate [LLVM] from update-alternatives to GCC-style hardcoded symlinks in the metapackage. Please don't.
The hardcoded links are the worst, especially in the cases when binaries are often added/removed. Could you elaborate what the issue would be? What I'm thinking of is having
Am 17.10.23 um 08:38 schrieb Michal Suchánek: the metapackage contain symlinks for all files in /usr/bin of the corresponding versioned package. So if llvm17 has /usr/bin/opt-17 (let's say we go with the major version only), then llvm = 17 would have /usr/bin/opt -> opt-17. Quite similar to GCC actually, except with more binaries (and man pages). And changing what the gcc links point to requires you .. to rebuild a
On Wed, Oct 18, 2023 at 12:59:05AM +0200, Aaron Puchert wrote: package. Pin it to specific revision that points to the version you want, if you find one.
And if you don't adapt the current one to point to a different version .. except that fails with ever-changing list of binaries.
And have one project per gcc version you want to build with, too.
So you're talking about usage in OBS, not in deployment? This is actually what worries me about the update-alternatives mechanism. Some packages want to build against older Clang versions, but use unversioned commands. This works as long as no other package (possibly transitively) requires a newer version for some reason. We've mostly been lucky until now, and the cases that I've seen were the other way around [1], but it's definitely fragile to require a package not to be there. (Just to be clear: we want to allow different compiler versions to coexist, so conflicts are not acceptable.)
In such cases I think the proper way is to tell the build system to use the versioned compiler, e.g. via CC=clang-V CXX=clang++-V (or similar for GCC), and not to twist the symlink. The symlinks should in my view point to the distribution default.
Which some build systems accept, and some don't.
Also you can build a package that links all binaries that clang-N provides into /usr/bin/clang-N or whereever and add this to your path.
Nonetheless, the alternatives tend to work reasonably well.
If it doesn't respect variables, then usually twisting the PATH variable in the build also can fix it. You'd do that by creating symlinks in the %build phase in an arbitrary directory, then prepending that directory to the PATH. It's a bit overkill, though. Fortunately, such a thing has been incredibly rare in my experience. I only encountered one package where that was needed, and I was able to figure out their build script to hack in support for variables instead. -- 真実はいつも一つ！/ Always, there's only one truth!
вівторок, 17 жовтня 2023 р. 01:40:20 EEST Aaron Puchert написано:
Am 29.09.23 um 21:11 schrieb Martin Schreiner via openSUSE Factory:
3 - libalternatives supports user overrides: non-root users can override the alternatives by creating their own local preference file in $HOME/.config/libalternatives.conf. This may be done by invoking "alts" directly, as it serves both purposes. This alone doesn't prohibit symlinks: local overrides could be done via symlinks in ~/bin, and it would still be transparent.
shebang usually looks like #!/usr/bin/ruby #!/usr/bin/python3 ..., etc... ~/bin links wouldn't affect run scripts. But I agree that starting a program which would then run another program is less transparent than symlinks. -- Kind regards, Mykola Krachkovsky -- Найкращі побажання, Микола Крачковський
Am 18.10.23 um 19:21 schrieb Mykola Krachkovsky:
вівторок, 17 жовтня 2023 р. 01:40:20 EEST Aaron Puchert написано:
Am 29.09.23 um 21:11 schrieb Martin Schreiner via openSUSE Factory:
3 - libalternatives supports user overrides: non-root users can override the alternatives by creating their own local preference file in $HOME/.config/libalternatives.conf. This may be done by invoking "alts" directly, as it serves both purposes. This alone doesn't prohibit symlinks: local overrides could be done via symlinks in ~/bin, and it would still be transparent.
shebang usually looks like #!/usr/bin/ruby #!/usr/bin/python3 ..., etc...
~/bin links wouldn't affect run scripts.
Script writers often use `/usr/bin/env <interpreter>`. So if you're just checking out a GitHub repo, it will likely use the interpreter of your choice. Scripts in packages use a hardcoded path, and rpmlint checks for that. But since they live in system directories, they should probably use the system default, not the per-user default. But I realize that this is a point of contention, and others might have a different opinion on that. Aaron
Hello again folks, As we've discussed and shared a considerable amount of information here, I'd like to share a new Wiki page I've set up for us to further discuss the points that were made here. Here's the link to the wiki page: https://en.opensuse.org/openSUSE:Update-alternatives_migration This page aims at bringing clarification and a more consolidated picture to the state and implications regarding update-alternatives and other alternative-like systems/choices. This Wiki page also as a Discussion page: https://en.opensuse.org/openSUSE_talk:Update-alternatives_migration I propose that we take this conversation over there, as this thread has grown long, and it has slowed down somewhat. As we discuss things over there, I'll keep updating the Wiki page until we've come to a conclusion, whatever that may be. Thanks, Martin
participants (16)
-
Aaron Puchert -
Adam Majer -
Andrei Borzenkov -
Christian Boltz -
Dominique Leuenberger -
Felix Niederwanger -
Jan Engelhardt -
Joe Salmeri -
Ludwig Nussel -
Martin Schreiner -
Michal Suchánek -
Mykola Krachkovsky -
Neal Gompa -
Richard Biener -
Simon Lees -
Thorsten Kukuk