Re: The alternative for "update-alternatives" ---> libalternatives
libalternatives is imho a big improvement over update-alternatives. Is there any plan to upstream its support? I think the Open Source ecosystem would greatly benefit from doing so and we wouldn't have to maintain libalternatives support for the packages. Plus, I am a big fan of upstreaming and letting the spec file just run make && make install. Let me know if you need any help regarding this. Regards, Danilo On Mon, 2021-09-20 at 09:19 +0200, Stefan Schubert wrote:
Hi,
most of you are already familiar with the usage of update- alternatives. Update-alternatives, at its core, is a symlink management system. This system is then mostly used as means for administrators of a system to select a default application. For example, if we have vim or emacs installed, the admin can then make one accessible as an editor application. Update-alternatives manages these tasks by symbolic file links which has worked out, that it could become quite instable in older updated systems. Adam Majer has established a new concept how to handle the update-alternatives tasks: libalternatives Libalternatives does not use file links anymore but little configuration files which is much more stable and quite easy to handle by the administrator. Have a look to https://github.com/openSUSE/libalternatives for more information. One additional benefit of libalternatives is that it does not create entries in the /etc directory anymore. In the future the /etc directory will be used for entries/changes ONLY which have been done by the administrator and do not belong to packages or have been created by package installation.
I have already switched about 40 packages to the new libalternatives which we need around MicroOS and have submitted it to factory.
This list includes a lot of python packages. Python packages are using nice RPM macros for handling update-alternatives stuff. Currently the fixes are made manually in the spec files and it will not be done by macros anymore. I have done this step at first in order to get a feeling what is needed for using/updating these macros at first. Now I will patch the alternatives macros or will write new one in order to simplify it again. And yes, I will update the regarding python packages again. So please, do not panic :-)
Please inform us if you have any comments, suggestions or even doubts.
Greetings Stefan
-- ********************************************************************* ********** Stefan Schubert e-mail: schubi@suse.de --------------------------------------------------------------------- ---------- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany
(HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer
-- Best Regards -- Danilo Spinella Software Engineer in Packaging SUSE Software Solutions Italy Srl.
On Tuesday 2021-09-21 09:47, Danilo Spinella wrote:
libalternatives is imho a big improvement over update-alternatives.
Is there any plan to upstream its support?
Upstream to where? Most if not all uses of update-alternatives exist because a *distro* chose to provide two or more implementations with the same name. /etc/alternatives/ctags exists because openSUSE wanted to ship both ctags and emacs-ctags. The emacs project in itself would not, and probably does not, care about the non-emacs-ctags variant in any way, in the scope of the upstream project.
Am 21.09.21 um 09:47 schrieb Danilo Spinella:
libalternatives is imho a big improvement over update-alternatives.
But the question remains: what about performance? If I have a small program that is called in a loop, is performance severely hampered? I.e. how does the following a=`cat <<EOF for ((i=0; i<10000; ++i)); do awk 'BEGIN { print "hallo" }' done EOF` #> time -c "$a" >/dev/null perform with libalternatives in contrast to /usr/bin/gawk?
Is there any plan to upstream its support? I think the Open Source ecosystem would greatly benefit from doing so and we wouldn't have to maintain libalternatives support for the packages. Plus, I am a big fan of upstreaming and letting the spec file just run make && make install.
Let me know if you need any help regarding this.
Regards, Danilo
On Mon, 2021-09-20 at 09:19 +0200, Stefan Schubert wrote:
Hi,
most of you are already familiar with the usage of update- alternatives. Update-alternatives, at its core, is a symlink management system. This system is then mostly used as means for administrators of a system to select a default application. For example, if we have vim or emacs installed, the admin can then make one accessible as an editor application. Update-alternatives manages these tasks by symbolic file links which has worked out, that it could become quite instable in older updated systems. Adam Majer has established a new concept how to handle the update-alternatives tasks: libalternatives Libalternatives does not use file links anymore but little configuration files which is much more stable and quite easy to handle by the administrator. Have a look to https://github.com/openSUSE/libalternatives for more information. One additional benefit of libalternatives is that it does not create entries in the /etc directory anymore. In the future the /etc directory will be used for entries/changes ONLY which have been done by the administrator and do not belong to packages or have been created by package installation.
I have already switched about 40 packages to the new libalternatives which we need around MicroOS and have submitted it to factory.
This list includes a lot of python packages. Python packages are using nice RPM macros for handling update-alternatives stuff. Currently the fixes are made manually in the spec files and it will not be done by macros anymore. I have done this step at first in order to get a feeling what is needed for using/updating these macros at first. Now I will patch the alternatives macros or will write new one in order to simplify it again. And yes, I will update the regarding python packages again. So please, do not panic :-)
Please inform us if you have any comments, suggestions or even doubts.
Greetings Stefan
-- ********************************************************************* ********** Stefan Schubert e-mail: schubi@suse.de --------------------------------------------------------------------- ---------- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany
(HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer
Am 21.09.21 um 10:47 schrieb Manfred Schwarb:
Am 21.09.21 um 09:47 schrieb Danilo Spinella:
libalternatives is imho a big improvement over update-alternatives.
But the question remains: what about performance? If I have a small program that is called in a loop, is performance severely hampered?
I.e. how does the following a=`cat <<EOF for ((i=0; i<10000; ++i)); do awk 'BEGIN { print "hallo" }' done EOF` #> time -c "$a" >/dev/null
time bash -c "$a" >/dev/null of course. Sorry.
perform with libalternatives in contrast to /usr/bin/gawk?
Is there any plan to upstream its support? I think the Open Source ecosystem would greatly benefit from doing so and we wouldn't have to maintain libalternatives support for the packages. Plus, I am a big fan of upstreaming and letting the spec file just run make && make install.
Let me know if you need any help regarding this.
Regards, Danilo
On Mon, 2021-09-20 at 09:19 +0200, Stefan Schubert wrote:
Hi,
most of you are already familiar with the usage of update- alternatives. Update-alternatives, at its core, is a symlink management system. This system is then mostly used as means for administrators of a system to select a default application. For example, if we have vim or emacs installed, the admin can then make one accessible as an editor application. Update-alternatives manages these tasks by symbolic file links which has worked out, that it could become quite instable in older updated systems. Adam Majer has established a new concept how to handle the update-alternatives tasks: libalternatives Libalternatives does not use file links anymore but little configuration files which is much more stable and quite easy to handle by the administrator. Have a look to https://github.com/openSUSE/libalternatives for more information. One additional benefit of libalternatives is that it does not create entries in the /etc directory anymore. In the future the /etc directory will be used for entries/changes ONLY which have been done by the administrator and do not belong to packages or have been created by package installation.
I have already switched about 40 packages to the new libalternatives which we need around MicroOS and have submitted it to factory.
This list includes a lot of python packages. Python packages are using nice RPM macros for handling update-alternatives stuff. Currently the fixes are made manually in the spec files and it will not be done by macros anymore. I have done this step at first in order to get a feeling what is needed for using/updating these macros at first. Now I will patch the alternatives macros or will write new one in order to simplify it again. And yes, I will update the regarding python packages again. So please, do not panic :-)
Please inform us if you have any comments, suggestions or even doubts.
Greetings Stefan
-- ********************************************************************* ********** Stefan Schubert e-mail: schubi@suse.de --------------------------------------------------------------------- ---------- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany
(HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer
On Tue, Sep 21, 2021 at 11:01:46AM +0200, Manfred Schwarb wrote:
Am 21.09.21 um 10:47 schrieb Manfred Schwarb:
But the question remains: what about performance? If I have a small program that is called in a loop, is performance severely hampered?
Your bottleneck will be an additional exec() call. If that is a signifiant overhead in the execution path, then your execution path already had it as a significant overhead. But I would not recommend doing things like abstracting /bin/sh in this manner. Symlinks still have their place. - Adam
Am 22.09.21 um 10:18 schrieb Adam Majer:
On Tue, Sep 21, 2021 at 11:01:46AM +0200, Manfred Schwarb wrote:
Am 21.09.21 um 10:47 schrieb Manfred Schwarb:
But the question remains: what about performance? If I have a small program that is called in a loop, is performance severely hampered?
Your bottleneck will be an additional exec() call. If that is a signifiant overhead in the execution path, then your execution path already had it as a significant overhead.
But I would not recommend doing things like abstracting /bin/sh in this manner. Symlinks still have their place.
OK, then I misunderstood something. I thought the intention was to replace all alternatives handling with libalternatives, i.e. also for potentially often called binaries as ksh or awk.
- Adam
On Wed, Sep 22, 2021 at 10:52:01AM +0200, Manfred Schwarb wrote:
Am 22.09.21 um 10:18 schrieb Adam Majer:
On Tue, Sep 21, 2021 at 11:01:46AM +0200, Manfred Schwarb wrote:
Am 21.09.21 um 10:47 schrieb Manfred Schwarb:
But the question remains: what about performance? If I have a small program that is called in a loop, is performance severely hampered?
Your bottleneck will be an additional exec() call. If that is a signifiant overhead in the execution path, then your execution path already had it as a significant overhead.
But I would not recommend doing things like abstracting /bin/sh in this manner. Symlinks still have their place.
OK, then I misunderstood something. I thought the intention was to replace all alternatives handling with libalternatives, i.e. also for potentially often called binaries as ksh or awk.
Which then brings the question why do it at all? Having one altertnatives system is better than two. Thanks Michal
On Thu, 2021-09-23 at 11:14 +0200, Michal Suchánek wrote:
OK, then I misunderstood something. I thought the intention was to replace all alternatives handling with libalternatives, i.e. also for potentially often called binaries as ksh or awk.
Which then brings the question why do it at all?
Having one altertnatives system is better than two.
Right. "alternatives --set alternatives libalternatives" sounds like genuine fun :-) But using one alternatives system for one set of options and the other for others sounds even crazier. Martin
On Thu, Sep 23, Michal Suchánek wrote:
On Wed, Sep 22, 2021 at 10:52:01AM +0200, Manfred Schwarb wrote:
Am 22.09.21 um 10:18 schrieb Adam Majer:
On Tue, Sep 21, 2021 at 11:01:46AM +0200, Manfred Schwarb wrote:
Am 21.09.21 um 10:47 schrieb Manfred Schwarb:
But the question remains: what about performance? If I have a small program that is called in a loop, is performance severely hampered?
Your bottleneck will be an additional exec() call. If that is a signifiant overhead in the execution path, then your execution path already had it as a significant overhead.
But I would not recommend doing things like abstracting /bin/sh in this manner. Symlinks still have their place.
OK, then I misunderstood something. I thought the intention was to replace all alternatives handling with libalternatives, i.e. also for potentially often called binaries as ksh or awk.
Yes, the intention is to replace update-alternatives. But not in all usages, that's impossible to do. For some problems we need other solutions, like for /bin/sh. And the funny part is: the people who wanted update-alternatives for /bin/sh in bash did never adjust their own packages, so that this was a completly useless exercise creating a lot of work, problems and broken systems :(
Which then brings the question why do it at all?
Because it solves many problems we face with update-alternatives. The /etc and /var usage by update-alternatives is e.g. a nightmare if you do image based deployments/updates (with images I mean images of the system binaries, not full disk images). The second worse project creating as massive problems is SELinux.
Having one altertnatives system is better than two.
Correct, and mid- to long term I hope we can drop update-alternatives :) Thorsten
Thanks
Michal
-- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Felix Imendoerffer (HRB 36809, AG Nürnberg)
On 2021/09/23 13:52:04 +0200, Thorsten Kukuk wrote:
On Thu, Sep 23, Michal Suchánek wrote:
On Wed, Sep 22, 2021 at 10:52:01AM +0200, Manfred Schwarb wrote:
Am 22.09.21 um 10:18 schrieb Adam Majer:
On Tue, Sep 21, 2021 at 11:01:46AM +0200, Manfred Schwarb wrote:
Am 21.09.21 um 10:47 schrieb Manfred Schwarb:
But the question remains: what about performance? If I have a small program that is called in a loop, is performance severely hampered?
Your bottleneck will be an additional exec() call. If that is a signifiant overhead in the execution path, then your execution path already had it as a significant overhead.
But I would not recommend doing things like abstracting /bin/sh in this manner. Symlinks still have their place.
OK, then I misunderstood something. I thought the intention was to replace all alternatives handling with libalternatives, i.e. also for potentially often called binaries as ksh or awk.
Yes, the intention is to replace update-alternatives. But not in all usages, that's impossible to do. For some problems we need other solutions, like for /bin/sh.
As I had a request for package man I've open an issue https://github.com/openSUSE/libalternatives/issues/9 as this bottleneck has to be resolved not only for the standard bourne shell.
And the funny part is: the people who wanted update-alternatives for /bin/sh in bash did never adjust their own packages, so that this was a completly useless exercise creating a lot of work, problems and broken systems :(
Which then brings the question why do it at all?
Because it solves many problems we face with update-alternatives. The /etc and /var usage by update-alternatives is e.g. a nightmare if you do image based deployments/updates (with images I mean images of the system binaries, not full disk images). The second worse project creating as massive problems is SELinux.
Having one altertnatives system is better than two.
Correct, and mid- to long term I hope we can drop update-alternatives :)
Thorsten
Werner -- "Having a smoking section in a restaurant is like having a peeing section in a swimming pool." -- Edward Burr
Thorsten Kukuk wrote:
On Thu, Sep 23, Michal Suchánek wrote:
On Wed, Sep 22, 2021 at 10:52:01AM +0200, Manfred Schwarb wrote:
Am 22.09.21 um 10:18 schrieb Adam Majer:
On Tue, Sep 21, 2021 at 11:01:46AM +0200, Manfred Schwarb wrote:
Am 21.09.21 um 10:47 schrieb Manfred Schwarb:
But the question remains: what about performance? If I have a small program that is called in a loop, is performance severely hampered?
Your bottleneck will be an additional exec() call. If that is a signifiant overhead in the execution path, then your execution path already had it as a significant overhead.
But I would not recommend doing things like abstracting /bin/sh in this manner. Symlinks still have their place.
OK, then I misunderstood something. I thought the intention was to replace all alternatives handling with libalternatives, i.e. also for potentially often called binaries as ksh or awk.
Yes, the intention is to replace update-alternatives. But not in all usages, that's impossible to do. For some problems we need other solutions, like for /bin/sh.
IMO we should neither use libalternatives nor update-alternatives for such core tools like /bin/sh. If there's an actual use case for building systems that need different implementations we can still solve that with conflicting packages. In case of /bin/sh I think container images by default use busybox as /bin/sh. Some workloads still also need bash though. Regular openSUSE installations always use bash. So my suggestion would be to - package "bash" without /usr/bin/sh - subpackage "bash-sh" that has a symlink /usr/bin/sh pointing to bash - "busbyox-sh" that has /usr/bin/sh pointing to busybox - both "bash-sh" and "busybox-sh" need to conflict and provide some common tag then. Maybe alternative(sh)? - patterns-base-base which requires "bash" would has to require "bash-sh" too. cu Ludwig -- (o_ Ludwig Nussel //\ V_/_ http://www.suse.com/ SUSE Software Solutions Germany GmbH, GF: Felix Imendörffer HRB 36809 (AG Nürnberg)
On 9/24/21 00:23, Ludwig Nussel wrote:
Thorsten Kukuk wrote:
On Thu, Sep 23, Michal Suchánek wrote:
On Wed, Sep 22, 2021 at 10:52:01AM +0200, Manfred Schwarb wrote:
Am 22.09.21 um 10:18 schrieb Adam Majer:
On Tue, Sep 21, 2021 at 11:01:46AM +0200, Manfred Schwarb wrote:
Am 21.09.21 um 10:47 schrieb Manfred Schwarb: > But the question remains: what about performance? > If I have a small program that is called in a loop, is performance severely hampered?
Your bottleneck will be an additional exec() call. If that is a signifiant overhead in the execution path, then your execution path already had it as a significant overhead.
But I would not recommend doing things like abstracting /bin/sh in this manner. Symlinks still have their place.
OK, then I misunderstood something. I thought the intention was to replace all alternatives handling with libalternatives, i.e. also for potentially often called binaries as ksh or awk.
Yes, the intention is to replace update-alternatives. But not in all usages, that's impossible to do. For some problems we need other solutions, like for /bin/sh.
IMO we should neither use libalternatives nor update-alternatives for such core tools like /bin/sh. If there's an actual use case for building systems that need different implementations we can still solve that with conflicting packages. In case of /bin/sh I think container images by default use busybox as /bin/sh. Some workloads still also need bash though. Regular openSUSE installations always use bash. So my suggestion would be to - package "bash" without /usr/bin/sh - subpackage "bash-sh" that has a symlink /usr/bin/sh pointing to bash - "busbyox-sh" that has /usr/bin/sh pointing to busybox - both "bash-sh" and "busybox-sh" need to conflict and provide some common tag then. Maybe alternative(sh)? - patterns-base-base which requires "bash" would has to require "bash-sh" too.
I'd probably go for patterns-base-base requiring the common tag and suggesting bash-sh which should have the same effect and won't cause conflicts with the pattern if someone wants to try something else. I also really like this idea, my interest in this concept comes from speeding up the build of slower packages, for these packages make often spawns alot of instances of /bin/sh so being able to BuildRequire: dash-sh on say a kernel spec file to first check it still builds and then see if the performance benefits are worth keeping. Cheers Simon -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
Hello, On Wed, 22 Sep 2021, Adam Majer wrote:
On Tue, Sep 21, 2021 at 11:01:46AM +0200, Manfred Schwarb wrote:
Am 21.09.21 um 10:47 schrieb Manfred Schwarb:
But the question remains: what about performance? If I have a small program that is called in a loop, is performance severely hampered?
Your bottleneck will be an additional exec() call.
Nope. It execve's a specific binary and to find out which it needs to open, read and parse all the associated config files on each execution. That might still not pose a significant problem, but that needs to be measured instead of just be hoped upon. (Generally I do like the libalternatives approach, but maybe not for /bin/sh, as you say). Ciao, Michael.
On Wed, Sep 22, 2021 at 01:31:11PM +0000, Michael Matz wrote:
Hello,
On Wed, 22 Sep 2021, Adam Majer wrote:
On Tue, Sep 21, 2021 at 11:01:46AM +0200, Manfred Schwarb wrote:
Am 21.09.21 um 10:47 schrieb Manfred Schwarb:
But the question remains: what about performance? If I have a small program that is called in a loop, is performance severely hampered?
Your bottleneck will be an additional exec() call.
Nope. It execve's a specific binary and to find out which it needs to open, read and parse all the associated config files on each execution.
That might still not pose a significant problem, but that needs to be measured instead of just be hoped upon. (Generally I do like the libalternatives approach, but maybe not for /bin/sh, as you say).
Yes, I believe that it's a little heavy for /bin/sh because it happens on *every* execution. This adds up if you put it on 1m servers and run them for a year. And there is no appreciable benefit for doing this (at least I can't imagine one). But in general, the overhead seems to be the exec call. I just did a small benchmark that just execs /bin/true either directly or via /bin/alts and it's as expected. https://gist.github.com/AdamMajer/89e1402d26024f5ebddb25d19d2f0890 The main benefit for libalternatives over update-alternatives is that each user can override the default. Another is that the selection is not fragile -- you can snapshots of / and restore /etc and /home from a different time and things will not fall apart like with update-alternatives. The negative is mostly that extra exec() overhead (and if you ever find it's significantly more in some scenario, it's a bug :). Another issue is it's not immediately clear *what* you are running since execution depends no the argv[0] and config file and not target binary of a symlink. This is an issue with things like AppArmor. - Adam
On Fri, Sep 24, 2021 at 02:02:21PM +0200, Adam Majer wrote:
On Wed, Sep 22, 2021 at 01:31:11PM +0000, Michael Matz wrote:
Hello,
On Wed, 22 Sep 2021, Adam Majer wrote:
On Tue, Sep 21, 2021 at 11:01:46AM +0200, Manfred Schwarb wrote:
Am 21.09.21 um 10:47 schrieb Manfred Schwarb:
But the question remains: what about performance? If I have a small program that is called in a loop, is performance severely hampered?
Your bottleneck will be an additional exec() call.
Nope. It execve's a specific binary and to find out which it needs to open, read and parse all the associated config files on each execution.
That might still not pose a significant problem, but that needs to be measured instead of just be hoped upon. (Generally I do like the libalternatives approach, but maybe not for /bin/sh, as you say).
Yes, I believe that it's a little heavy for /bin/sh because it happens on *every* execution. This adds up if you put it on 1m servers and run them for a year. And there is no appreciable benefit for doing this (at least I can't imagine one).
But in general, the overhead seems to be the exec call. I just did a small benchmark that just execs /bin/true either directly or via /bin/alts and it's as expected.
https://gist.github.com/AdamMajer/89e1402d26024f5ebddb25d19d2f0890
The main benefit for libalternatives over update-alternatives is that each user can override the default. Another is that the selection is not fragile -- you can snapshots of / and restore /etc and /home from a different time and things will not fall apart like with update-alternatives.
The negative is mostly that extra exec() overhead (and if you ever find it's significantly more in some scenario, it's a bug :). Another issue is it's not immediately clear *what* you are running since execution depends no the argv[0] and config file and not target binary of a symlink. This is an issue with things like AppArmor.
How is it a problem with apparmor? If it limits exec() it will just happen one step later when libalternative does it. Or am I missing something? Thanks Michal
On 24.09.2021 16:28, Michal Suchánek wrote:
The negative is mostly that extra exec() overhead (and if you ever find it's significantly more in some scenario, it's a bug :). Another issue is it's not immediately clear *what* you are running since execution depends no the argv[0] and config file and not target binary of a symlink. This is an issue with things like AppArmor.
How is it a problem with apparmor?
If it limits exec() it will just happen one step later when libalternative does it.
Or am I missing something?
Apparmor needs to allow every possible alternative binary instead of just one canonical path.
On 24.09.21 14:02, Adam Majer wrote:
each user can override the default. Another is that the selection is not fragile -- you can snapshots of / and restore /etc and /home from a different time and things will not fall apart like with update-alternatives.
That's just a proof that the current implementation of update-alternatives is exceptionally -- ahem -- suboptimal. I never understood what these symlinks to /etc/alternatives and then again to the target are good for. Why not just link the target directly. The update-alternatives implementation in the opkg tool (at least a few years ago, when I looked last) which is used on embedded systems for package management was much more straightforward for example, even if it was implemented in plain shell which made it slow. -- Stefan Seyfried "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." -- Richard Feynman
On Fri, Sep 24, 2021 at 04:30:04PM +0200, Stefan Seyfried wrote:
On 24.09.21 14:02, Adam Majer wrote:
each user can override the default. Another is that the selection is not fragile -- you can snapshots of / and restore /etc and /home from a different time and things will not fall apart like with update-alternatives.
That's just a proof that the current implementation of update-alternatives is exceptionally -- ahem -- suboptimal. I never understood what these symlinks to /etc/alternatives and then again to the target are good for. Why not just link the target directly.
With double links you can update the alternatives with readonly /usr for example. The symlink that makes the binary an alternative is static, only the alternative configuration is dynamic. It follows from FHS also. Thanks Michal
Am 21.09.21 um 10:47 schrieb Manfred Schwarb:
Am 21.09.21 um 09:47 schrieb Danilo Spinella:
libalternatives is imho a big improvement over update-alternatives.
But the question remains: what about performance? If I have a small program that is called in a loop, is performance severely hampered?
I.e. how does the following a=`cat <<EOF for ((i=0; i<10000; ++i)); do awk 'BEGIN { print "hallo" }' done EOF` #> time -c "$a" >/dev/null
perform with libalternatives in contrast to /usr/bin/gawk?
BTW: Is mawk really considered a full-fledged alternative to gawk? mawk has not got any real feature development since 2010, and is only kept alive since then by Tom Dickey. It misses utf8 support as far as I know. So I would vote to rip out alternative support from gawk and mawk, declaring gawk as the default "awk" for openSUSE. I would even volounteer to write SR's to do so, if it is deemed viable/acceptable. Thoughts?
Is there any plan to upstream its support? I think the Open Source ecosystem would greatly benefit from doing so and we wouldn't have to maintain libalternatives support for the packages. Plus, I am a big fan of upstreaming and letting the spec file just run make && make install.
Let me know if you need any help regarding this.
Regards, Danilo
On Mon, 2021-09-20 at 09:19 +0200, Stefan Schubert wrote:
Hi,
most of you are already familiar with the usage of update- alternatives. Update-alternatives, at its core, is a symlink management system. This system is then mostly used as means for administrators of a system to select a default application. For example, if we have vim or emacs installed, the admin can then make one accessible as an editor application. Update-alternatives manages these tasks by symbolic file links which has worked out, that it could become quite instable in older updated systems. Adam Majer has established a new concept how to handle the update-alternatives tasks: libalternatives Libalternatives does not use file links anymore but little configuration files which is much more stable and quite easy to handle by the administrator. Have a look to https://github.com/openSUSE/libalternatives for more information. One additional benefit of libalternatives is that it does not create entries in the /etc directory anymore. In the future the /etc directory will be used for entries/changes ONLY which have been done by the administrator and do not belong to packages or have been created by package installation.
I have already switched about 40 packages to the new libalternatives which we need around MicroOS and have submitted it to factory.
This list includes a lot of python packages. Python packages are using nice RPM macros for handling update-alternatives stuff. Currently the fixes are made manually in the spec files and it will not be done by macros anymore. I have done this step at first in order to get a feeling what is needed for using/updating these macros at first. Now I will patch the alternatives macros or will write new one in order to simplify it again. And yes, I will update the regarding python packages again. So please, do not panic :-)
Please inform us if you have any comments, suggestions or even doubts.
Greetings Stefan
-- ********************************************************************* ********** Stefan Schubert e-mail: schubi@suse.de --------------------------------------------------------------------- ---------- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany
(HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer
On Thu, Sep 23, Manfred Schwarb wrote:
Am 21.09.21 um 10:47 schrieb Manfred Schwarb:
Am 21.09.21 um 09:47 schrieb Danilo Spinella:
libalternatives is imho a big improvement over update-alternatives.
But the question remains: what about performance? If I have a small program that is called in a loop, is performance severely hampered?
I.e. how does the following a=`cat <<EOF for ((i=0; i<10000; ++i)); do awk 'BEGIN { print "hallo" }' done EOF` #> time -c "$a" >/dev/null
perform with libalternatives in contrast to /usr/bin/gawk?
BTW: Is mawk really considered a full-fledged alternative to gawk? mawk has not got any real feature development since 2010, and is only kept alive since then by Tom Dickey. It misses utf8 support as far as I know.
So I would vote to rip out alternative support from gawk and mawk, declaring gawk as the default "awk" for openSUSE. I would even volounteer to write SR's to do so, if it is deemed viable/acceptable.
Thoughts?
I'm not sure if mawk is the only alternative, but if yes, I would say go ahead. Thorsten
Is there any plan to upstream its support? I think the Open Source ecosystem would greatly benefit from doing so and we wouldn't have to maintain libalternatives support for the packages. Plus, I am a big fan of upstreaming and letting the spec file just run make && make install.
Let me know if you need any help regarding this.
Regards, Danilo
On Mon, 2021-09-20 at 09:19 +0200, Stefan Schubert wrote:
Hi,
most of you are already familiar with the usage of update- alternatives. Update-alternatives, at its core, is a symlink management system. This system is then mostly used as means for administrators of a system to select a default application. For example, if we have vim or emacs installed, the admin can then make one accessible as an editor application. Update-alternatives manages these tasks by symbolic file links which has worked out, that it could become quite instable in older updated systems. Adam Majer has established a new concept how to handle the update-alternatives tasks: libalternatives Libalternatives does not use file links anymore but little configuration files which is much more stable and quite easy to handle by the administrator. Have a look to https://github.com/openSUSE/libalternatives for more information. One additional benefit of libalternatives is that it does not create entries in the /etc directory anymore. In the future the /etc directory will be used for entries/changes ONLY which have been done by the administrator and do not belong to packages or have been created by package installation.
I have already switched about 40 packages to the new libalternatives which we need around MicroOS and have submitted it to factory.
This list includes a lot of python packages. Python packages are using nice RPM macros for handling update-alternatives stuff. Currently the fixes are made manually in the spec files and it will not be done by macros anymore. I have done this step at first in order to get a feeling what is needed for using/updating these macros at first. Now I will patch the alternatives macros or will write new one in order to simplify it again. And yes, I will update the regarding python packages again. So please, do not panic :-)
Please inform us if you have any comments, suggestions or even doubts.
Greetings Stefan
-- ********************************************************************* ********** Stefan Schubert e-mail: schubi@suse.de --------------------------------------------------------------------- ---------- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany
(HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer
-- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Felix Imendoerffer (HRB 36809, AG Nürnberg)
Hello, On Thu, Sep 23, 2021 at 06:52:54AM +0200, Thorsten Kukuk wrote:
On Thu, Sep 23, Manfred Schwarb wrote:
Am 21.09.21 um 10:47 schrieb Manfred Schwarb:
Am 21.09.21 um 09:47 schrieb Danilo Spinella:
libalternatives is imho a big improvement over update-alternatives.
But the question remains: what about performance? If I have a small program that is called in a loop, is performance severely hampered?
I.e. how does the following a=`cat <<EOF for ((i=0; i<10000; ++i)); do awk 'BEGIN { print "hallo" }' done EOF` #> time -c "$a" >/dev/null
perform with libalternatives in contrast to /usr/bin/gawk?
BTW: Is mawk really considered a full-fledged alternative to gawk? mawk has not got any real feature development since 2010, and is only kept alive since then by Tom Dickey. It misses utf8 support as far as I know.
So I would vote to rip out alternative support from gawk and mawk, declaring gawk as the default "awk" for openSUSE. I would even volounteer to write SR's to do so, if it is deemed viable/acceptable.
Thoughts?
I'm not sure if mawk is the only alternative, but if yes, I would say go ahead.
The problem is that gawk and mawk are mutually incompatible, and some software expects non-GNU awk semantics and assumes /usr/bin/awk is traditional awk. Of course, it should provide a way to set which awk to use but many develoipers think that the awk flavor they use is the only true awk and fishing it from non-standard location is not needed. Then the alternatives are the workaround for such projects. Thanks Michal
Thorsten
Is there any plan to upstream its support? I think the Open Source ecosystem would greatly benefit from doing so and we wouldn't have to maintain libalternatives support for the packages. Plus, I am a big fan of upstreaming and letting the spec file just run make && make install.
Let me know if you need any help regarding this.
Regards, Danilo
On Mon, 2021-09-20 at 09:19 +0200, Stefan Schubert wrote:
Hi,
most of you are already familiar with the usage of update- alternatives. Update-alternatives, at its core, is a symlink management system. This system is then mostly used as means for administrators of a system to select a default application. For example, if we have vim or emacs installed, the admin can then make one accessible as an editor application. Update-alternatives manages these tasks by symbolic file links which has worked out, that it could become quite instable in older updated systems. Adam Majer has established a new concept how to handle the update-alternatives tasks: libalternatives Libalternatives does not use file links anymore but little configuration files which is much more stable and quite easy to handle by the administrator. Have a look to https://github.com/openSUSE/libalternatives for more information. One additional benefit of libalternatives is that it does not create entries in the /etc directory anymore. In the future the /etc directory will be used for entries/changes ONLY which have been done by the administrator and do not belong to packages or have been created by package installation.
I have already switched about 40 packages to the new libalternatives which we need around MicroOS and have submitted it to factory.
This list includes a lot of python packages. Python packages are using nice RPM macros for handling update-alternatives stuff. Currently the fixes are made manually in the spec files and it will not be done by macros anymore. I have done this step at first in order to get a feeling what is needed for using/updating these macros at first. Now I will patch the alternatives macros or will write new one in order to simplify it again. And yes, I will update the regarding python packages again. So please, do not panic :-)
Please inform us if you have any comments, suggestions or even doubts.
Greetings Stefan
-- ********************************************************************* ********** Stefan Schubert e-mail: schubi@suse.de --------------------------------------------------------------------- ---------- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany
(HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer
-- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Felix Imendoerffer (HRB 36809, AG Nürnberg)
On Thu, Sep 23, Michal Suchánek wrote:
The problem is that gawk and mawk are mutually incompatible, and some software expects non-GNU awk semantics and assumes /usr/bin/awk is traditional awk.
Of course, it should provide a way to set which awk to use but many develoipers think that the awk flavor they use is the only true awk and fishing it from non-standard location is not needed.
Then the alternatives are the workaround for such projects.
But we have much more packages which expects that /usr/bin/awk is gawk instead of anything else. So in the end you would break other packages. If a package requires a specific awk implementation (independent of gawk or mawk), it's much better to patch the package to explicit use gawk or mawk. Thorsten
Thanks
Michal
Thorsten
Is there any plan to upstream its support? I think the Open Source ecosystem would greatly benefit from doing so and we wouldn't have to maintain libalternatives support for the packages. Plus, I am a big fan of upstreaming and letting the spec file just run make && make install.
Let me know if you need any help regarding this.
Regards, Danilo
On Mon, 2021-09-20 at 09:19 +0200, Stefan Schubert wrote:
Hi,
most of you are already familiar with the usage of update- alternatives. Update-alternatives, at its core, is a symlink management system. This system is then mostly used as means for administrators of a system to select a default application. For example, if we have vim or emacs installed, the admin can then make one accessible as an editor application. Update-alternatives manages these tasks by symbolic file links which has worked out, that it could become quite instable in older updated systems. Adam Majer has established a new concept how to handle the update-alternatives tasks: libalternatives Libalternatives does not use file links anymore but little configuration files which is much more stable and quite easy to handle by the administrator. Have a look to https://github.com/openSUSE/libalternatives for more information. One additional benefit of libalternatives is that it does not create entries in the /etc directory anymore. In the future the /etc directory will be used for entries/changes ONLY which have been done by the administrator and do not belong to packages or have been created by package installation.
I have already switched about 40 packages to the new libalternatives which we need around MicroOS and have submitted it to factory.
This list includes a lot of python packages. Python packages are using nice RPM macros for handling update-alternatives stuff. Currently the fixes are made manually in the spec files and it will not be done by macros anymore. I have done this step at first in order to get a feeling what is needed for using/updating these macros at first. Now I will patch the alternatives macros or will write new one in order to simplify it again. And yes, I will update the regarding python packages again. So please, do not panic :-)
Please inform us if you have any comments, suggestions or even doubts.
Greetings Stefan
-- ********************************************************************* ********** Stefan Schubert e-mail: schubi@suse.de --------------------------------------------------------------------- ---------- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 90409 Nürnberg Germany
(HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer
-- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Felix Imendoerffer (HRB 36809, AG Nürnberg)
-- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Felix Imendoerffer (HRB 36809, AG Nürnberg)
Hello, On 2021-09-23 12:10, Thorsten Kukuk wrote:
On Thu, Sep 23, Michal Suchánek wrote:
The problem is that gawk and mawk are mutually incompatible, and some software expects non-GNU awk semantics and assumes /usr/bin/awk is traditional awk.
Of course, it should provide a way to set which awk to use but many develoipers think that the awk flavor they use is the only true awk and fishing it from non-standard location is not needed.
Then the alternatives are the workaround for such projects.
But we have much more packages which expects that /usr/bin/awk is gawk instead of anything else. So in the end you would break other packages. If a package requires a specific awk implementation (independent of gawk or mawk), it's much better to patch the package to explicit use gawk or mawk.
In general if "fool" needs a specific "tool" but "fool" only calls for anything named "tool" it is clear where the fault is. I think we should not provide workarounds for faults because that will never work reliably and even worse: When such a workaround exists "fools" will rely on it and may even demand that it is the responsibility of the workaround to make "foolish" things work ;-) Kind Regards Johannes Meixner -- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 - 90409 Nuernberg - Germany (HRB 36809, AG Nuernberg) GF: Felix Imendoerffer
On Thu, Sep 23, 2021 at 4:11 AM Thorsten Kukuk <kukuk@suse.de> wrote:
On Thu, Sep 23, Michal Suchánek wrote:
The problem is that gawk and mawk are mutually incompatible, and some software expects non-GNU awk semantics and assumes /usr/bin/awk is traditional awk.
Of course, it should provide a way to set which awk to use but many develoipers think that the awk flavor they use is the only true awk and fishing it from non-standard location is not needed.
Then the alternatives are the workaround for such projects.
But we have much more packages which expects that /usr/bin/awk is gawk instead of anything else. So in the end you would break other packages. If a package requires a specific awk implementation (independent of gawk or mawk), it's much better to patch the package to explicit use gawk or mawk.
There is also the original awk aka nawk (see https://github.com/onetrueawk/awk). Christoph
Thorsten
Thanks
Michal
Thorsten
Is there any plan to upstream its support? I think the Open Source ecosystem would greatly benefit from doing so and we wouldn't have to maintain libalternatives support for the packages. Plus, I am a big fan of upstreaming and letting the spec file just run make && make install.
Let me know if you need any help regarding this.
Regards, Danilo
On Mon, 2021-09-20 at 09:19 +0200, Stefan Schubert wrote: > Hi, > > most of you are already familiar with the usage of update- > alternatives. > Update-alternatives, at its core, is a symlink management system. > This > system is then mostly used as means for administrators of a system to > select a default application. > For example, if we have vim or emacs installed, the admin can then > make > one accessible as an editor application. > Update-alternatives manages these tasks by symbolic file links which > has > worked out, that it could become quite instable in older updated > systems. > Adam Majer has established a new concept how to handle the > update-alternatives tasks: libalternatives > Libalternatives does not use file links anymore but little > configuration > files which is much more stable and quite easy to handle by the > administrator. > Have a look to https://github.com/openSUSE/libalternatives for more > information. > One additional benefit of libalternatives is that it does not create > entries in the /etc directory anymore. In the future the /etc > directory > will be used for entries/changes ONLY which > have been done by the administrator and do not belong to packages or > have been created by package installation. > > I have already switched about 40 packages to the new libalternatives > which we need around MicroOS and have submitted it to factory. > > This list includes a lot of python packages. Python packages are > using > nice RPM macros for handling update-alternatives stuff. > Currently the fixes are made manually in the spec files and it will > not > be done by macros anymore. > I have done this step at first in order to get a feeling what is > needed > for using/updating these macros at first. > Now I will patch the alternatives macros or will write new one in > order > to simplify it again. > And yes, I will update the regarding python packages again. So > please, > do not panic :-) > > Please inform us if you have any comments, suggestions or even > doubts. > > Greetings > Stefan > > -- > ********************************************************************* > ********** > Stefan Schubert > e-mail: schubi@suse.de > --------------------------------------------------------------------- > ---------- > SUSE Software Solutions Germany GmbH > Maxfeldstr. 5 > 90409 Nürnberg > Germany > > (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer
-- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Felix Imendoerffer (HRB 36809, AG Nürnberg)
-- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Felix Imendoerffer (HRB 36809, AG Nürnberg)
-- Christoph Junghans Web: http://www.compphys.de
Am 24.09.21 um 01:16 schrieb Christoph Junghans:
On Thu, Sep 23, 2021 at 4:11 AM Thorsten Kukuk <kukuk@suse.de> wrote:
On Thu, Sep 23, Michal Suchánek wrote:
The problem is that gawk and mawk are mutually incompatible, and some software expects non-GNU awk semantics and assumes /usr/bin/awk is traditional awk.
Of course, it should provide a way to set which awk to use but many develoipers think that the awk flavor they use is the only true awk and fishing it from non-standard location is not needed.
Then the alternatives are the workaround for such projects.
But we have much more packages which expects that /usr/bin/awk is gawk instead of anything else. So in the end you would break other packages. If a package requires a specific awk implementation (independent of gawk or mawk), it's much better to patch the package to explicit use gawk or mawk.
There is also the original awk aka nawk (see https://github.com/onetrueawk/awk).
Yes, but for openSUSE, the otawk package is found only in the utilities project and has no update-alternatives support so far. I have put 2 SRs to remove update-alternatives support for awk: https://build.opensuse.org/request/show/921253 https://build.opensuse.org/request/show/921254 The spec files get quite a bit prettier this way! Let's see how this works out... Cheers, Manfred
Christoph
Thorsten
Thanks
Michal
Thorsten
participants (15)
-
Adam Majer -
Andrei Borzenkov -
Christoph Junghans -
Danilo Spinella -
Dr. Werner Fink -
Jan Engelhardt -
jsmeix -
Ludwig Nussel -
Manfred Schwarb -
Martin Wilck -
Michael Matz -
Michal Suchánek -
Simon Lees -
Stefan Seyfried -
Thorsten Kukuk