Dash (debian shell) as /bin/sh
I noticed this being added a few weeks ago: https://build.opensuse.org/package/rdiff/openSUSE:Factory/dash?linkrev=base&... Anyone knows how supported is this / when is it going to get switched on by default?
On Thu, Feb 10, dziobian wrote:
It is as supported as anthing else in Tumbleweed. Using dash as default doesn't make any sense, this would only mean one more shell beside bash and increasing the system without benefit You can use it already today if you want e.g. a bash free system and if there are no other requirements for bash. So clearly special cases, not default. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Ivo Totev (HRB 36809, AG Nürnberg)
On 2/10/22 09:07, Thorsten Kukuk wrote:
Using dash as the default shell is meant to help avoid bashisms, i.e. bash-specific shell code which would break on other shells if a user decided to replace bash as their default shell. Defaulting to dash for system scripts is actually a pretty sensible thing to do. Adrian
On 2/10/22 18:41, John Paul Adrian Glaubitz wrote:
Another place where it would likely be useful is it may well provide a reasonable speed up in building packages that still use traditional makefiles. THis benefit probably doesn't off set the amount of effort to fix all the broken things, doing some experiments with BuildRequires: dash-sh on longer building packages is probably worth the effort though. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
Hello, On 2022-02-10 09:11, John Paul Adrian Glaubitz wrote:
Is "#!/bin/bash" no longer supported? How does using a non-bash shell avoid bashisms? I think using a non-bash shell causes errors at the user for scripts that use bashisms without proper shebang i.e. end users get errors for faults of others. Not really end user friendly. But strictly technically right - of course - yeah! Kind Regards Johannes Meixner -- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 - 90409 Nuernberg - Germany (HRB 36809, AG Nuernberg) GF: Ivo Totev
On Thu, Feb 10, Johannes Meixner wrote:
Of course it is. There are many features of bash which do not exist in a POSIX conform sh and for which is no replacement available. But you have to use "#!/bin/bash" if you use such features, using "#!/bin/sh" and expect that bash features work is plain wrong.
How does using a non-bash shell avoid bashisms?
Because your script stops working until you fix it to either: - use /bin/bash - remove the bashism.
Depends on from which angle you are looking at this. There are enough cases where you don't have bash available and you have to use a POSIX shell, even for Tumbleweed. For this users (which are not the Tumbleweed Desktop users, I agree), it is really helpful and very user friendly. Thorsten
-- Thorsten Kukuk, Distinguished Engineer, Senior Architect SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Ivo Totev (HRB 36809, AG Nürnberg)
On 2022/02/10 10:29:42 +0100, Johannes Meixner wrote:
AFAIK /bin == /usr/bin on Tumbleweed, damit gibt es eine /bin/bash Werner -- "Having a smoking section in a restaurant is like having a peeing section in a swimming pool." -- Edward Burr
Hi Johannes,
Is "#!/bin/bash" no longer supported? How does using a non-bash shell avoid bashisms?
scripts that use bash extensions need to call themselves via bash. there is no constraint on using /bin/bash currently. There are projects and use-cases where size matters, and bash is relatively large. being able to know which package sets can be composed to run with e.g. busybox or dash (which is significantly smaller than bash) and without bash is useful knowledge to have. in order to know that somebody needs to do the work in assessing how bad it is. thats what I'm doing. Greetings, Dirk
Hello, On Thu, Feb 10, 2022 at 08:29:27PM +0100, Dirk Müller wrote:
I used to use Debian a lot which allows a bunch of shells that claim to be Bourne-compatible as /bin/sh. The problem with this that no Bourne-compatible nor POSIX-compatible shell exists. All have bugs that make them slightly non-compliant. Each one different bugs. The development of none of the shell projects is hyperfast at fixing bugs so at any given time all the shells are buggy. These bugs are conserved in historical releases of the distribution. Recall how web developers jump through hoops to detect which specific version of which browser you are running so they can craft the page to the specific deficient subset of HTML your browser supports? It becomes a lot like that when you are writing shell code that's supposed to run on any shell. So I would prefer if only one shell was used for /bin/sh with canoniocal set of features and bugs. Thanks Michal
On 2022-02-10 20:53, Michal Suchánek wrote:
So I would prefer if only one shell was used for /bin/sh with canoniocal set of features and bugs.
Why not specify which exact shell to use, instead? If a script has been tested with dash, just write /bin/dash. :-? -- Cheers / Saludos, Carlos E. R. (from 15.3 x86_64 at Telcontar)
On Thu, 2022-02-10 at 09:07 +0100, Thorsten Kukuk wrote:
One user could use dash because it's significantly faster than bash, so having it as default shell will have a faster boot and/or system theorically. In practice the difference is negligible, assuming dash being 1~2s faster at best.
-- Best Regards, Danilo Spinella
On 2/10/22 19:15, Danilo Spinella wrote:
10 years ago when everyone was using sysvinit and large number of shell scripts were launched at boot the difference was probably far larger then now when most things are done via systemd without launching a bunch of shell scripts. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
On 2022-02-10 09:45, Danilo Spinella wrote:
Faster how? Faster execution, or faster loading? The later, can it be noticed in a modern computer with SSD disk? Is it worth it? If it is faster loading, is that really so after the first time bash/dash is loaded? Shouldn't the code be cached or reused on each instance?
Carlos E. R. (from 15.3 x86_64 at Telcontar)
On Thu, 2022-02-10 at 11:12 +0100, Carlos E. R. wrote:
This a page that summarize the benefits of dash: https://wiki.archlinux.org/title/Dash About being faster, there is this thread linked in the page above: https://unix.stackexchange.com/questions/148035/is-dash-or-some-other-shell-...
I doubt that it would make any noticible difference.
-- Best Regards, Danilo Spinella
On 2022-02-10 12:06, Danilo Spinella wrote:
Ok :-)
About being faster, there is this thread linked in the page above:
https://unix.stackexchange.com/questions/148035/is-dash-or-some-other-shell-...
Huh, does not seem to be linked there, AFAICS. Interesting link. -- Cheers / Saludos, Carlos E. R. (from 15.3 x86_64 at Telcontar)
On 2022/02/10 12:20:26 +0100, Carlos E. R. wrote:
Ah ... yes for minor problem the dash is much faster, nevertheless for string handling, logic, missing integer and array handling you have to use in dash external programs, that is every bournce shell not knowing about [[ ... ]] (1) might be fast for simple approaches but nevertheless remains a scrubby implmentation for a bourne shell. Werner (1) https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html section 2.4 reverse() { local -a R local -i i local rlen temp eval R=\( \"\$\{$1\[@\]\}\" \) rlen=${#R[@]} for ((i=0; i < rlen/2; i++ )) do temp=${R[i]} R[i]=${R[rlen-i-1]} R[rlen-i-1]=$temp done eval $1=\( \"\$\{R\[@\]\}\" \) } A=(1 2 3 4 5 6 7) echo "${A[@]}" reverse A echo "${A[@]}" reverse A echo "${A[@]}" -- "Having a smoking section in a restaurant is like having a peeing section in a swimming pool." -- Edward Burr
On Thu, 2022-02-10 at 14:19 +0100, Dr. Werner Fink wrote:
Here's a quick test made with shellbench [1] on my TW system (parameters: -t 10 -w 5). If this is representative in any way, dash is indeed the fastest shell for many simple problems. Not 4x faster than bash as some claim, but 2-3x. dash is slower than bash in the "[" and "printf" tests. ksh is the only shell that consistently outperforms bash, while showing less compatibility issues than the other competitors (I recall now that this was what Jörg had been saying back in 2019). busybox, too, peforms remarkably well. I still doubt that these differences would be noticeable on a typical workstation. The most important complex every-day shell script to optimize is dracut, and it requires bash, as discussed elsewhere in this thread. Martin ------------------------------------------------------------------------------------------------ name bash dash bosh zsh ksh busybox ------------------------------------------------------------------------------------------------ assign.sh: positional params 309,803 897,773 216,649 298,668 593,941 842,861 assign.sh: variable 455,889 1,442,742 314,917 827,981 725,265 1,021,224 assign.sh: local var 449,377 1,650,951 325,661 917,863 error 1,037,797 assign.sh: local var (typeset) 442,434 error error 920,843 671,163 error cmp.sh: [ ] 253,913 147,891 180,249 182,454 662,943 578,257 cmp.sh: [[ ]] 358,636 error error 301,634 762,106 error cmp.sh: case 434,090 1,514,925 322,818 342,450 776,736 960,664 count.sh: posix 326,934 1,157,364 239,637 601,361 461,185 618,382 count.sh: typeset -i 313,352 error error 623,128 470,047 error count.sh: increment 410,586 error error 809,779 707,555 error eval.sh: direct assign 231,794 975,803 207,147 113,298 407,481 757,182 eval.sh: eval assign 131,344 144,008 132,045 86,521 256,234 461,215 eval.sh: command subs 2,930 5,837 3,691 3,107 93,022 4,646 func.sh: no func 451,311 1,425,775 339,526 362,524 701,703 1,016,482 func.sh: func 252,077 982,378 227,488 108,049 437,161 812,489 null.sh: blank error error error 1,073,487 error error null.sh: assign variable 466,156 1,663,488 334,585 920,170 732,371 909,327 null.sh: define function 468,874 1,773,816 395,646 886,402 725,691 1,069,598 null.sh: undefined variable 383,014 1,647,805 337,086 347,519 589,197 1,022,063 null.sh: : command 450,359 1,429,708 339,355 369,330 709,197 1,003,313 output.sh: echo 277,827 648,581 197,196 273,605 656,219 543,575 output.sh: printf 260,370 150,397 178,891 271,055 410,795 453,425 output.sh: print error error error 277,520 503,866 error subshell.sh: no subshell 427,013 1,376,562 305,978 371,355 705,553 1,019,329 subshell.sh: brace 424,110 1,330,574 301,697 297,153 710,964 1,007,819 subshell.sh: subshell 3,596 6,678 4,171 3,571 165,097 5,050 subshell.sh: command subs 3,328 6,164 4,015 3,655 124,921 4,083 subshell.sh: external command 1,940 3,038 2,823 1,648 2,444 1,944 [1] https://github.com/shellspec/shellbench/tree/master/sample
On Thursday 2022-02-10 12:06, Danilo Spinella wrote:
And to add some not-overly-scientific measurement: ./configure from github.com/grommunio/gromox runs in 1.6s (average of 2 runs with dash) vs 1.9s (2 runs with bash). something like 15%. ./runme from github.com/jengelh/sh-speed - a really arbitrary 5-minute-hacked-up test heavier on compiler use than anything else - is 24.6 vs 25.8. but 5% is still 5%.
On Thu, 2022-02-10 at 08:45 +0000, Danilo Spinella wrote:
Unfortunately Jörg Schilling isn't with us anymore, as he'd certainly have commented here. His previous posts on the topic are an interesting read. [*] *If* we install something else as /bin/bash by default I would strongly advocate a fully POSIX-compliant shell, which dash allegedly is not. I'm sure Jörg would also claim that it's faster than anything else (dunno if that's true). Martin [*] Thread "Can we assume that /bin/sh is bash?" from 3y ago - unfortunately the factory archive seems to be down, so I can't post a link.
On 2022-02-10 11:43, Martin Wilck wrote:
I found the thread in my personal archive, but it was the time of the old list server, so the headers do not contain an exact link to the archive. Date: Fri, 08 Feb 2019 13:22:54 +0100 Ok, archive is working for me. This is the start of thread: <https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/ASAVU5DT3KAL3O2OZ4W3MDMWI5U37OWB/> -- Cheers / Saludos, Carlos E. R. (from 15.3 x86_64 at Telcontar)
On Thu, Feb 10, 2022 at 4:40 AM dziobian <brunopitrus@hotmail.com> wrote:
what I do not understand is why this was not done with alternatives instead, I would also like to experiment with a different shell as /bin/sh.. Yes, dash is much faster but since the advent of systemd there are less shell scripts driving the system and the overhead is less.. other alternative would be: a) working in making bash faster, thi may not be easy :-) b) selectively use dash for things like PATH_BSHELL so the C library executes everything with dash as sh.. c) find sources of user-noticeable slowdown and fix those.
On Thu, 2022-02-10 at 10:59 -0300, Cristian Rodríguez wrote:
it was done using u-a from * Fri Feb 08 2019 Jan Engelhardt <…> - Put "sh" under control of update-alternatives until * Fri Oct 22 2021 Stefan Schubert <…> - Using package bash-sh instead of the update-alternative mechanism. u-a in all its glory is not perfect neither. so the approach with a 'drop-in' package was implemented (currently we have bash-sh, busybox-sh and dash-sh available - up to the user to shoot his own feet) Cheers, Dominique
On 2022/02/10 12:39:31 -0300, Cristian Rodríguez wrote:
bash-sh means that /bin/sh is the bash and nothing more ... it does not mean that there is no /bin/bash anymore ... and mkinitrd should use /bin/bash if it depends on bash extension <sigh> Werner -- "Having a smoking section in a restaurant is like having a peeing section in a swimming pool." -- Edward Burr
On Thu, Feb 10, Cristian Rodríguez wrote:
It's not a workaround, it's the correct fix. Please open a bug report for this. Ok, I did now... The grub2-check-default scripts uses many instructions which are undefined in POSIX sh. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Ivo Totev (HRB 36809, AG Nürnberg)
Hey,
Anyone knows how supported is this / when is it going to get switched on by default
As I'm the author of that change I can introduce my thinking here a bit. * There is no intention in my doing to achieve a switch the default /bin/sh away from bash as it is today. * My goal is to assess how *expensive* in work (one time and continuous) a potential switch would be. For that reason I am running my personal work station with /bin/sh being dash, also because I happen to be the dash package maintainer. * I am also testing a full DVD size package set with /bin/sh set to dash and fixing the fallout. so a couple of packages already have and will get over the next few days and weeks submitrequests or pull request from me that fix things. * There is a lot of fallout. I started with 90% failing packages, and now I'm down to just 3%. Still 3% on the size of tumbleweed is still a long way to go. * I encourage anyone to help with this effort, by e.g. using checkbashisms(1) on your script or making things that are not easy to do posix-compatible declare a bash dependency rather than a /bin/sh dependency. * Doing the work doesn't mean that we'll do any change on the distro at all. Don't feed fear uncertainty or doubt (you did not, just my general feeling from a quick glance over all the emails in this mailthread). Hope this clarifies. Please use https://bugzilla.suse.com/show_bug.cgi?id=1195391 for tracking of fixes you do in light of this effort. Thanks, Dirk
On 10.02.22 08:39, dziobian wrote:
I really hope openSUSE is not following Debian's lead of dash being the default /bin/sh. For multiple years I have been maintaining large pools of Debian/Ubuntu client installations and CI pools, and switching back to bash for /bin/sh is always the first thing we need to do. Yes, in theory scripts should just use /bin/bash if they use bash-isms. In practice, that just doesn't happen. It hasn't happened in the many years now that Debian has switched to dash, and I can't see it happen any time soon. Debugging user failures with some random script that some proprietary tool shipped that expects bash for /bin/sh is not worth the minimal benefit we get these days from dash (systemd, SSD, etc. being factors here). I've tried again and again to keep dash as default, and repeatedly regretted it after debugging for several hours why some functionality of a large proprietary doesn't work. It's sad, but it's a fight we lost: /bin/sh is bash in reality. Philipp
Hi Philipp, Am Do., 10. Feb. 2022 um 23:28 Uhr schrieb Philipp Wagner <mail@philipp-wagner.com>:
I really hope openSUSE is not following Debian's lead of dash being the default /bin/sh.
There is no such plan that I know of to change the default shell for the general purpose version of openSUSE. There might be very limited usecases that are not open for general computing usecases (think embedded, IoT, container) where no custom or 3rd party code ever runs or where an admin could login and do stuff. In these special circumstances moving away from bash can become an option. There were other community members introducing busybox in openSUSE many years ago and it also didn't raise any fear that bash would go away. dash is in no way different regard except that I happen to be the dash maintainer rather than the busybox maintainer. if openSUSE distribution packages are already prepared to be compatible with such usecases, then that is far less painful to get such a use case implemented at the end. Greetings, Dirk
On 11.02.22 00:35, Dirk Müller wrote:
Thanks for your feedback, Dirk! Yeah, I'm all for fixing packages where we can to have them correctly indicate in their scripts what shell they're looking for, which should make it possible to run an "openSUSE-packages-only" system with dash (or any other "POSIX shell", actually) as /bin/sh. Containers/embedded use cases are a great example for such an environment. Things get only messy once third-party software comes into play, as it does on a general purpose client or server installation. In short: Sorry for the noise, and keep up the good work on making dash available! Philipp
On Fri, Feb 11, 2022 at 12:35:15AM +0100, Dirk Müller wrote:
And that's exactly the problem. There are people already using busybox as a limited shell, and now you are adding dash to the mix. So long as there is something somewhere released that uses the configuration with one of these limited shells packages have to work with them. Any incompatibility between package you maintain and one of these shells is a user visible bug. If it were limited to bash shells using sh shebang that would be easy to fix but more subtle and hard to debug issues in the different shell implementations do exist. Thanks Michal
Hi Michal, Am Fr., 11. Feb. 2022 um 12:08 Uhr schrieb Michal Suchánek <msuchanek@suse.de>:
And that's exactly the problem. There are people already using busybox as a limited shell, and now you are adding dash to the mix.
I'm not forcing anyone to use the work that I'm doing. I understand your concern, and your concern is valid, that more variations in places where "it does not matter" just adds overhead and wins nothing. As my initial email indicated it is an experiment, and I submitted fixes for the fallout of that experiment and will continue to do so. I'm not forcing anyone to accept it though. In addition to that, what I learned while doing this *experiment* is that this problem is far simpler than I initially thought. The bulk of shell scripts that we run in package scriptlets are absolutely trivial and do not use nontrivial globbing or other features that could expose the commonly expected working subset of any posix like shell. There were issues like incorrect escaping or comments or plain syntax errors that bash happened to ignore or happened to just parse correctly. as these are undocumented quirks, it is possible that the next bash update causes them to fail mysteriously. Fixing this cruft is the absolutely right thing to do. Your webbrowser example is a good reason as well: rather than the webpage being "optimized to be viewed in IE", we want our webpages to be "optimized to run in any browser". I know anyones feedback if we happen to have some mandatory tool to use somewhere that doesn't work with firefox. Interoperability, open standards and open source belong together. Greetings, Dirk
On 2022-02-11 12:36, Dirk Müller wrote:
Wild idea: Create a shebang that lists more than one shell that can be used to run the (current) script. So instead of specifying bash, say that it can run with bash, dash, busybox... whichever. That way no need to "argue" what means /bin/sh ;-) -- Cheers / Saludos, Carlos E. R. (from 15.3 x86_64 at Telcontar)
On Thu, Feb 10, dziobian wrote:
It is as supported as anthing else in Tumbleweed. Using dash as default doesn't make any sense, this would only mean one more shell beside bash and increasing the system without benefit You can use it already today if you want e.g. a bash free system and if there are no other requirements for bash. So clearly special cases, not default. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Ivo Totev (HRB 36809, AG Nürnberg)
On 2/10/22 09:07, Thorsten Kukuk wrote:
Using dash as the default shell is meant to help avoid bashisms, i.e. bash-specific shell code which would break on other shells if a user decided to replace bash as their default shell. Defaulting to dash for system scripts is actually a pretty sensible thing to do. Adrian
On 2/10/22 18:41, John Paul Adrian Glaubitz wrote:
Another place where it would likely be useful is it may well provide a reasonable speed up in building packages that still use traditional makefiles. THis benefit probably doesn't off set the amount of effort to fix all the broken things, doing some experiments with BuildRequires: dash-sh on longer building packages is probably worth the effort though. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
Hello, On 2022-02-10 09:11, John Paul Adrian Glaubitz wrote:
Is "#!/bin/bash" no longer supported? How does using a non-bash shell avoid bashisms? I think using a non-bash shell causes errors at the user for scripts that use bashisms without proper shebang i.e. end users get errors for faults of others. Not really end user friendly. But strictly technically right - of course - yeah! Kind Regards Johannes Meixner -- SUSE Software Solutions Germany GmbH Maxfeldstr. 5 - 90409 Nuernberg - Germany (HRB 36809, AG Nuernberg) GF: Ivo Totev
On Thu, Feb 10, Johannes Meixner wrote:
Of course it is. There are many features of bash which do not exist in a POSIX conform sh and for which is no replacement available. But you have to use "#!/bin/bash" if you use such features, using "#!/bin/sh" and expect that bash features work is plain wrong.
How does using a non-bash shell avoid bashisms?
Because your script stops working until you fix it to either: - use /bin/bash - remove the bashism.
Depends on from which angle you are looking at this. There are enough cases where you don't have bash available and you have to use a POSIX shell, even for Tumbleweed. For this users (which are not the Tumbleweed Desktop users, I agree), it is really helpful and very user friendly. Thorsten
-- Thorsten Kukuk, Distinguished Engineer, Senior Architect SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany Managing Director: Ivo Totev (HRB 36809, AG Nürnberg)
On 2022/02/10 10:29:42 +0100, Johannes Meixner wrote:
AFAIK /bin == /usr/bin on Tumbleweed, damit gibt es eine /bin/bash Werner -- "Having a smoking section in a restaurant is like having a peeing section in a swimming pool." -- Edward Burr
participants (17)
-
Carlos E. R. -
Cristian Rodríguez -
Danilo Spinella -
Dirk Müller -
Dominique Leuenberger / DimStar -
Dr. Werner Fink -
dziobian -
Jan Engelhardt -
Johannes Meixner -
John Paul Adrian Glaubitz -
Martin Wilck -
Michal Suchánek -
Philipp Wagner -
Simon Lees -
Stephan Kulow -
Sy retia -
Thorsten Kukuk