Re: [opensuse-factory] Can we assume that /bin/sh is bash?
Hi, On Mon, Feb 18, 2019 at 6:46 PM Martin Wilck <mwilck@suse.com> wrote:
On Mon, 2019-02-18 at 13:36 +0100, Joerg Schilling wrote:
Mischa Salle <mischa.salle@gmail.com> wrote:
I might have missed part of this (very long) discussion, but in my opinion - using /bin/sh means you SHOULD only use constructions which are POSIX compliant. - when you want to use bashisms, use /bin/bash. The same applies to other shells.
This is finally the right conclusion even thouh POSIX does not make any statements on what /bin/sh is.
I don't think that this is the right conclusion. Here's why.
I've been told in this discussion that the POSIX shell standard for "allows extensions". IOW, it only defines a minimal set of features, and implementations are free to add more features on top and still "comply" with the standard. Therefore the fact that bash, invoked as "/bin/sh", supports the non-POSIX '[[ ]]' syntax doesn't make it non- compliant. Correct, *but* I'm saying the opposite: if you stick to the POSIX set of features, your script should be portable, which is not the case if you use extensions.
We need to define without ambiguity under what conditions a _shell script_ complies with the feature set of "/bin/sh". That means that we need to have a _test procedure_ (#). We could say that "code is compliant iff it runs without errors under every POSIX-compliant shell (*)". But that's hard to test, as we'd have to test with every POSIX- compliant shell out there, and we may not even know them all. This is indeed a hard(er) question. I'd say checkbashisms (part of Leap's main repo) would be a good start. That's also what Debian is doing in lintian AFAIK (Debian by the way has dash as the default /bin/sh). In general dash and ash are considered more POSIX close that bash, zsh, ksh etc. That doesn't mean you should per se use dash or ash as your default shell, but you can use them as a step in checking for POSIX compliance.
We also don't want to test by human beings parsing the code (even if that's basically what Jan is doing today). Rather, we need to settle on an actual shell implementation, and define "compliant" by reference to this implementation: "shell code is compliant iff it runs without errors under this implementation" (*). Indeed, which is why AFAIK Debian went for dash. Note by the way that there is a long thread from about 9 years ago, starting with https://lists.opensuse.org/opensuse-packaging/2010-03/msg00076.html
That implementation could be bash in POSIX mode, meaning that e.g. '[[ ]]' would be allowed. Alternatively, it could be a minimal shell like "pbosh".
So far the discussion in this thread did not come up with a prime candidate for being that reference shell. dash has been propoposed and then harshly dismissed. ash, likewise. I suppose that Jörg would propose "pbosh", and it seems to come closer to the ideal of a "strict" POSIX shell as anything else mentioned up to now. But I'd call it a bit exotic. It doesn't seem to have a lot of real-world users under openSUSE.
I still fail to see the benefit of choosing anything else than bash in POSIX mode for /bin/sh, unless we want to pursue the long-term goal to be able to run (a minimal) openSUSE completely without bash. The point is not what to choose as /bin/sh, but which constructs to accept in shell scripts. It would be bad to use bashisms, zshisms or any other isms in a /bin/sh, but that's about the code, not about the underlying shell. If we now start using bashisms in the shell scripts, because currently /bin/sh is /bin/bash it will make it much harder to ever change bash for another shell, while if you stick to POSIX construct only, it's a single change.
Best, Mischa -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Mischa Salle <mischa.salle@gmail.com> wrote:
I've been told in this discussion that the POSIX shell standard for "allows extensions". IOW, it only defines a minimal set of features, and implementations are free to add more features on top and still "comply" with the standard. Therefore the fact that bash, invoked as "/bin/sh", supports the non-POSIX '[[ ]]' syntax doesn't make it non- compliant. Correct, *but* I'm saying the opposite: if you stick to the POSIX set of features, your script should be portable, which is not the case if you use extensions.
A script that does not use non-POSIX features and that does not depend on unspecified behavior (another important issue) is called "fully compliant".
This is indeed a hard(er) question. I'd say checkbashisms (part of Leap's main repo) would be a good start. That's also what Debian is doing in lintian AFAIK (Debian by the way has dash as the default /bin/sh). In general dash and ash are considered more POSIX close that bash, zsh, ksh etc. That doesn't mean you should per se use dash or ash as your default shell, but you can use them as a step in checking for POSIX compliance.
That's a joke, dash and ash are not POSIX because they do not support multi-byte characters, which is required unless you are on a tiny embedded system. ash was so buggy that people created dash from ash by fixing bugs. The general idea on why Debian switched to dash seems to be useful. zsh is so far from POSIX that it is not even able to run "configure" in it's default behavior. When switched into it's POSIX mode, it still has noticeable deviations from POSIX. Given that it is as slow as bash, I see nothing that could cause an interest to install zsh as default shell. ksh88 on the other side was the blueprint for the definitions that define the "POSIX shell".
We also don't want to test by human beings parsing the code (even if that's basically what Jan is doing today). Rather, we need to settle on an actual shell implementation, and define "compliant" by reference to this implementation: "shell code is compliant iff it runs without errors under this implementation" (*). Indeed, which is why AFAIK Debian went for dash. Note by the way that there is a long thread from about 9 years ago, starting with https://lists.opensuse.org/opensuse-packaging/2010-03/msg00076.html
You finally need to test....
I still fail to see the benefit of choosing anything else than bash in POSIX mode for /bin/sh, unless we want to pursue the long-term goal to be able to run (a minimal) openSUSE completely without bash. The point is not what to choose as /bin/sh, but which constructs to accept in shell scripts. It would be bad to use bashisms, zshisms or any other isms in a /bin/sh, but that's about the code, not about the underlying shell. If we now start using bashisms in the shell scripts, because currently /bin/sh is /bin/bash it will make it much harder to ever change bash for another shell, while if you stick to POSIX construct only, it's a single change.
So I recommend to try out other shells and to check the current scripts for kshisms and bashisms. It may of course help to firsh use something like "checkbashisms" on scripts that have #!/bin/sh inside or that do not have #! at all. This has been done with Gentoo Linux 2 years ago with pbosh and this helped to identify and fix things that have been related to non-POSIX behavior in pbosh as a result that this shell is a descendent from the original Bourne Shell and the fact that not all differences between Bourne Shell and POSIX shell had been documented before. So my advise is: do it and check whether you have system scripts that depend on non standard behavior. You may have luck and there is few to change. BTW: In 2008, OpenSolaris has been converted to have ksh93 as /bin/sh and several scripts have been changed. I needed to introduce minor changes (mainly to avoid [[ ]]) into three system scripts in order to be able to install pbosh as /bin/sh. Given that this discussion is more than a simple question that ends after someone posted an answer, you should run such a test and fix scripts that include #!/bin/sh but depend on non standard- or on undefined behavior. Jörg -- EMail:joerg@schily.net (home) Jörg Schilling D-13353 Berlin joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/' -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Tue, Feb 19, 2019 at 12:28:02PM +0100, Joerg Schilling wrote:
So I recommend to try out other shells and to check the current scripts for kshisms and bashisms. It may of course help to firsh use something like "checkbashisms" on scripts that have #!/bin/sh inside or that do not have #! at all.
This has been done with Gentoo Linux 2 years ago with pbosh and this helped to identify and fix things that have been related to non-POSIX behavior in pbosh as a result that this shell is a descendent from the original Bourne Shell and the fact that not all differences between Bourne Shell and POSIX shell had been documented before. So my advise is: do it and check whether you have system scripts that depend on non standard behavior. You may have luck and there is few to change.
BTW: In 2008, OpenSolaris has been converted to have ksh93 as /bin/sh and several scripts have been changed. I needed to introduce minor changes (mainly to avoid [[ ]]) into three system scripts in order to be able to install pbosh as /bin/sh.
Ouch ... IMHO [[...]] is a major win for a bourne shell. If you're teaching shell syntax there is no nead to mention [...] but only [[...]] with ksh93 and bash. I do not see any advantage to use a minimal set of POSIX bourne shell. Here we have the fully control which bourne shell should be used for /bin/sh. If systems will be ported from an old UNIX system (and IMHO OpenSolaris is old ;) most most scripts will run even if written for minimal POSIX bourne shell. There is no need to use a castrated bourne shell aka minimal POSIX bourne shell on Linux. -- "Having a smoking section in a restaurant is like having a peeing section in a swimming pool." -- Edward Burr
"Dr. Werner Fink" <werner@suse.de> wrote:
Ouch ... IMHO [[...]] is a major win for a bourne shell. If you're teaching shell syntax there is no nead to mention [...] but only [[...]] with ksh93 and bash. I do not see any advantage to use a minimal set
The advantage of using [[ instead of [ is minimal and you don't really need it as you could write scripts that avoid it. ksh93 has other real advantages that people usually miss and that BTW are just being destroyed by the Redhat people that work on the https://github.com/att/ast/ repository.
of POSIX bourne shell. Here we have the fully control which bourne shell should be used for /bin/sh. If systems will be ported from an old UNIX system (and IMHO OpenSolaris is old ;) most most scripts will run even if
Correct, OpenSolaris is a true descendent from the original UNIX Code and as a result, it will be able to look back at 50 years of of UNIX history in August this year. If you look at the kernel, there are many modern concepts inside Solaris, that are still missing in Linux, so what do you understand by "old"?
written for minimal POSIX bourne shell. There is no need to use a castrated bourne shell aka minimal POSIX bourne shell on Linux.
The advantage of using a small but POSIX shell as the default shell (in special if it supports "modern" features like vfork() that are unsupported by bash) is a faster and more robust startup. Even on a true UNIX, there are other shells in addition, but these shells are used after the basic startrup is ready. Finally: /bin/sh is not mentioned in POSIX it is not "the POSIX shell" of a platform, it is just the default shell. In theory, you could install something that stinks like "fish" as /bin/sh if you adjust your scripts and if you follow the official rules for providing a POSIX shell. The POSIX shell is run after you type: PATH=`getconf PATH` sh and typically is in a path like /usr/xpg?/bin/sh Jörg -- EMail:joerg@schily.net (home) Jörg Schilling D-13353 Berlin joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/' -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
participants (3)
-
Dr. Werner Fink -
Joerg Schilling -
Mischa Salle