Re: [opensuse-factory] Can we assume that /bin/sh is bash?
On Fri, 8 Feb 2019 at 18:28, Martin Wilck
I'm not sure it needs to be bash, I'm open to the idea of it changing, and I'm even willing to help with the insane amount of fallout that could be caused if we do decide to change it ;)
You seem to be arguing for dash: https://wiki.ubuntu.com/DashAsBinSh. But that article also painfully demonstrates all the stuff people need to avoid if they want to write compliant code.
Yeah, but if major distributions like Debian and Ubuntu have already taken that step in 2006 and we've got reviewers like Jan who've already been generally espousing good practice, maybe this is not beyond our ability to do. Extrapolating Jans '7 out of 120' package sample, I'd estimate we have probably around 700 packages out of the 11000 in the distribution that could need attention. We fixed ~350 when we flattened /var into a single btrfs subvolume, this exercise seems only twice as bad ;) I remain to be convinced if it's really worth the effort, so I'd like to encourage others to weigh in with their opinions, but I'm certainly open to the idea. -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Fri, 8 Feb 2019 19:01:52 +0100
Richard Brown
On Fri, 8 Feb 2019 at 18:28, Martin Wilck
wrote: I'm not sure it needs to be bash, I'm open to the idea of it changing, and I'm even willing to help with the insane amount of fallout that could be caused if we do decide to change it ;)
You seem to be arguing for dash: https://wiki.ubuntu.com/DashAsBinSh. But that article also painfully demonstrates all the stuff people need to avoid if they want to write compliant code.
Yeah, but if major distributions like Debian and Ubuntu have already taken that step in 2006 and we've got reviewers like Jan who've already been generally espousing good practice, maybe this is not beyond our ability to do.
Extrapolating Jans '7 out of 120' package sample, I'd estimate we have probably around 700 packages out of the 11000 in the distribution that could need attention.
We fixed ~350 when we flattened /var into a single btrfs subvolume, this exercise seems only twice as bad ;)
I remain to be convinced if it's really worth the effort, so I'd like to encourage others to weigh in with their opinions, but I'm certainly open to the idea.
To what end? Exchange one set of bugs for another? If you look at bug lists in a distro such as Debian that sorts bugs per package you will find no shell is fully compliant. There is even a bug filed against dash for not failing to parse a perfectly valid shell code that bash and posh reject. There are even bugs filed pointing out differences in behavior and comments pointing out that both is compliant. Urgh. If one shell was good enough for the past 20 years let's keep its bugs and not introduce new ones unless there is *really* good reason backed by solid data. Thanks Michal -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Sat, 2019-02-09 at 00:19 +0100, Michal Suchánek wrote:
If you look at bug lists in a distro such as Debian that sorts bugs per package you will find no shell is fully compliant. There is even a bug filed against dash for not failing to parse a perfectly valid shell code that bash and posh reject. There are even bugs filed pointing out differences in behavior and comments pointing out that both is compliant. Urgh. If one shell was good enough for the past 20 years let's keep its bugs and not introduce new ones unless there is *really* good reason backed by solid data.
Ack. And *if* we do so, we might as well simply define that /bin/sh is
"bash", and consequently allow scripts and scriptlets to use the full
feature set that bash has.
Regards,
Martin
--
Dr. Martin Wilck
Michal Suchánek
If you look at bug lists in a distro such as Debian that sorts bugs per package you will find no shell is fully compliant. There is even a bug filed against dash for not failing to parse a perfectly valid shell code that bash and posh reject. There are even bugs filed pointing out differences in behavior and comments pointing out that both is compliant. Urgh. If one shell was good enough for the past 20 years let's keep its bugs and not introduce new ones unless there is *really* good reason backed by solid data.
Could you point to that bug please? BTW: - $(cmd) works best with bosh and mksh - $((expr)) works best in bosh and ksh93 and has problems in bash and significant problems in dash. In general however, the typical shell scripts on a platform should not cause related problems as the typical cases are handled correctly by most shells. Jörg -- EMail:joerg@schily.net (home) Jörg Schilling D-13353 Berlin joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/' -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On Mon, 11 Feb 2019 12:18:29 +0100
Joerg Schilling
Michal Suchánek
wrote: If you look at bug lists in a distro such as Debian that sorts bugs per package you will find no shell is fully compliant. There is even a bug filed against dash for not failing to parse a perfectly valid shell code that bash and posh reject. There are even bugs filed pointing out differences in behavior and comments pointing out that both is compliant. Urgh. If one shell was good enough for the past 20 years let's keep its bugs and not introduce new ones unless there is *really* good reason backed by solid data.
Could you point to that bug please?
BTW:
- $(cmd) works best with bosh and mksh
- $((expr)) works best in bosh and ksh93 and has problems in bash and significant problems in dash.
In general however, the typical shell scripts on a platform should not cause related problems as the typical cases are handled correctly by most shells.
Here: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=382276 I have seen code artifically balancing case parentheses in our kernel build scripts and added some myself but I am not sure if the problem is in rpm spec parser or bash in this case. Thanks Michal -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Michal Suchánek
In general however, the typical shell scripts on a platform should not cause related problems as the typical cases are handled correctly by most shells.
Here: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=382276
I have seen code artifically balancing case parentheses in our kernel build scripts and added some myself but I am not sure if the problem is in rpm spec parser or bash in this case.
This is a bash bug that has been fixed in bash-5. Please note: If you have any problem with $(cmd), I recommend to test with bosh, as bosh has the most correct known implementation. Jörg -- EMail:joerg@schily.net (home) Jörg Schilling D-13353 Berlin joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/' -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
Joerg Schilling
This is a bash bug that has been fixed in bash-5.
Let me give some background information: When David Korn introduced $(cmd) in ksh88, he thought that this could be implemented with a simple parser that just counts brackets. To permit this, he introduced the openning bracket: "switch ... case (x)" but it turns out that there are many other cases in the shell command language that need a recursive parser to support $(cmd), e.g. here documents that are par of a $(cmd) construct. Let me explain how bosh and mksh implement it: - First the parser is called with the string similar to: "cmd)" and told to stop parsing at a superfluous ")". - The syntax tree that is the output from the parser is then fed into the code that prints shell input from the binary tree (this has been introduced for printing functions. - This string is than kept in the syntax tree from the main parser call. If you do it this way, there is no need to fail in the parser... Jörg -- EMail:joerg@schily.net (home) Jörg Schilling D-13353 Berlin joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/' -- To unsubscribe, e-mail: opensuse-factory+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-factory+owner@opensuse.org
On 09/02/2019 04:31, Richard Brown wrote:
On Fri, 8 Feb 2019 at 18:28, Martin Wilck
wrote: I'm not sure it needs to be bash, I'm open to the idea of it changing, and I'm even willing to help with the insane amount of fallout that could be caused if we do decide to change it ;)
You seem to be arguing for dash: https://wiki.ubuntu.com/DashAsBinSh. But that article also painfully demonstrates all the stuff people need to avoid if they want to write compliant code.
Yeah, but if major distributions like Debian and Ubuntu have already taken that step in 2006 and we've got reviewers like Jan who've already been generally espousing good practice, maybe this is not beyond our ability to do.
Extrapolating Jans '7 out of 120' package sample, I'd estimate we have probably around 700 packages out of the 11000 in the distribution that could need attention.
We fixed ~350 when we flattened /var into a single btrfs subvolume, this exercise seems only twice as bad ;)
I remain to be convinced if it's really worth the effort, so I'd like to encourage others to weigh in with their opinions, but I'm certainly open to the idea.
I'm just picking this email to piggy back off rather then replying to 5 individually. I think since the introduction of systemd the general benefit is much less, the reality is much less of the system depends on shell scripts compared to say 10 years ago. From a packaging perspective I don't mind so much either way if people want to put in the work. I have never rejected a review because of bashism's in a #!/bin/sh i'm guessing that I like many others simply don't know everything that is and isn't a bashism. I don't think that its reasonable to expect all developers to learn this. So if we did decide to enforce this kind of rule personally i'd like to see some form of automated checking whether its in rpmlint or something else so everyone isn't expected to just know. Until that point I don't think we should enforce it in reviews. I also don't think without some form of checking we would be able to properly guarantee that we hadn't accidentally broken something and as such I wouldn't put it at the level of something we should officially support as part of openSUSE. An openQA test that installs all the core packages with dash instead of bash could be another alternative but I expect it would be kinda hard to maintain. -- Simon Lees (Simotek) http://simotek.net Emergency Update Team keybase.io/simotek SUSE Linux Adelaide Australia, UTC+10:30 GPG Fingerprint: 5B87 DB9D 88DC F606 E489 CEC5 0922 C246 02F0 014B
On Sat, 2019-02-09 at 14:36 +1030, Simon Lees wrote:
From a packaging perspective I don't mind so much either way if people want to put in the work. I have never rejected a review because of bashism's in a #!/bin/sh i'm guessing that I like many others simply don't know everything that is and isn't a bashism. I don't think that its reasonable to expect all developers to learn this. So if we did decide to enforce this kind of rule personally i'd like to see some form of automated checking whether its in rpmlint or something else so everyone isn't expected to just know. Until that point I don't think we should enforce it in reviews.
There's the "checkbashisms" script from Debian. It's available on
factory as part of the "devscripts" package. I guess it should be
possible to add that to a build check. We could even force it to run
over any script with /bin/sh shebang...
... except that "checkbashisms" itself doesn't work under factory; it
spits out all kinds of perl syntax problems. It's kind of ironic that
this happens with just this script.
Martin
--
Dr. Martin Wilck
participants (5)
-
Joerg Schilling
-
Martin Wilck
-
Michal Suchánek
-
Richard Brown
-
Simon Lees