[opensuse] system scripts using BASHism's but invoked as "#!/bin/sh"

newer
Re: [opensuse] Advice on gigabit...

Linda Walsh

24 Jan 2013 24 Jan '13

08:42

I'm constantly seeing system scripts starting with #!/bin/sh which tells BASH to run in POSIX conformance mode, yet occasionally, more or less randomly, I'll see "BASHism's" in the script. Case in point: the cron job for logrotate references the Array PIPESTATUS to get at the exit value of log rotate. Arrays and PIPESTATUS never used to be part of POSIX. Also a quick grep turns up startup scripts, using Bash's '[[' syntax, such as: boot.localfs boot.kdump dhcpd dhcpd6 kbd dictd cifs monit mysql named zfs-fuse Shouldn't these scripts be invoked as #!/bin/bash? Maybe some of them run because they are invoked as bash -c <script>, but it seems problematic to state that it is to be used with #!/bin/sh at the top of a script but be using "Bashism's" in the script. On my own system, I change the line at the top to say #!/bin/bash if the script uses Bash syntax. Maybe that should become the standard prefix if system scripts are going to be using Bash syntax? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Show replies by date

Bernhard Voelker

24 Jan 24 Jan

08:52

On 01/24/2013 09:42 AM, Linda Walsh wrote:

...

I'm constantly seeing system scripts starting with #!/bin/sh which tells BASH to run in POSIX conformance mode, yet occasionally, more or less randomly, I'll see "BASHism's" in the script.

Wow, not bad (on a 12.1 system): $ find /etc/init.d -maxdepth 1 -mindepth 1 -type f | \ xargs checkbashisms 2>&1 | wc -l 400 Have a nice day, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Linda Walsh

09:21

Bernhard Voelker wrote:

...

On 01/24/2013 09:42 AM, Linda Walsh wrote:

...
I'm constantly seeing system scripts starting with #!/bin/sh which tells BASH to run in POSIX conformance mode, yet occasionally, more or less randomly, I'll see "BASHism's" in the script.

Wow, not bad (on a 12.1 system):

$ find /etc/init.d -maxdepth 1 -mindepth 1 -type f | \ xargs checkbashisms 2>&1 | wc -l 400

Have a nice day, Berny

---- So you are saying it's no problem? whatever. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Bernhard Voelker

09:47

On 01/24/2013 10:21 AM, Linda Walsh wrote:

...

So you are saying it's no problem? whatever.

Reading my own mail, I'd say that I wrote that I was astonished to find so many bashisms in my own /etc/init.d/, not more. And as such, just simply demonstrating a few numbers for your statement. Without appraisal. Have a nice day, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Dr. Werner Fink

09:32

On Thu, Jan 24, 2013 at 12:42:51AM -0800, Linda Walsh wrote:

...

I'm constantly seeing system scripts starting with #!/bin/sh which tells BASH to run in POSIX conformance mode, yet occasionally, more or less randomly, I'll see "BASHism's" in the script.

Case in point: the cron job for logrotate references the Array PIPESTATUS to get at the exit value of log rotate.

True.

...

Arrays and PIPESTATUS never used to be part of POSIX.

True.

...

Also a quick grep turns up startup scripts, using Bash's '[[' syntax, such as:

AFAIK the `[[ ... ]]' is known in latest POSIX, see e.g. http://manuals.ts.fujitsu.com/file/8867/posix_k.pdf on page 44 and page 57ff. And not only the bash does it use but also the ksh, IMHO most user should use this for any Conditional Expression instead of the `test ...' or the '[ ... ]' builtin to be sure not run into trouble if regular experssions are used ;) Also the `let ...' builtin ... or its equivalent `(( ... ))' for integer arithmetic's are well known, see page 448ff.

...

boot.localfs boot.kdump dhcpd dhcpd6 kbd dictd cifs monit mysql named zfs-fuse

Shouldn't these scripts be invoked as #!/bin/bash?

Maybe some of them run because they are invoked as bash -c <script>, but it seems problematic to state that it is to be used with #!/bin/sh at the top of a script but be using "Bashism's" in the script.

On my own system, I change the line at the top to say #!/bin/bash if the script uses Bash syntax. Maybe that should become the standard prefix if system scripts are going to be using Bash syntax?

Werner -- "Having a smoking section in a restaurant is like having a peeing section in a swimming pool." -- Edward Burr -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Linda Walsh

17:22

New subject: [opensuse] Re: system scripts using BASHism's but invoked as "#!/bin/sh"

Dr. Werner Fink wrote:

...

On Thu, Jan 24, 2013 at 12:42:51AM -0800, Linda Walsh wrote:

...
I'm constantly seeing system scripts starting with #!/bin/sh which tells BASH to run in POSIX conformance mode, yet occasionally, more or less randomly, I'll see "BASHism's" in the script.

Case in point: the cron job for logrotate references the Array PIPESTATUS to get at the exit value of log rotate.

True.

...
Arrays and PIPESTATUS never used to be part of POSIX.

True.

...
Also a quick grep turns up startup scripts, using Bash's '[[' syntax, such as:

AFAIK the `[[ ... ]]' is known in latest POSIX, see e.g.

http://manuals.ts.fujitsu.com/file/8867/posix_k.pdf

on page 44 and page 57ff. And not only the bash does it use but also the ksh, IMHO most user should use this for any Conditional Expression instead of the `test ...' or the '[ ... ]' builtin to be sure not run into trouble if regular experssions are used ;)

Also the `let ...' builtin ... or its equivalent `(( ... ))' for integer arithmetic's are well known, see page 448ff.

Does this imply that the check Bernhard Voelker did that checks for "bashisms", is faulty, or has it been updated to allow for the newest/greatest/latest POSIX-of-the-day standard? It's hard to keep track which has been added and which is Bash specific -- PIPESTATUS and ARRAYS -- they weren't part of the original POSIX, but POSIX 2013? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Linda Walsh

26 Jan 26 Jan

01:24

New subject: [opensuse] Re: system scripts using BASHism's but invoked as "#!/bin/sh"

Dr. Werner Fink wrote:

...

AFAIK the `[[ ... ]]' is known in latest POSIX, see e.g.

http://manuals.ts.fujitsu.com/file/8867/posix_k.pdf

---- Checking the posix source @ http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html Shows that while (()) are used in arithmetic, [[]] are not posix compliant. Neither are arrays supported. ----------------- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Werner Flamme

27 Jan 27 Jan

14:24

New subject: [opensuse] Re: system scripts using BASHism's but invoked as "#!/bin/sh"

[26.01.2013 02:24] [Linda Walsh]:

...

Dr. Werner Fink wrote:

...
AFAIK the `[[ ... ]]' is known in latest POSIX, see e.g.

http://manuals.ts.fujitsu.com/file/8867/posix_k.pdf

This is from 2009

...

---- Checking the posix source @ http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html

This is from 2004

...

Shows that while (()) are used in arithmetic,

[[]] are not posix compliant.

Neither are arrays supported.

May things have changed in 5 years? -- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Anton Aylward

24 Jan 24 Jan

12:51

Linda Walsh said the following on 01/24/2013 03:42 AM:

...

[...]

On my own system, I change the line at the top to say #!/bin/bash if the script uses Bash syntax. Maybe that should become the standard prefix if system scripts are going to be using Bash syntax?

If you think this is a bug/defect then report it as such. -- "Don't get complacent. Push yourself out of your comfort zone and set higher standards of achievement for yourself. Once you've achieved a standard of excellence, never let it rest--push yourself even higher." -- Dave Anderson -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Michal Hrusecky

13:09

Linda Walsh - 0:42 24.01.13 wrote:

...

...

Also a quick grep turns up startup scripts, using Bash's '[[' syntax, such as:

boot.localfs boot.kdump dhcpd dhcpd6 kbd dictd cifs monit mysql

mysql uses #!/bin/bash -- Michal HRUSECKY SUSE LINUX, s.r.o openSUSE Team Lihovarska 1060/12 PGP 0xFED656F6 19000 Praha 9 mhrusecky[at]suse.cz Czech Republic http://michal.hrusecky.net http://www.suse.cz -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Ken Schneider - openSUSE

17:02

On 01/24/2013 08:09 AM, Michal Hrusecky pecked at the keyboard and wrote:

...

Linda Walsh - 0:42 24.01.13 wrote:

...
...

Also a quick grep turns up startup scripts, using Bash's '[[' syntax, such as:

boot.localfs boot.kdump dhcpd dhcpd6 kbd dictd cifs monit mysql

mysql uses #!/bin/bash

The #! line determines the shell environment that the script will use. It is up to the programmer to determine which shell provides the most #! for the buck. -- Ken Schneider SuSe since Version 5.2, June 1998 -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Linda Walsh

17:23

New subject: [opensuse] Re: system scripts using BASHism's but invoked as "#!/bin/sh"

Michal Hrusecky wrote:

...

mysql uses #!/bin/bash

--- A gem among the the scripts for declaring it's flavor! ;-) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Cristian Rodríguez

17:25

El 24/01/13 05:42, Linda Walsh escribió:

...

boot.localfs boot.kdump

Those will go away as systemd no longer supports early boot init scripts. so they can be left as is.

...

dhcpd dhcpd6 kbd dictd cifs monit mysql named zfs-fuse

If absent, those need native systemd units that do not have this issue, as there is no shell available for them at all. Many of those are redundant now and can be safely ignored. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Linda Walsh

20:01

New subject: [opensuse] script support and partition requirements of systemd? (was Re: system scripts using BASHism's but invoked as "#!/bin/sh")

Cristian Rodr�� wrote:

...

...
dhcpd dhcpd6 kbd dictd cifs monit mysql named zfs-fuse

If absent, those need native systemd units that do not have this issue, as there is no shell available for them at all.

Many of those are redundant now and can be safely ignored.

What about local startup scripts. I have a few to start services and monitor things of my own. That's one thing that seems to be a pain with the new systemd service, is that writing a shell script is pretty much standard SysAdmin knowledge. But is configuring a non-standards-compliant service like systemd going to be something so easily configured? Another problem I've been having lately is being unable to boot unless I first boot into 'S' and mount /usr. Since that works... maybe a variation on that could be the workaround for booting with separate user directly from disk? I.e. run levels won't be used anymore, but I can use run-level 'S' to mount /usr and then run init 3 to bring up my system. Perhaps I could have a new init-3 script that would first mount /usr (what is done manually by me in 'S'? Is there a reason why that wouldn't work -- since "procedurally", it seems to work now -- with the downside that I must mount /usr manually and then type in init 3. Is there a reason why that can't be automated as a workaround for needing /usr available during boot? I'm still unclear about /usr/share. Supposedly /usr/share is supposed to be architecture independent and is often made exportable -- and makes sense as it's own partition. Does it also have to be mounted before boot? What file systems is systemd capable of mounting before it brings up the rest of the system? Or rather what other file systems besides /usr is it not capable of mounting? /tmp? /var/tmp? /var? /var/cache /proc? For that matter, why can systemd mount /proc and /sys, but not /usr? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Cristian Rodríguez

20:21

New subject: [opensuse] script support and partition requirements of systemd? (was Re: system scripts using BASHism's but invoked as "#!/bin/sh")

El 24/01/13 17:01, Linda Walsh escribió:

...

What about local startup scripts. I have a few to start services and monitor things of my own.

Please read what I've said above. I stated that "early boot sysv scripts" are no longer supported in upcoming systemd versions, this scripts are provided by distributions to setup stuff *before* any user provided scripts are even taken into consideration.

...

That's one thing that seems to be a pain with the new systemd service, is that writing a shell script is pretty much standard SysAdmin knowledge. But is configuring a non-standards-compliant service like systemd going to be something so easily configured?

Another problem I've been having lately is being unable to boot unless I first boot into 'S' and mount /usr.

both /boot and /usr are mounted by the initrd (either created by old mkinitrd or dracut) I dont understand what problem you are facing exactly.

...

What file systems is systemd capable of mounting before it brings up the rest of the system? Or rather what other file systems besides /usr is it not capable of mounting? /tmp? /var/tmp? /var? /var/cache /proc? For that matter, why can systemd mount /proc and /sys, but not /usr?

Systemd only mounts either the pseudofilesystems itself requires to function or what you supplied in /etc/fstab or in .mount or .automount units. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Linda Walsh

25 Jan 25 Jan

05:02

New subject: [opensuse] Re: script support and partition requirements of systemd? (was Re: system scripts using BASHism's but invoked as "#!/bin/sh")

Cristian Rodríguez wrote:

...

El 24/01/13 17:01, Linda Walsh escribió:

...
What about local startup scripts. I have a few to start services and monitor things of my own.

Please read what I've said above. I stated that "early boot sysv scripts" are no longer supported in upcoming systemd versions, this scripts are provided by distributions to setup stuff *before* any user provided scripts are even taken into consideration.

...
Another problem I've been having lately is being unable to boot unless I first boot into 'S' and mount /usr.

both /boot and /usr are mounted by the initrd (either created by old mkinitrd or dracut) I dont understand what problem you are facing exactly.

I am using a vanilla kernel downloaded from kernel.org. It has the hardware options needed for most of my purposes (and probably a few I don't need -- it's hard to find out which ones can be useful) built into the kernel, so the kernel boots and immediately starts to run startup scripts from the hard disk. It used to be that it would use the pre-level 1 startup scripts like boot.localfs, to mount local file systems. It no longer does that. I have to boot to level 'S' (level 1 won't work) and mount /usr manually. /boot doesn't appear to be necessary for the system to come up because the kernel is already loaded (why does '/boot' need to be mounted during system startup -- it needs to be mounted when I update my kernel, but at system startup? Only a few modules are really needed after boot and are loaded out of /lib/modules on demand after boot: # lsmod Module Size Used by sch_sfq 10080 3 sch_htb 15123 1 mousedev 11440 1 acpi_cpufreq 7577 1 mperf 1348 1 acpi_cpufreq processor 35852 1 acpi_cpufreq ---------- Actual module usage is higher as one can tell from /sys/modules: # ls /sys/module/ 8250_core/ edd/ kgdboc/ processor/ acpi/ efivars/ libahci/ pstore/ acpi_cpufreq/ ehci_hcd/ libata/ rcutree/ acpi_power_meter/ firewire_ohci/ lockd/ sch_htb/ ahci/ fscache/ lockdep/ sch_sfq/ auth_rpcgss/ gpio_ich/ loop/ scsi_mod/ block/ hid/ megaraid_sas/ scsi_transport_fc/ bnx2/ i2c_algo_bit/ mousedev/ scsi_transport_iscsi/ bonding/ i2c_i801/ mperf/ sg/ brd/ i2o_core/ mptbase/ spurious/ cifs/ i7300_idle/ mptsas/ sr_mod/ coretemp/ i7core_edac/ mptscsih/ sunrpc/ cpuidle/ i8042/ nbd/ tcp_htcp/ dca/ intel_idle/ netpoll/ uhci_hcd/ dcdbas/ ioatdma/ nf_conntrack/ uinput/ debug_core/ ipmi_msghandler/ nf_conntrack_ipv4/ usbcore/ dell_rbu/ ipmi_si/ nfs/ usbhid/ dns_resolver/ ixgbe/ nfsd/ vt/ drm/ kdb_main/ pcie_aspm/ xt_recent/ e1000e/ kernel/ pnp/ xz_dec/ edac_core/ keyboard/ printk/ ---------- For some reason, the startup scripts don't mount file systems like /usr -- so it has to be done manually, by first going into level 'S' mounting it, then issuing an init 3... So I'm wondering how I have another boot level, maybe 'B', (for boot) like 'S', where it does things like mounting the local disks necessary for systemd to function, since except for /usr, it seems to be able to mount the rest of the file systems. There was a problem running mount after a recent update, but that was because some libraries it needed were put in /usr/lib64. Putting them in /lib64 and links in /usr/lib64 solved that problem. So I'm wondering how I can get my own "pre-boot" script run, that can automatically mount /usr, and do basic system startup, so systemd can be happy?

...

Systemd only mounts either the pseudofilesystems itself requires to function or what you supplied in /etc/fstab or in .mount or .automount units.

---- I supplied /usr in /etc/fstab, but it doesn't mount that one. Of course, none of this helps when everything is link dynamically and most of your utils were built with a custom glibc that needs OW_CRYPT_1 which isn't included, by default, when you install the latest glibc from their website. You can't even 'sync' the disks before going down -- so disk corrupt is a given and a complete restore from backups becomes necessary in such cases. Things are getting very fragile... Am afraid to reboot my system these days (last uptime was over 40 days but made mistake of trying to update glibc, not knowing all of suse's apps were linked with with a non-default option so none function after installing the latest new glibc (they are supposed to be backwards compatible, but not unless you know what special symbols have been linked in... ;-)... So how can I run a setup script to mount /usr without using init 's'? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Anton Aylward

13:49

New subject: [opensuse] Re: script support and partition requirements of systemd? (was Re: system scripts using BASHism's but invoked as "#!/bin/sh")

Linda Walsh said the following on 01/25/2013 12:02 AM:

...

So how can I run a setup script to mount /usr without using init 's'?

The short answer is implicit in Cristian's observation about the initrd. Yes you should have a initrd that mounts both /boot and /usr. There's a lot out there on making custom initrd. And as you say, the distribution one has a lot you don't need (and is very large and slow) since it was designed to be able to run on any platform, any IO. If you're not using LVM, XFS ... but are using crypt, then you have a *VERY* *STRONG* motivation to make a custom initrd. The mechanics of making an initrd are, as I say, well documented. The hard part is deciding what to leave in and what to leave out :-) Your distribution kernel is what? 15M or so in 12.2 You should be able to reduce that to around 5M I forget the commands to list all that's in an initrd, but it was mentioned here a few weeks ago. Look it up and run it but please keep your "OMG! I Don't Need All That!" reaction to yourself, eh, we *know* you don't need all the 15M of the distribution kernel; few people do, That's why people rebuild their initrds. -- A computer without COBOL and Fortran is like a piece of chocolate cake without ketchup and mustard. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Linda Walsh

17:25

New subject: [opensuse] Re: script support and partition requirements of systemd? (was Re: system scripts using BASHism's but invoked as "#!/bin/sh")

Anton Aylward wrote:

...

Linda Walsh said the following on 01/25/2013 12:02 AM:

...
So how can I run a setup script to mount /usr without using init 's'?

The short answer is implicit in Cristian's observation about the initrd. Yes you should have a initrd that mounts both /boot and /usr.

Why? If the system comes up without them in 'S' mode, and I can mount them manually and the system then comes up, why can't I automate that with a script? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Anton Aylward

18:52

New subject: [opensuse] Re: script support and partition requirements of systemd? (was Re: system scripts using BASHism's but invoked as "#!/bin/sh")

Linda Walsh said the following on 01/25/2013 12:25 PM:

...

Anton Aylward wrote:

...
Linda Walsh said the following on 01/25/2013 12:02 AM:

...
So how can I run a setup script to mount /usr without using init 's'?

The short answer is implicit in Cristian's observation about the initrd. Yes you should have a initrd that mounts both /boot and /usr.

Why? If the system comes up without them in 'S' mode, and I can mount them manually and the system then comes up, why can't I automate that with a script?

DUH? I don't understand why 'the lady doth protest too much'. Why bother with a script to do it when you can do it simply 'by proper configuration' If the system comes up with them mounted does it matter to you if initrd does it 'behind the scenes (sorry, no Hamlet quote here) or a script does it behind the scenes? The way I look at it the CORRECT way is to have an initrd that does what you want and only what you want. Back in the "The 12.2 installer is smarter than I thought" that I started Per Jessen suggested lsinitrd /boot/initrd shows me a 16M initrd with piles and piles and piles of stuff I don't need. Drivers, file systems, fsck and more. Per also says he has initrds that are only 4M or 5M. Proper configuration or script? Set up the configuration properly (drakut makes this easier) and you get a properly built kernel and initrd every time :-) Of course you could always pre-empt the future and run a system that has a single unified root+/usr. That's how its going to be. That's how I'm setting up my machines from now on. And since I've been using LVM for years its easy for me to convert to that. What's that? yes I need grub2 and despite what Felix says I've not had any problems with grub2 on any of my machines, not just opensuse12 but fedora and mageia. And some of the machines are a bit .. pressed and yes they run samba and Apache and other services. There's a lot of literature out there on mkinitrd and a growing amount on drakut. The thing is that drakut is much more obvious and easy to use than mkinitrd, deceptively so. Heck, it even aborts and complains when you tell it to do something stupid! I wish more s/w were like that. -- Most of what we call management consists of making it difficult for people to get their work done. --Peter F. Drucker -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Linda Walsh

19:50

New subject: [opensuse] Re: script support and partition requirements of systemd? (was Re: system scripts using BASHism's but invoked as "#!/bin/sh")

Anton Aylward wrote:

...

The way I look at it the CORRECT way is to have an initrd that does what you want and only what you want.

What does initrd have on it that does the mounting of '/usr'? Isn't it a "script" and a copy of "mount"?

...

Proper configuration or script? Set up the configuration properly (drakut makes this easier) and you get a properly built kernel and initrd every time :-)

drakut determines what modules your kernel needs, and unless you add plugins, it won't add mounting code for /usr. According to it's mission statement, it is designed to only put the *modules* on the initrd necessary for booting your specific HW. Something else must then take control and execute some copy of 'mount' to make sure /usr is mounted.

...

Of course you could always pre-empt the future and run a system that has a single unified root+/usr. That's how its going to be.

---- Ok, but what about all the directories under /usr? How many of them also have to be mounted? /usr/homes could be a place where people put home directories. /usr/share is supposed to be arch independent material that can be 1) quite large, 2) should be shareable, 3) quite likely should be 0n a separate partition from /root and /usr as the size of /usr/share can easily be larger than the combined size of root and /usr. 15G vs. 10.6G on my system. However, reviewing my boot log just now, I see that arbitrary directories seem to be required from /usr/share for boot: udevd[602]: failed to execute '/usr/share/virtualbox/VBoxCreateUSBNode.sh' '/usr/share/virtualbox/VBoxCreateUSBNode.sh 189 512 09': No such file or directory udevd[603]: failed to execute '/usr/share/virtualbox/VBoxCreateUSBNode.sh' '/usr/share/virtualbox/VBoxCreateUSBNode.sh 189 896 09': No such file or directory udevd[604]: failed to execute '/usr/share/virtualbox/VBoxCreateUSBNode.sh' '/usr/share/virtualbox/VBoxCreateUSBNode.sh 189 256 09': No such file or directory udevd[607]: failed to execute '/usr/share/virtualbox/VBoxCreateUSBNode.sh' '/usr/share/virtualbox/VBoxCreateUSBNode.sh 189 384 09': No such file or directory udevd[608]: failed to execute '/usr/share/virtualbox/VBoxCreateUSBNode.sh' '/usr/share/virtualbox/VBoxCreateUSBNode.sh 189 768 09': No such file or directory udevd[609]: failed to execute '/usr/share/virtualbox/VBoxCreateUSBNode.sh' '/usr/share/virtualbox/VBoxCreateUSBNode.sh 189 0 09': No such file or directory udevd[610]: failed to execute '/usr/share/virtualbox/VBoxCreateUSBNode.sh' '/usr/share/virtualbox/VBoxCreateUSBNode.sh 189 128 09': No such file or directory udevd[611]: failed to execute '/usr/share/virtualbox/VBoxCreateUSBNode.sh' '/usr/share/virtualbox/VBoxCreateUSBNode.sh 189 640 09': No such file or directory udevd[619]: failed to execute '/usr/share/virtualbox/VBoxCreateUSBNode.sh' '/usr/share/virtualbox/VBoxCreateUSBNode.sh 189 769 00': No such file or directory udevd[623]: failed to execute '/usr/share/virtualbox/VBoxCreateUSBNode.sh' '/usr/share/virtualbox/VBoxCreateUSBNode.sh 189 1 09': No such file or directory udevd[631]: failed to execute '/usr/share/virtualbox/VBoxCreateUSBNode.sh' '/usr/share/virtualbox/VBoxCreateUSBNode.sh 189 3 00': No such file or directory udevd[632]: failed to execute '/usr/share/virtualbox/VBoxCreateUSBNode.sh' '/usr/share/virtualbox/VBoxCreateUSBNode.sh 189 2 00': No such file or directory ---- This means arbitrary shell scripts are needed to *cleanly* "boot" before I could even get a shell prompt. Are those "signed" kernel modules being loaded? I don't get a good feeling about the security of needing to load shell scripts off of /usr/share in order to boot. In short -- we see shell scripts being called even when booting to mode 'S'... Requiring /usr is bad enough (8.0G v. root at 4.6G). Now it used to be root was larger and /usr was smaller. But the only reason /usr needs to be mounted is the 2-4G of material that was moved off of /root into /usr. I.e. it created a problem that didn't exist: # ll -h /backups/Ishtar/root/*-0-* -rw-rw---- 1 6.7G Sep 1 04:32 /backups/Ishtar/root/root-120901-0-0430.dump -rw-rw---- 1 7.1G Oct 1 04:32 /backups/Ishtar/root/root-121001-0-0430.dump -rw-rw---- 1 7.4G Nov 1 04:31 /backups/Ishtar/root/root-121101-0-0430.dump -rw-rw---- 1 3.7G Dec 1 04:31 /backups/Ishtar/root/root-121201-0-0430.dump -rw-rw---- 1 4.1G Jan 1 04:31 /backups/Ishtar/root/root-130101-0-0430.dump # lh /backups/Ishtar/usr/*-0-* -rw-rw---- 1 6.5G Sep 1 04:35 /backups/Ishtar/usr/usr-120901-0-0432.dump -rw-rw---- 1 7.3G Oct 1 04:36 /backups/Ishtar/usr/usr-121001-0-0432.dump -rw-rw---- 1 7.1G Nov 1 04:35 /backups/Ishtar/usr/usr-121101-0-0431.dump -rw-rw---- 1 7.5G Dec 1 04:34 /backups/Ishtar/usr/usr-121201-0-0431.dump -rw-rw---- 1 7.6G Jan 1 04:35 /backups/Ishtar/usr/usr-130101-0-0431.dump As I've upgraded from 12.1->12.[23], more and more files are being moved off /root onto /usr. So let's say I combine /usr and /root. Where does it stop? Do I need all of /usr/share as well -- the largest parts being font, icons, clipart and docs: # du -sh *|hsort -s|tail -10 184M clipart 198M kde4 246M locale 338M man 714M texmf 753M icons 1.7G doc 6.5G fonts ---- ----- 12.5G TOTAL My system disk currently fits in <18G. (that doesn't include 'share'). It was sized small with a fast (15K SCSI) disk, with the idea that most larger progs could be on the LVM-managed slower-seek + faster throughput large RAID. Will /usr/local also be required for boot? That's historically been for site-local material. This is why I am asking for specifics -- when you say /usr, are you including those who have /usr/home? (likely not as that's an uncommon config, BUT, running shell scripts off of /usr/share in order to boot used to be uncommon as well. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Anton Aylward

22:44

New subject: [opensuse] Re: script support and partition requirements of systemd? (was Re: system scripts using BASHism's but invoked as "#!/bin/sh")

Linda Walsh said the following on 01/25/2013 02:50 PM:

...

Anton Aylward wrote:

...
The way I look at it the CORRECT way is to have an initrd that does what you want and only what you want.

What does initrd have on it that does the mounting of '/usr'?

Why don't you run 'lsinitrd' and find out?

...

Isn't it a "script" and a copy of "mount"?

No, its what it takes to boot the system.

...

...
Proper configuration or script? Set up the configuration properly (drakut makes this easier) and you get a properly built kernel and initrd every time :-)

drakut determines what modules your kernel needs, and unless you add plugins, it won't add mounting code for /usr. According to it's mission statement, it is designed to only put the *modules* on the initrd necessary for booting your specific HW.

Conversely you could look at it as drakut takes a lot in by default unless you tell it to exclude stuff. The config file can be viewed both ways :-) My point is that you *DO* get to choose. You specify what you want. You have stated you want /usr mounted. So that determines the module you want included. You don't want, say the XFS file system or the Btrfs file system, so you exclude those and you boot faster since you have a smaller initrd. The point is that *YOU* are in control. With the distribution kernel, which you said you were using, you have to take what you are given whether you want it or not, and if it lacks something, like mounting /usr, then tough. The decision as to what modules are part of the initrd have been taken out of your hands. You can either live with it, and have to finagle work-arounds, as you are doing, as you seem unhappy with or we wouldn't be having this thread, or you can do something about it, build a new initrd. That's mkinitrd or drakut. Either way, you are going to have to specify, somehow or other, what to include and what to exclude.

...

Something else must then take control and execute some copy of 'mount' to make sure /usr is mounted.

Do you want to take a diversion into the innards of how drakut/initrd work and how the initial boot does it stuff and pivots? I'd really rather not. You can google for all that and read it up at your leisure and emerge from that process knowing, in all probability, more about it than I do.

...

...
Of course you could always pre-empt the future and run a system that has a single unified root+/usr. That's how its going to be.

Ok, but what about all the directories under /usr?

What about them? I have /usr/share as a separate partition. I suppose that's what you mean, partitions not directories. I have no problem.

...

How many of them also have to be mounted? /usr/homes could be a place where people put home directories. /usr/share is supposed to be arch independent material that can be 1) quite large, 2) should be shareable, 3) quite likely should be 0n a separate partition from /root and /usr as the size of /usr/share can easily be larger than the combined size of root and /usr. 15G vs. 10.6G on my system.

Oh yes, there's a lot in there, all the docs and manual pages and stuff like that :-) Your point being?

...

However, reviewing my boot log just now, I see that arbitrary directories seem to be required from /usr/share for boot:

...

udevd[602]: failed to execute '/usr/share/virtualbox/VBoxCreateUSBNode.sh' '/usr/share/virtualbox/VBoxCreateUSBNode.sh 189 512 09': No such file or directory

There is the boot that initrd does and then system 'pivots' and what you probably think of boot takes over. Stop and think why those appear in your log files. To be there the logger has to be running. When does the logger start? Well it may be /etc/init.d/syslog or in my systemd-driven 11.2 its in syslog.target. So to get there the RC stuff or systemd mist be running. That's away along, after the 'pivot'. Then we get to where udevd starts running to start up virtualbox. I won't go down that avenue, but it may have something to do with other aspects of configuration.

...

---- This means arbitrary shell scripts are needed to *cleanly* "boot" before I could even get a shell prompt. Are those "signed" kernel modules being loaded? I don't get a good feeling about the security of needing to load shell scripts off of /usr/share in order to boot.

I agree with that last line and I think that is an artefact of the way you have configured your system, not a necessary way the system has to be configured. I don't start virtualbox on boot so I'm not able to comment. But I disagree with the idea that your first sentence is a necessity.

...

In short -- we see shell scripts being called even when booting to mode 'S'... Requiring /usr is bad enough (8.0G v. root at 4.6G).

Linda, all you are talking about is really after boot, after initrd has done its stuff. The issue of whether /usr should be a separate partition has been debated elsewhere. I was - still am to a degree - in favour of partitioning over OneBigFileSystem. Many reasons such as Oh-squared. But there are many things that get changed which old fogies like myself who grew up with V7 on a PDP-11 and 10M TR think the young whipper-snappers have gone wild with, things like *shock* *horror* "virtual memory" -- heck even having more than 1 megabytes of memory! But, like they say "change happens". If I can live with systems with more than I megabyte of memory I'm sure you can live with the huge drives we have today that have one partition for root and /usr/. And Gee-wow! I use LVM, so the conversion was not a big deal for me.

...

Now it used to be root was larger and /usr was smaller. But the only reason /usr needs to be mounted is the 2-4G of material that was moved off of /root into /usr.

So? If that's all, then move it back.

...

Where does it stop? Do I need all of /usr/share as well -- the largest parts being font, icons, clipart and docs:

Don't be ridiculous! Those are only needed when you go into graphics mode. Nothing to do with boot.

...

Will /usr/local also be required for boot? That's historically been for site-local material.

Why should it be?

...

This is why I am asking for specifics -- when you say /usr, are you including those who have /usr/home? (likely not as that's an uncommon config, BUT, running shell scripts off of /usr/share in order to boot used to be uncommon as well.

You seem to be confusing boot with 'going multi user' or with 'going to graphics mode'. If no user logs in then why does a /home' need to be mounted? In fact we've discussed 'roaming share' where the "home" is on the NFS server and is only mounted when the use logs in at whatever workstation he or she logs in at. As I've pointed out before, SUN were doing this back in the 1980s. If you're running sysvinit and your inittab says to go to mode 1 or 3 and you have the entry in fstab for /usr, then you have a problem with something you've configured earlier on that preventing the system getting past S. Once I had a thing like that; my / needed a good fsck and boot wasn't doing it. Maybe something similar here. Its why I don't use /etc/3 for / any more. My suspicion is that you've done something non-normal that necessitates this odd behaviour. Be it config or fsck. Yes, I'm used to fixing this kind of thing, but that's in 'admin consulting' and 'hands on' mode, where I can scan for 'suspicious changes' and all the things you're not showing us. As I say, what initrd does and init -- process #1 -- does are different. You seem confused over the two. -- Teachers open the door. You enter by yourself. Chinese Proverb -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Werner Flamme

08:01

Linda Walsh [24.01.2013 09:42]:

...

I'm constantly seeing system scripts starting with #!/bin/sh which tells BASH to run in POSIX conformance mode, yet occasionally, more or less randomly, I'll see "BASHism's" in the script.

Why don't you open a ticket (or one per script) at the bugzilla? -- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Linda Walsh

17:35

New subject: [opensuse] Re: system scripts using BASHism's but invoked as "#!/bin/sh"

Werner Flamme wrote:

...

Linda Walsh [24.01.2013 09:42]:

...
I'm constantly seeing system scripts starting with #!/bin/sh which tells BASH to run in POSIX conformance mode, yet occasionally, more or less randomly, I'll see "BASHism's" in the script.

Why don't you open a ticket (or one per script) at the bugzilla?

Why would you suggest such a huge waste of time? Berhard Voelker was able to run one command on his system that pointed to 400 scripts with problems. You really think it would be the best use of anyone's resources to open up a bug for each of those manually when the problem is a systemic one that is found with a script in 1 step? It almost seems like you don't care -- and if that's the case why would you want the bug reports filed in the first place? Is this your equivalent of "go play in the streets?... ;^/ -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Per Jessen

17:43

New subject: [opensuse] Re: system scripts using BASHism's but invoked as "#!/bin/sh"

Linda Walsh wrote:

...

Werner Flamme wrote:

...
Linda Walsh [24.01.2013 09:42]:

...
I'm constantly seeing system scripts starting with #!/bin/sh which tells BASH to run in POSIX conformance mode, yet occasionally, more or less randomly, I'll see "BASHism's" in the script.

Why don't you open a ticket (or one per script) at the bugzilla?

Why would you suggest such a huge waste of time?

He didn't - notice the 'or'.

...

Berhard Voelker was able to run one command on his system that pointed to 400 scripts with problems. You really think it would be the best use of anyone's resources to open up a bug for each of those manually when the problem is a systemic one that is found with a script in 1 step?

s/systemic/habitual/ methinks - I can only speak for myself, but I habitually also use /bin/sh even if I might be using bash-only features in a script. Is this perhaps something we can test for during build time? I doubt if we can expect much testing on systems which default to a non-bash shell, so it would be better to test during packaging.

...

It almost seems like you don't care -- and if that's the case why would you want the bug reports filed in the first place?

Just _a_ bug report. -- Per Jessen, Zürich (-2.4°C) http://www.dns24.ch/ - free DNS hosting, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Linda Walsh

20:02

New subject: [opensuse] Re: system scripts using BASHism's but invoked as "#!/bin/sh"

Per Jessen wrote:

...

Linda Walsh wrote:

...
Werner Flamme wrote:

...
Linda Walsh [24.01.2013 09:42]:

...
I'm constantly seeing system scripts starting with #!/bin/sh yet I'll see "BASHism's" in the script. Why don't you open a ticket (or one per script) at the bugzilla?

Why would you suggest such a huge waste of time?

...

Just _a_ bug report.

---- Far more reasonable: Bug 800607 Submitted system scripts should be scan'ed for bashims, and have 1st line update to bash if used -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Linda Walsh

21:42

New subject: [opensuse] Re: system scripts using BASHism's but invoked as "#!/bin/sh"

Linda Walsh wrote:

...

Per Jessen wrote:

...
Just _a_ bug report.

---- Far more reasonable:

Bug 800607 Submitted system scripts should be scan'ed for bashims, and have 1st line update to bash if used

https://bugzilla.novell.com/show_bug.cgi?id=800607 https://bugzilla.novell.com/show_bug.cgi?id=800607#c1 Guido Berhörster changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |CLOSED CC| |gber@opensuse.org Resolution| |INVALID --- Comment #1 from Guido Berhörster 2013-01-25 20:17:57 UTC --- That already exists as part of rpmlint which runs checkbashisms and dash -n on every script. Automatically changing the shebang is out of question since both are in no way reliable. =================== CLOSED/INVALID.... and you wonder why I think filing bug reports is a waste of time. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Anders Johansson

27 Jan 27 Jan

03:30

New subject: [opensuse] Re: system scripts using BASHism's but invoked as "#!/bin/sh"

On Friday 25 January 2013 13:42:17 Linda Walsh wrote:

...

Linda Walsh wrote:

...
Per Jessen wrote:

...
Just _a_ bug report.

----

Far more reasonable: Bug 800607 Submitted system scripts should be scan'ed for bashims, and have 1st line update to bash if used

https://bugzilla.novell.com/show_bug.cgi?id=800607

https://bugzilla.novell.com/show_bug.cgi?id=800607#c1

Guido Berhörster changed:

What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |CLOSED CC| |gber@opensuse.org Resolution| |INVALID

--- Comment #1 from Guido Berhörster 2013-01-25 20:17:57 UTC --- That already exists as part of rpmlint which runs checkbashisms and dash -n on every script. Automatically changing the shebang is out of question since both are in no way reliable.

=================== CLOSED/INVALID.... and you wonder why I think filing bug reports is a waste of time.

You were given a perfectly valid answer, the checks you requested are already being performed. I think the real problem is that you opened a bug for something you thought was a bug, but really isn't. Werner already pointed out that what you thought wasn't in POSIX is really in there If you find a script that fails when run with sh, which requires bash, that is a bug. "I think this requires bash" is not. especially since you apparently don't know what requires bash and what doesn't. Anders -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

Cristian Rodríguez

04:15

New subject: [opensuse] Re: system scripts using BASHism's but invoked as "#!/bin/sh"

El 27/01/13 00:30, Anders Johansson escribió:

...

I think the real problem is that you opened a bug for something you thought was a bug, but really isn't. Werner already pointed out that what you thought wasn't in POSIX is really in there

If you find a script that fails when run with sh, which requires bash, that is a bug. "I think this requires bash" is not.

Also, just for Linda's information, do not bother filling bug reports in init scripts, those will likely not get fixed. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org

4108

Age (days ago)

4111

Last active (days ago)

List overview

Download

27 comments

10 participants

participants (10)

Anders Johansson
Anton Aylward
Bernhard Voelker
Cristian Rodríguez
Dr. Werner Fink
Ken Schneider - openSUSE
Linda Walsh
Michal Hrusecky
Per Jessen
Werner Flamme