[opensuse] systemd - zgrep -m1 gives me "gzip: stdout: Broken pipe" ?
Okay, I need another pair of eyes - in a bash script, I have "zgrep -m1 SOA file.gz". The script is being run under systemd. The line with SOA is found, but I always get "gzip: stdout: Broken pipe". zgrep is obviously "gzip -dc | grep". I guess grep closes early when the first line is found, but why does this produce the error when running under systemd? If I remove the '-m1', no problem, no message. In the console, if I run the "zgrep -m1" manually or from within a script, no problem, no message. The script itself (suitably redacted) is nothing much: #!/bin/sh sec=47 addr=1.2.3.4 name=myscript version=1.1 echo "$name started, version $version, cwd=$PWD" trap "echo \"$name terminated\"" exit while true do a=$(date +%S) a=${a#0} w=$(((60+sec-a)%60)) sleep $w echo "running rsync ..." rsync -rvt -q -4 --address=$addr etc etc zgrep -m1 SOA newfile.gz make -r -C /srv/rbldnsd 2>&1 >>updatelog sleep 1 done -- Per Jessen, Zürich (3.9°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Per Jessen wrote:
Okay, I need another pair of eyes -
in a bash script, I have "zgrep -m1 SOA file.gz". The script is being run under systemd. The line with SOA is found, but I always get "gzip: stdout: Broken pipe".
zgrep is obviously "gzip -dc | grep".
I guess grep closes early when the first line is found, but why does this produce the error when running under systemd? If I remove the '-m1', no problem, no message.
Adding '2>/dev/null' to the zgrep gets rid of the message, and I guess StandardError=null in the service unit would do the same, but why is the behaviour different when running under systemd ? -- Per Jessen, Zürich (3.9°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 11/29/2017 11:01 AM, Per Jessen wrote:
Per Jessen wrote:
Okay, I need another pair of eyes -
in a bash script, I have "zgrep -m1 SOA file.gz". The script is being run under systemd. The line with SOA is found, but I always get "gzip: stdout: Broken pipe".
zgrep is obviously "gzip -dc | grep".
I guess grep closes early when the first line is found, but why does this produce the error when running under systemd? If I remove the '-m1', no problem, no message.
Adding '2>/dev/null' to the zgrep gets rid of the message, and I guess StandardError=null in the service unit would do the same, but why is the behaviour different when running under systemd ?
Maybe SIGPIPE is ignored / not ignored there? Compare $ ( trap '' PIPE; seq inf | sed 1q ) 1 seq: write error: Broken pipe $ ( seq inf | sed 1q ) 1 Have a nice day, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Bernhard Voelker wrote:
On 11/29/2017 11:01 AM, Per Jessen wrote:
Per Jessen wrote:
Okay, I need another pair of eyes -
in a bash script, I have "zgrep -m1 SOA file.gz". The script is being run under systemd. The line with SOA is found, but I always get "gzip: stdout: Broken pipe".
zgrep is obviously "gzip -dc | grep".
I guess grep closes early when the first line is found, but why does this produce the error when running under systemd? If I remove the '-m1', no problem, no message.
Adding '2>/dev/null' to the zgrep gets rid of the message, and I guess StandardError=null in the service unit would do the same, but why is the behaviour different when running under systemd ?
Maybe SIGPIPE is ignored / not ignored there?
Good suggestion, that sounds likely - on the console: (trap '' pipe; zgrep -m1 SOA file.gz ) $SOA 180 dev.null zone.jessen.ch 1511967099 180 180 604800 180 gzip: stdout: Broken pipe So I ought to be able to get rid of that message by adding trap /bin/true pipe to my script, I would think? -- Per Jessen, Zürich (3.7°C) http://www.dns24.ch/ - free dynamic DNS, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Per Jessen wrote:
Bernhard Voelker wrote:
On 11/29/2017 11:01 AM, Per Jessen wrote:
Per Jessen wrote:
Okay, I need another pair of eyes -
in a bash script, I have "zgrep -m1 SOA file.gz". The script is being run under systemd. The line with SOA is found, but I always get "gzip: stdout: Broken pipe".
zgrep is obviously "gzip -dc | grep".
I guess grep closes early when the first line is found, but why does this produce the error when running under systemd? If I remove the '-m1', no problem, no message.
Adding '2>/dev/null' to the zgrep gets rid of the message, and I guess StandardError=null in the service unit would do the same, but why is the behaviour different when running under systemd ?
Maybe SIGPIPE is ignored / not ignored there?
Good suggestion, that sounds likely -
on the console: (trap '' pipe; zgrep -m1 SOA file.gz ) $SOA 180 dev.null zone.jessen.ch 1511967099 180 180 604800 180 gzip: stdout: Broken pipe
So I ought to be able to get rid of that message by adding
trap /bin/true pipe
to my script, I would think?
I'll just stick to the 2>/dev/null :-) -- Per Jessen, Zürich (3.5°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 11/29/2017 03:57 PM, Per Jessen wrote:
So I ought to be able to get rid of that message by adding
trap /bin/true pipe
trap -- PIPE or maybe change the systemd unit (man systemd.exec): IgnoreSIGPIPE= Takes a boolean argument. If true, causes SIGPIPE to be ignored in the executed process. Defaults to true because SIGPIPE generally is useful only in shell pipelines. Have a nice day, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Bernhard Voelker wrote:
On 11/29/2017 03:57 PM, Per Jessen wrote:
So I ought to be able to get rid of that message by adding
trap /bin/true pipe
trap -- PIPE
or maybe change the systemd unit (man systemd.exec):
IgnoreSIGPIPE= Takes a boolean argument. If true, causes SIGPIPE to be ignored in the executed process. Defaults to true because SIGPIPE generally is useful only in shell pipelines.
Thanks Berny, I hadn't read that far in the man page :-) -- Per Jessen, Zürich (2.4°C) http://www.hostsuisse.com/ - dedicated server rental in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 11/29/2017 05:37 PM, Per Jessen wrote:
I hadn't read that far in the man page:-)
I just saw it by chance because I'm searching for something different there ... Have a nice day, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wednesday, 2017-11-29 at 10:51 +0100, Per Jessen wrote:
Okay, I need another pair of eyes -
in a bash script, I have "zgrep -m1 SOA file.gz". The script is being run under systemd. The line with SOA is found, but I always get "gzip: stdout: Broken pipe".
zgrep is obviously "gzip -dc | grep".
Here it is a 235 lines bash script. Are you sure it resolves to that? You could edit the script and add a -x to find out what it really does in your case. - -- Cheers, Carlos E. R. (from openSUSE 42.2 x86_64 "Malachite" at Telcontar) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAloes9UACgkQtTMYHG2NR9UjJgCfQLC/rAz4EGWcFOqkR1Z6gLKJ gW0Anif2jEUhibvoXb6PdeaDbOiZwTpn =qK/g -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Carlos E. R. wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Wednesday, 2017-11-29 at 10:51 +0100, Per Jessen wrote:
Okay, I need another pair of eyes -
in a bash script, I have "zgrep -m1 SOA file.gz". The script is being run under systemd. The line with SOA is found, but I always get "gzip: stdout: Broken pipe".
zgrep is obviously "gzip -dc | grep".
Here it is a 235 lines bash script. Are you sure it resolves to that?
I am fairly positive, yes. Even if it doesn't, what I want is still "gzip -dc | grep -m1" and that produces the message. -- Per Jessen, Zürich (4.1°C) http://www.hostsuisse.com/ - virtual servers, made in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
29.11.2017 12:51, Per Jessen пишет:
Okay, I need another pair of eyes -
in a bash script, I have "zgrep -m1 SOA file.gz". The script is being run under systemd. The line with SOA is found, but I always get "gzip: stdout: Broken pipe".
zgrep is obviously "gzip -dc | grep".
I guess grep closes early when the first line is found, but why does this produce the error when running under systemd? If I remove the '-m1', no problem, no message.
In the console, if I run the "zgrep -m1" manually or from within a script, no problem, no message.
If you look with strace you will see that gzip *does* get SIGPIPE. And if you look at signal mask, you will also see that programs started by systemd have SIGPIPE ignored (by default). And gzip will leave such signal ignored as well. Which means, under shell gzip gets SIGPIPE and silently exits from within signal handler (because it is normally not an error, at least, something outside of program control), while under systemd it never sees SIGPIPE in the first place, so write function will print this error and abort. So here we have situation when program *expects* and handles SIGPIPE; hiding it from the program actually makes things worse :) -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Andrei Borzenkov wrote:
29.11.2017 12:51, Per Jessen пишет:
Okay, I need another pair of eyes -
in a bash script, I have "zgrep -m1 SOA file.gz". The script is being run under systemd. The line with SOA is found, but I always get "gzip: stdout: Broken pipe".
zgrep is obviously "gzip -dc | grep".
I guess grep closes early when the first line is found, but why does this produce the error when running under systemd? If I remove the '-m1', no problem, no message.
In the console, if I run the "zgrep -m1" manually or from within a script, no problem, no message.
If you look with strace you will see that gzip *does* get SIGPIPE. And if you look at signal mask, you will also see that programs started by systemd have SIGPIPE ignored (by default).
Right, that is what Bernard wrote about IgnoreSIGPIPE= etc.
And gzip will leave such signal ignored as well. Which means, under shell gzip gets SIGPIPE and silently exits from within signal handler (because it is normally not an error, at least, something outside of program control), while under systemd it never sees SIGPIPE in the first place, so write function will print this error and abort.
aha! Thanks for taking the time to explain that, I couldn't quite work it out.
So here we have situation when program *expects* and handles SIGPIPE; hiding it from the program actually makes things worse :)
Hehe, yeah. So I were to add "IgnoreSIGPIPE=false", my script would presumably work as expected. -- Per Jessen, Zürich (2.3°C) http://www.cloudsuisse.com/ - your owncloud, hosted in Switzerland. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (4)
-
Andrei Borzenkov -
Bernhard Voelker -
Carlos E. R. -
Per Jessen