[opensuse-kernel] script failure
I have encountered a bug when running a certain script.
The following is the simplest script I can make that
triggers the bug:
1) create a cpio archive:
gzip -dc /boot/initrd >initrd.cpio
2) run this script:
if cpio --quiet -t
On Tuesday 04 of December 2012 13:05EN, Giacomo Comes wrote:
But if I add the option -q to grep: if cpio --quiet -t
&2 ; else echo false-$? >&2 ; fi | grep -q libz the output is: false-141 instead of the correct one: true-0
Return code 141 means the process was killed by signal 141 - 128 = 13 which is SIGPIPE. And grep(1) tells you that with "-q", grep exits immediately as soon as it finds a match. Unfortunately cpio doesn't handle SIGPIPE so it fails when grep exits. You can either try to report it as a cpio bug or use a simple workaround like if ... fi | cat | grep -q libz Anyway, this is definitely not a kernel problem. Michal Kubeček -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On Wed, Dec 05, 2012 at 08:11:24AM +0100, Michal Kubeček wrote:
On Tuesday 04 of December 2012 13:05EN, Giacomo Comes wrote:
But if I add the option -q to grep: if cpio --quiet -t
&2 ; else echo false-$? >&2 ; fi | grep -q libz the output is: false-141 instead of the correct one: true-0
Return code 141 means the process was killed by signal 141 - 128 = 13 which is SIGPIPE. And grep(1) tells you that with "-q", grep exits immediately as soon as it finds a match. Unfortunately cpio doesn't handle SIGPIPE so it fails when grep exits.
You can either try to report it as a cpio bug or use a simple workaround like
if ... fi | cat | grep -q libz
Anyway, this is definitely not a kernel problem.
There is certanly a SIGPIPE issue here, but I'm not sure cpio is the one
to blame, or at least the only one, for the following reasons:
1) the script fails if I run it on openSUSE 12.2 using the cpio binary from 11.3, but
it doesn't fail if I run it on openSUSE 11.3 using the cpio binary from 12.2
(the version of cpio is 2.11 since openSUSE 11.3)
2) simply running:
cpio --quiet -t
On Wed, Dec 05, 2012 at 11:22:07AM -0400, Giacomo Comes wrote:
There is certanly a SIGPIPE issue here, but I'm not sure cpio is the one to blame, or at least the only one, for the following reasons:
1) the script fails if I run it on openSUSE 12.2 using the cpio binary from 11.3, but it doesn't fail if I run it on openSUSE 11.3 using the cpio binary from 12.2 (the version of cpio is 2.11 since openSUSE 11.3)
I guess it is up to you to find out what exactly is different. Maybe different contents of initrd, maybe different pipe buffer size. Using strace on cpio may help a lot.
2) simply running: cpio --quiet -t
...and so does false | true ; echo $? In both cases it doesn't say anything about the return code of first command in the pipeline. See bash(1): "The return status of a pipeline is the exit status of the last command, unless the pipefail option is enabled."
If cpio doesn't handle SIGPIPE it should fail here as well.
...and it most likely does. Michal Kubeček -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
Le mercredi 05 décembre 2012 à 19:41 +0100, Michal Kubecek a écrit :
On Wed, Dec 05, 2012 at 11:22:07AM -0400, Giacomo Comes wrote:
There is certanly a SIGPIPE issue here, but I'm not sure cpio is the one to blame, or at least the only one, for the following reasons:
1) the script fails if I run it on openSUSE 12.2 using the cpio binary from 11.3, but it doesn't fail if I run it on openSUSE 11.3 using the cpio binary from 12.2 (the version of cpio is 2.11 since openSUSE 11.3)
I guess it is up to you to find out what exactly is different. Maybe different contents of initrd, maybe different pipe buffer size. Using strace on cpio may help a lot.
2) simply running: cpio --quiet -t
...and so does
false | true ; echo $?
In both cases it doesn't say anything about the return code of first command in the pipeline. See bash(1): "The return status of a pipeline is the exit status of the last command, unless the pipefail option is enabled."
Note that you can access the status of every command in a pipe using ${PIPESTATUS[0]}, ${PIPESTATUS[1]} etc. -- Jean Delvare Suse L3 -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On Wed, Dec 05, 2012 at 07:41:45PM +0100, Michal Kubecek wrote:
On Wed, Dec 05, 2012 at 11:22:07AM -0400, Giacomo Comes wrote:
There is certanly a SIGPIPE issue here, but I'm not sure cpio is the one to blame, or at least the only one, for the following reasons:
1) the script fails if I run it on openSUSE 12.2 using the cpio binary from 11.3, but it doesn't fail if I run it on openSUSE 11.3 using the cpio binary from 12.2 (the version of cpio is 2.11 since openSUSE 11.3)
I guess it is up to you to find out what exactly is different. Maybe different contents of initrd, maybe different pipe buffer size. Using strace on cpio may help a lot.
2) simply running: cpio --quiet -t
...and so does
false | true ; echo $?
In both cases it doesn't say anything about the return code of first command in the pipeline. See bash(1): "The return status of a pipeline is the exit status of the last command, unless the pipefail option is enabled."
If cpio doesn't handle SIGPIPE it should fail here as well.
...and it most likely does.
Right. I run the following command: gzip -dc /boot/initrd | cpio --quiet -t | grep -q libz ; echo ${PIPESTATUS[*]} and here are the results: openSUSE 11.3: 0 0 0 openSUSE 11.4: 0 141 0 openSUSE 12.1: 141 141 0 openSUSE 12.2: 141 141 0 I'm unable to find where the bug is. The only thing I'm sure is that because of it some shell script that used to work now fail. Giacomo -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On Friday 07 of December 2012 11:57EN, Giacomo Comes wrote:
Right. I run the following command: gzip -dc /boot/initrd | cpio --quiet -t | grep -q libz ; echo ${PIPESTATUS[*]} and here are the results:
openSUSE 11.3: 0 0 0 openSUSE 11.4: 0 141 0 openSUSE 12.1: 141 141 0 openSUSE 12.2: 141 141 0
I'm unable to find where the bug is. The only thing I'm sure is that because of it some shell script that used to work now fail.
Change of the result doesn't necessarily mean a regression. The fact that "grep -q" finishes as soon as it finds a match, is a documented behaviour. If you write to a pipe and reader finishes, you get SIGPIPE, that's standard behaviour. If you are lucky and the output is short, you can avoid being shut down by SIGPIPE thanks to buffering but the only reliable way to avoid it is to ignore or handle it. And I seriously doubt gzip and/or cpio handled/ignored SIGPIPE in older versions and they stopped to do so. It is much more likely that you were just lucky not to hit the problem earlier. If you want to avoid it reliably, use the workaround suggested by Jiri Bohac, i.e. redirecting output of grep to /dev/null instead of "-q". Anyway, this discussion is definitely off-topic in opensuse-kernel list for some time. So far I haven't seen anything indicating this is a kernel problem. Michal Kubeček -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On Wednesday 05 of December 2012 08:11EN, Michal Kubeček wrote:
You can either try to report it as a cpio bug or use a simple workaround like
if ... fi | cat | grep -q libz
For the record, the fact that this worked for me was just a piece of luck that thanks to buffering, cat didn't finish before cpio passed all its output and finished. In general, this doesn't help as cat closes its input and quits when it receives SIGPIPE. Michal Kubeček -- To unsubscribe, e-mail: opensuse-kernel+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-kernel+owner@opensuse.org
On Thu, Dec 06, 2012 at 10:56:45AM +0100, Michal Kubeček wrote:
In general, this doesn't help as cat closes its input and quits when it receives SIGPIPE.
... so I think the best solution is:
if ... fi | grep libz > /dev/null
instead of grep -q
--
Jiri Bohac
participants (5)
-
Giacomo Comes -
Jean Delvare -
Jiri Bohac -
Michal Kubecek -
Michal Kubeček