Hello, On Tue, 22 May 2012, David C. Rankin wrote:
On 05/22/2012 08:56 AM, David Haller wrote:
Confused yet?
Yes - thoroughly...
Digesting the wisdom, it seems that there is a difference in how calc is treating stdin depending on whether it is called from within a while loop or otherwise.
Nope, calc seems to _always_ read stdin.
The bug/bag example was great exposing what calc was choking on. But I can't for the life of me see how calc can inherit stdin differently when called within a while loop or for loop.
inside a 'while ...; done <file;' loop, stdin is 'file'. For 'read' as well as for any other process inside the loop. Instead of using 'while read' you could also use: $ unset done; while test "$done" != "done"; do read x || done="done"; echo "x=$x; done=$done"; \ done < <(printf "bar %s\n" $(seq 4)) x=bar 1; done= x=bar 2; done= x=bar 3; done= x=bar 4; done= x=; done=done Now, if you add there another command that reads it's stdin, e.g. calc, what happens? $ unset done; while test "$done" != "done"; do \ read x || done="done"; calc -p "21+21"; echo "x=$x; done=$done";\ done < <(printf "bar %s\n" $(seq 4)) 42 "bar" is undefined x=bar 1; done= 42 x=; done=done
I see in the for loop there is no stdin to inherit -- I get that.
Exactly. That's the point why it works in the for-loop.
I also see in the while loop that the potential is there due to redirecting gtkrc-file.txt to feed the while loop -- but that is where things cloud over and my eyes roll back in my head.
See above: inside 'while ... done' all processes share the same stdin unless stdin is redirected. The read immediately after while, inside the loop or any other program. Compare: $ printf "bar %s\n" $(seq 8) | while read x; read y; do \ read z; echo "x=$x; y=$y; z=$z"; \ done x=bar 1; y=bar 2; z=bar 3 x=bar 4; y=bar 5; z=bar 6 x=bar 7; y=bar 8; z=
Where the train wreck occurs in my mind is that regardless of while or for, the line is passed into variable 'l' which is parsed into 'rgb' and then into 'r', 'g' & 'b' --before-- being fed to calc. How does calc even know of a different stdin? Passing ' </dev/null' to calc does indeed fix the problem, but I would never have expected there to be some stdin cause after the closing parenthesis in the original call: _r=$(calc -p "255 * $r"). How is there some stdin left open after the closing '"' in $(calc -p "255 * $r")?
Nope, 'read l' and calc _both_ read the same stdin, as demonstrated above. UNLESS you redirect calc's stdin, it while also read stdin and thus read the file. Again: $ printf "bar %s\n" $(seq 8) | while read x; read y; do \ z=$(calc -p '21+21'); echo "x=$x; y=$y; z=$z"; \ done x=bar 1; y=bar 2; z=bar 3 x=bar 4; y=bar 5; z=bar 6 x=bar 7; y=bar 8; z= $ printf "bar %s\n" $(seq 8) | while read x; read y; do \ z=$(calc -p '21+21'); echo "x=$x; y=$y; z=$z"; \ done "bar" is undefined x=bar 1; y=bar 2; z=42 And stdin 'is empty', calc has read all lines, read x doesn't see any more lines: $ printf "bar %s\n" $(seq 8) | while read x; read y; do \ z=$(strace -s 40 -eread calc -p '21+21'); echo "x=$x; y=$y; z=$z"; \ done [more reads of ELF binaries snipped and pruned] read(3, "\177ELF..., 832) = 832 read(0, "bar 3\nbar 4\nbar 5\nbar 6\nbar 7\nbar 8\n", 4096) = 36 "bar" is undefined x=bar 1; y=bar 2; z=42 As you can see in the last 'read', calc read 'fd 0', i.e. stdin upto and _including_ the last line containing 'bar 8\n'. Tries to parse that stuff and barfs on the 'bar 3' on line 3 ('read x' and 'read y' read lines 1 and 2, as you can see in the 'echo' output).
Just to really confuse myself, I decided to re-write the script using a for loop instead of a while loop (I prefer while loops anyway because they will always read the last line of the file...) So:
#!/bin/bash
IFS=$'\n'
for l in $(<gtkrc-file.txt); do
With the IFS, that should work. It'll break if IFS you ever forget to set IFS ;) [..]
_r=$(calc -p "255 * $r") _g=$(calc -p "255 * $g") _b=$(calc -p "255 * $b")
Again: you could also use bc (that read _either_ stdin or it's arguments ;), redirect calc's stdin or feed calc's stdin yourself using echo ('_r=$(echo "255 * $r" | calc -p)') etc.
It works without a hitch! So, I learned something new -- even though I'm not sure I understand what I learned. BASH handles while and for differently somehow regarding the inheritance of stdin.
With 'for l in $(< ... ); do' there is no stdin redirected, stdin is the stdin of the script, and calc will _still_ read that! The part inside the '{ }' replaces your script! $ printf "bar %s\n" $(seq 4) | { \ IFS=$'\n'; \ for x in $(seq 10 14); do \ z=$(calc -p "$x+1"); \ echo "x=$x; z=$z"; \ done; } "bar" is undefined x=10; z=11 x=11; z=12 x=12; z=13 x=13; z=14 x=14; z=15 But: the for-loop doesn't abort once calc has read stdin (i.e. the 'printf 'bar %s\n' $(seq 4)' output), as the for loop does not depend on stdin.
I can prove to myself that there is indeed this difference, but other than stumbling across it on a case-by-case basis, I'm certain I still do not understand a 'rule' that tells me I when should look out for this type of problem before I run into it... :)
You should have a look what reads stdin. Using calc, you probably should do this in your script: #!/bin/bash exec </dev/null IFS=$'\n' for l in $(<gtkrc-file.txt); do .... Compare: $ printf "bar %s\n" $(seq 4) | { exec </dev/null; IFS=$'\n'; \ for x in $(seq 10 14); do \ z=$(calc -p "$x+1"); echo "x=$x; z=$z"; done; } x=10; z=11 x=11; z=12 x=12; z=13 x=13; z=14 x=14; z=15 That way, you redirect your script's stdin from /dev/null, and thus 'calc' can't read anything from there that might confuse it.
Ouch -- my head hurts... dnh, all, I tip my hats to you in your ability to step through and debug this. That isn't something I've mastered yet. Thanks.
Well, it's a lot of basic(s) understanding, years of experience and not being afraid use set -x/set -v, strace and ltrace and wade through their output. Oh, and if neccessary, to have a look into the sources ;) Coming up with a solution is often not that difficult, but understanding _why_ something happens, what's the cause and the correct fix of something, that's taking the extra work. As evidenced in this thread. There were fast a couple of workarounds. But I hope in my mails I could explain cause and effects (and how you can diagnose that stuff). Still confused? Play around with my samples a bit (replacing my seq stuff by files if you care). Exursion: Play around with 'strace'. Maybe use strace -f -s 128 -eread,file,process -o yourscript.strace bash ./yourscript and look for strings from your input file in that strace, like that 'read(0, ...)' above, look from what process id that read is, search back to a 'fork' ... For example, using the second to last example without the exec </dev/null where calc barfs on the 'bar'[1] in a bash -c (quoting adjusted), strace tuned to be more exact: $ strace -f -s 40 -eread,execve bash -c \ 'printf "bar %s\n" $(seq 4) | { IFS=$'"'"\n"'"'; \ for x in 10 11 12 13 14; do \ z=$( calc -p "$x+1"); \ echo "x=$x; z=$z"; \ done; }' I find rather early: [pid 32143] read(0, "bar 1\nbar 2\nbar 3\nbar 4\n", 4096) = 24 "bar" is undefined Going back from there, looking for 'execve', I find: [pid 32143] execve("/usr/bin/calc", ["calc", "-p", "10+1"], [/* 131 vars */] <unfinished ...> So, pid 32143 is 'calc' reading my stdin ;) Thus I _know_ that calc has gobbled up the printf+seq 4 output from stdin ... In the normal '-e process' strace, you'll have either a 'fork' or a 'clone' call a couple of lines before the 'execve', that's where the process is created, the 'execve' then starts it. Following that, I find the next call of calc (execve): [pid 32145] execve("/usr/bin/calc", ["calc", "-p", "11+1"], [/* 131 vars */]) = 0 [pid 32145] read(0, "", 4096) = 0 Process 32144 resumed Again: proof that calc reads stdin, no matter what. Nassssty calc! ;) HTH, and ask if you're still (or again) confused, -dnh [1] I wonder how this might gets people using a search-engine to find this mail ;) -- $SUPPLIER said "next day delivery". Unfortunately, they didn't specify which day it would be next to. -- James Cort -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org