[opensuse] I thought I knew the basics - "tee" not working as I expected
Is there a way to force tee to flush it's buffers? (Or tee may not be the problem.) == details I have an interactive program I want to capture the output of. It only has one prompt, then it runs to completion It's a python app. I'm running inside konsole and my basic command is: python preg.py | tee output_file I'm not getting any output in konsole nor in the output_file until the app ends. If I run it interactively, I get the prompt fine, but all the resulting output spits up on the screen and it is multiple screens of output. I realize the program is not the greatest at creating savable output, but I didn't write it! Thanks Greg -- Greg Freemyer -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 11/12/2014 10:13 AM, Greg Freemyer wrote:
Is there a way to force tee to flush it's buffers? (Or tee may not be the problem.)
== details
I have an interactive program I want to capture the output of. It only has one prompt, then it runs to completion
It's a python app. I'm running inside konsole and my basic command is:
python preg.py | tee output_file
I'm not getting any output in konsole nor in the output_file until the app ends.
If I run it interactively, I get the prompt fine, but all the resulting output spits up on the screen and it is multiple screens of output.
I realize the program is not the greatest at creating savable output, but I didn't write it!
Two things come to mind. The first is that a pipe takes stdout and connects to stdin. if the output of a program doesn't go to stdout or if it closes and reopens its output or if it sends output to /dev/tty, then its not going to stdout. Yes it seems a crazy thing to do but there you are. The second is to use 'script' and run in a sub shell and have the screen output all saved. Or perhaps not if the idiot program decided to send to /dev/tty ... Can you look at the source? You don't need to be a python programmer to recognise something like ... Open("/dev/tty ... -- A: Yes. > Q: Are you sure? >> A: Because it reverses the logical flow of conversation. >>> Q: Why is top posting frowned upon? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Wed, Nov 12, 2014 at 10:21 AM, Anton Aylward <opensuse@antonaylward.com> wrote:
On 11/12/2014 10:13 AM, Greg Freemyer wrote:
Is there a way to force tee to flush it's buffers? (Or tee may not be the problem.)
== details
I have an interactive program I want to capture the output of. It only has one prompt, then it runs to completion
It's a python app. I'm running inside konsole and my basic command is:
python preg.py | tee output_file
I'm not getting any output in konsole nor in the output_file until the app ends.
If I run it interactively, I get the prompt fine, but all the resulting output spits up on the screen and it is multiple screens of output.
I realize the program is not the greatest at creating savable output, but I didn't write it!
Two things come to mind.
The first is that a pipe takes stdout and connects to stdin. if the output of a program doesn't go to stdout or if it closes and reopens its output or if it sends output to /dev/tty, then its not going to stdout.
Yes it seems a crazy thing to do but there you are.
The second is to use 'script' and run in a sub shell and have the screen output all saved.
Or perhaps not if the idiot program decided to send to /dev/tty ...
Can you look at the source? You don't need to be a python programmer to recognise something like
... Open("/dev/tty ...
First, tee is successfully splitting the output. it goes to both the logfile and the screen. The issue is it doesn't go either place until the app ends. Not really cool for an interactive app! Anyway, there are 400 or so python source files (*.py) and yes I can look through the source extracted from the tarball. "grep /dev/tty" turns up nothing. 'grep tty" is more interesting. But I don't think any of these are what you are looking for. Note that the app parsers lots of file types looking for info, so the parsers folder is likely not worrying with user I/O. The frontend folder is likely full of various user I/O related front-ends to the parsers. I don't know if preg.py calls plasm.py. grep -R "tty" * ./parsers/utmpx.py: construct.String('tty_name', 32), ./parsers/utmpx.py: terminal, _, _ = entry.tty_name.partition('\x00') ./parsers/utmpx.py: tty_name, _, _ = header.tty_name.partition('\x00') ./parsers/utmpx.py: if tty_name or hostname: ./parsers/utmpx_test.py: self.assertEqual(event_object.terminal, u'ttys002') ./parsers/utmpx_test.py: u'Terminal: ttys002') ./parsers/pcap.py: (i.e. HTTP Request) and the prettyfied string for the protocols. ./parsers/cups_ipp.py: pretty_name = self.NAME_PAIR_TRANSLATION.get(name, name) ./parsers/cups_ipp.py: data_dict.setdefault(pretty_name, []).append(value) ./parsers/utmp_test.py: self.assertEqual(event_object.terminal, u'tty4') ./parsers/utmp_test.py: u'Terminal: tty4 ' ./frontend/pinfo.py: self._printer = pprint.PrettyPrinter(indent=8) ./frontend/plasm.py: eventtype_filename = ".{0:s}_evtt_{1:s}".format( ./frontend/plasm.py: if os.path.isfile(eventtype_filename): ./frontend/plasm.py: x = open(eventtype_filename, 'rb') ./frontend/plasm.py: evttypes = pickle.load(x) ./frontend/plasm.py: evttype_indices = pickle.load(x) ./frontend/plasm.py: evttype_candidates = {} ./frontend/plasm.py: if candidate in evttype_candidates: ./frontend/plasm.py: evttype_candidates[candidate] += 1 ./frontend/plasm.py: evttype_candidates[candidate] = 1 ./frontend/plasm.py: evttypes = [] ./frontend/plasm.py: evttype_indices = {} ./frontend/plasm.py: for candidate, score in evttype_candidates.iteritems(): ./frontend/plasm.py: evttype_indices[candidate] = len(evttypes) ./frontend/plasm.py: evttypes.append(candidate) ./frontend/plasm.py: del evttype_candidates ./frontend/plasm.py: x = open(eventtype_filename, 'wb') ./frontend/plasm.py: pickle.dump(evttypes, x) ./frontend/plasm.py: pickle.dump(evttype_indices, x) ./frontend/plasm.py: return (evttypes, evttype_indices) Thanks Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 11/12/2014 10:45 AM, Greg Freemyer wrote:
Yes it seems a crazy thing to do but there you are.
Hmm. It seems it wasn't _that_ crazy. -- The most important thing in communication is to hear what isn't being said. --Peter F. Drucker -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 11/12/2014 04:13 PM, Greg Freemyer wrote:
Is there a way to force tee to flush it's buffers? (Or tee may not be the problem.)
== details
I have an interactive program I want to capture the output of. It only has one prompt, then it runs to completion
It's a python app. I'm running inside konsole and my basic command is:
python preg.py | tee output_file
Have you try stdbuf(1)? $ stdbuf --output=0 python preg.py | tee output_file If that doesn't help (read about the limitations in the Texinfo manual): I don't know the python interpreter, but e.g. for perl there is a common command to enable auto-flushing: # set autoflush for STDOUT $| = 1; Maybe there's something similar for python. Have a nice day, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Wed, Nov 12, 2014 at 10:32 AM, Bernhard Voelker <mail@bernhard-voelker.de> wrote:
On 11/12/2014 04:13 PM, Greg Freemyer wrote:
Is there a way to force tee to flush it's buffers? (Or tee may not be the problem.)
== details
I have an interactive program I want to capture the output of. It only has one prompt, then it runs to completion
It's a python app. I'm running inside konsole and my basic command is:
python preg.py | tee output_file
Have you try stdbuf(1)?
$ stdbuf --output=0 python preg.py | tee output_file
You are the magic man. It works perfectly! Thanks Greg -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 11/12/2014 05:38 PM, Greg Freemyer wrote:
On Wed, Nov 12, 2014 at 10:32 AM, Bernhard Voelker <mail@bernhard-voelker.de> wrote:
$ stdbuf --output=0 python preg.py | tee output_file
It works perfectly!
Thanks for the feedback. This is one of the not-so-prominent programs of the coreutils package, so it's nice it helped. Have a nice day, Berny -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Quoting Greg Freemyer <greg.freemyer@gmail.com>:
Is there a way to force tee to flush it's buffers? (Or tee may not be the problem.)
== details
I have an interactive program I want to capture the output of. It only has one prompt, then it runs to completion
It's a python app. I'm running inside konsole and my basic command is:
python preg.py | tee output_file
I'm not getting any output in konsole nor in the output_file until the app ends.
Have you tried running the program in a "screen" session and logging the session to a file? This is how my programming students turned in their homework, log to a file, run the program, print the log, and turn it in. I no longer remember the details, but from "man screen", it looks like "screen -L" will do the job. HTH, Jeffrey -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
participants (4)
-
Anton Aylward -
Bernhard Voelker -
Greg Freemyer -
Jeffrey L. Taylor