Correlation problem between stdout/stderr when output is redirected to a log file

[nachtgeist]

Hi,

I like running i-b like i-b $params 2>&1 | tee logfile. Problem is, if an error occurs, a possible message on stderr (like from a chroot script) or the python stack trace is not placed below the preceding informational output on stdout but usually several dozens of lines earlier. This makes it pretty hard to make sense of such a log - especially with chroot scripts, which produce a lot of output by themselves.

I could solve the problem by running i-b under python -u. Would you consider changing the default shebang?

[hartwork]

Hi!

Changing the default for everyone does not sound like a good idea to me right now. (Also, we cannot append to #! /usr/bin/env python2 as is because #! /usr/bin/env python2 -u will pass python2 -u as a single parameter to env. env has -S but I'm not sure yet how portable that would be.)

I'd be up for a new command line parameter --unbuffered that modifies stdout and stderr accordingly.

[nachtgeist]

Good point! I wasn't aware of env's single-parameter issue (just looked up the -S support and it's contained only starting with buster btw). I was looking into something like --unbuffered prior to python -u as well and found it to be an ugly PITA. I guess the simplest thing would be for me to call python2 -u image-bootstrap in my wrapper-script.

Though, a short section about this issue in the README might me useful. I'll submit a PR for that in a bit.

[hartwork]

I think rather than python2 -u image-bootstrap you want python2 -u -m image-bootstrap with -m.

[hartwork]

PS: Here's one more hack we might have been able to abuse for this purpose from inside a running process: https://github.com/hartwork/jawanndenn/blob/master/jawanndenn/__main__.py#L40