test-stdio-closed failing on some AIX environments

[sxa]

Reference: #8375 (FYI @gireeshpunathil @Trott @mhdawson)
Happening on current versions of node 4 and 6 (4.7.0/6.9.2) and on AIX 6.1TL09

The fix that was applied in e65a2d7 does not appear to make the test pass universally on all AIX systems. I have a mix of some that it works for, and others which are still returning "42" which is consistent with the other platforms, so is failing this assertion:

assert.strictEqual(exitCode, common.isAix ? 126 : 42);

On the system that it works on, if I set the LANG variable to anything other the en_US I get the "42" result (not certain yet why that is making a difference) although on one of my "failing" machines which is giving 42 setting it to en_US doesn't resolve it, despite it being in the output of locale -a. We could, potentially, be seeing a timing issue where going down the path of a non-default locale is changing things, but I'm guessing so far at this point.

The AIX level on the "failing" box (the one defaulting to en_GB which is always giving me 42) is patched to a slightly later level AIX level - .6100-09-08-1642 vs 6100-09-07-1614. On my 6100-07 system (defaults to en_US) it's always returning 42 as well.

I get the feeling we may need to be able to trap both cases for this test to pass reliably (or change the exit(126) to exit(42), or understand why we're getting the differences that caused the initial patch to be required.

[Trott]

It would be great to track down exactly why it sometimes works one way on AIX and sometimes works another way. But seeing as both behaviors are spec-compliant (I think), I'd be OK with replacing:

assert.strictEqual(exitCode, common.isAix ? 126 : 42);

...with something like:

if (common.isAix)
  assert([42, 126].includes(exitCode));
else
  assert.strictEqual(exitCode, 42);

I agree, though, that tracking down the source of the behavior difference would be a superior solution...

[sxa]

Sounds reasonable, and it means we'll spot if the same issue shows up on another OS since the acceptance of 126 is still in an AIX-specific clause. I've checked it and it seems ok so I'll get that submitted, especially since I don't think Gireesh will be around for a while who was involved in the previous discussion on this topic.

[jBarz]

node [args] 1>&-

is supposed to start node with fd=1 (stdout) closed and not open. (upon which node will reopen it to point to /dev/null)
However it is open which is why fstat returns a pass.
More weirdly, fd=1 is open on the following catalog file
/usr/lib/nls/msg/en_US/execerr.cat

Renaming the catalog file to something else produces the expected outcome (i.e. fd=1 is closed).
I think this is a bug in the shell (ksh) on aix since the documentation for ksh states that 1>&- must close fd=1.

[Trott]

So, on AIX, /bin/sh is ksh? If it's ksh-specific, might something like this work around the issue until there's a fix for the bug?

Instead of this:

const proc = spawn('/bin/sh', ['-c', cmd], { stdio: 'inherit' });

...maybe something like this?:

// Insert comment here explaining the bug so that the next line is not a mystery.
const shell = common.isAIX ? '/bin/bash' : '/bin/sh';
const proc = spawn(shell, ['-c', cmd], { stdio: 'inherit' });

[Trott]

Another possibility: If we identify the version of ksh that has this bug and when it is fixed, check the version of ksh before running the test and skip if we are using a buggy version?

[jBarz]

Unfortunately, bash fails the same as ksh. However "csh" does the job.
We could use change it to csh.

Another possible fix (more comprehensive i think) is to do the following

If stdout is open and it is read-only (which is the case on aix/ksh/bash), then we assume that the user originally intended to close stdout. So, we

• close it
• reopen it on /dev/null

jBarz@25d758d
This produces consistent behavior on both aix machines that originally produced different behavior.
This also means that the test case would be the same for aix and other platforms.

[jBarz]

I can submit a PR for the above suggestion if it seems valid to you guys.

[richardlau]

On the system that it works on, if I set the LANG variable to anything other the en_US I get the "42" result (not certain yet why that is making a difference) although on one of my "failing" machines which is giving 42 setting it to en_US doesn't resolve it, despite it being in the output of locale -a. We could, potentially, be seeing a timing issue where going down the path of a non-default locale is changing things, but I'm guessing so far at this point.

Might be a good guess, based on:

More weirdly, fd=1 is open on the following catalog file
/usr/lib/nls/msg/en_US/execerr.cat

Renaming the catalog file to something else produces the expected outcome (i.e. fd=1 is closed).
I think this is a bug in the shell (ksh) on aix since the documentation for ksh states that 1>&- must close fd=1.

I'm wondering if the shell did close fd=1 but something has then opened the catalog file before node checks if fd=1 is closed. The contents of /usr/lib/nls/msg/en_US/execerr.cat on the AIX systems I have access to look to be error messages relating to execing a process (probably should have guessed based on the filename 😁 ) -- Perhaps the shell is actually reporting an error (which if it is might be tricky to get at since we've closed stdout and stderr)? Maybe running with truss could help?

[richardlau]

Perhaps the shell is actually reporting an error (which if it is might be tricky to get at since we've closed stdout and stderr)?

Actually thinking a bit more we only close stdout and stderr for the node process -- The shell's stdout and stderr should be being piped back to the parent (the test) so we could probably capture it in the test (maybe assert the output is empty?).

[Trott]

@jBarz wrote:

Another possible fix (more comprehensive i think) is to do the following

If stdout is open and it is read-only (which is the case on aix/ksh/bash), then we assume that the user originally intended to close stdout. So, we

• close it
• reopen it on /dev/null

jBarz@25d758d

/cc @bnoordhuis