regression: child_process stdin pipe close event not emitted

[jbunton-atlassian]

Summary

• Version: v8.12.0 and newer, all v10.x and all v11.x
• Platform: Linux and macOS amd64
• Subsystem: child_process

Since #18701 Node.js doesn't emit a close event when a child process closes its stdin pipe. The close is only detected if the parent tries to write to the child's stdin and receives an EPIPE.

The original behaviour is preferable because it allows immediate detection of the close event, rather than waiting for a failed write.

Reproducer

This works as expected in v8.11.4 and fails in all newer versions of v8.x, as well as all v10.x and all v11.x.

const {spawn} = require('child_process');

const cp = spawn(
    'node', [
        '-e',
        'fs.closeSync(0); setTimeout(() => {}, 2000)'
    ],
    {stdio: ['pipe', 'inherit', 'inherit']}
)

setTimeout(() => {
    console.log('BUGGY! stdin close event was not emitted')
    process.exit(1);
}, 1000);

cp.stdin.on('close', () => {
    console.log('Ok!')
    process.exit(0);
});

Problem detail

As far as I can tell the only way the close event can be emitted is if the Socket constructor calls this.read(0). This triggers onStreamRead() to be called, which does stream.push(null) and eventually results in the close event. This is a little weird because stdin isn't a readable stream :)

In Node 8.11.4 this worked because child_process.js:createSocket() called the Socket constructor with options.readable === undefined. So the Socket constructor sees that options.readable !== false and runs this.read(0)

In PR #18701 createSocket() was changed to call the Socket constructor with options.readable === true. This stops this.read(0) from being called and the close event is not emitted.

Hacky solution

--- a/lib/internal/child_process.js
+++ b/lib/internal/child_process.js
@@ -286,3 +286,3 @@ function flushStdio(subprocess) {
 function createSocket(pipe, readable) {
-  return net.Socket({ handle: pipe, readable, writable: !readable });
+  return net.Socket({ handle: pipe, readable, writable: !readable, childProcess: true });
 }
--- a/lib/net.js
+++ b/lib/net.js
@@ -315,3 +315,5 @@ function Socket(options) {
   // buffer.  if not, then this will happen when we connect
-  if (this._handle && options.readable !== false) {
+  if (options.childProcess) {
+    this.read(0);
+  } else if (this._handle && options.readable !== false) {
     if (options.pauseOnCreate) {

This passes the full test suite with a minor change to test-pipewrap.js. I haven't raised a PR because I'm sure somebody could think of a better solution. If someone can point me in the right direction I'm happy to try and implement it.

Thanks :)

[Fishrock123]

The provided test case also emits the following uv error for me, but only after the script is ended, so I presume it comes from the child.

Assertion failed: (fd > STDERR_FILENO), function uv__close, file ../deps/uv/src/unix/core.c, line 539.

[jbunton-atlassian]

hmm, I don't see that. Maybe I didn't compile with assertions enabled.

I agree that it should be coming from the child process. You could test by using a Python child instead :)

const cp = spawn(
    'python', [
        '-c',
        'import os, time; os.close(0); time.sleep(2)'
    ],
    {stdio: ['pipe', 'inherit', 'inherit']}
);

[adarshsaraogi]

shall I work on this issue.

[oyyd]

Reverting #18701 on master doesn't resolve this issue for me and I guess that's because of 67e2a15.

Maybe we could call stream.socket.read(0)(or socket.resume(), IDK) around here:

node/lib/internal/child_process.js

Line 390 in 80873ec

stream.socket = createSocket(this.pid !== 0 ?

rather than passing childProcess to net module If this is a specific issue of child_process.

[ronag]

Is this still a problem on e.g. v14?

[jbunton-atlassian]

@ronag, I just ran the test script with v14.12.0 and it is still broken.

[jasnell]

There's been no further discussion or activity on this in quite some time. Is there still stuff to do here?

[Tseian]

There's been no further discussion or activity on this in quite some time. Is there still stuff to do here?

@jasnell I reproduced it in the version 25. I am going to find an approach to fix it.

[Tseian]

Conclusion:

In v8.11.4, the read side was unexpectedly opened as well, allowing the entire socket lifecycle to be driven to completion by read-side events (EOF/HUP). Calling read(0) on the write side is somewhat unusual and unnecessary.

TL;DR
createSocket() first executes new net.Socket({ handle: pipe }).

Because options.readable is not explicitly passed, the net.Socket constructor assumes the socket is readable and calls read(0).

read(0) triggers _read(), which in turn calls the underlying _handle.readStart(), causing libuv to start listening for read events on this handle.

Later, createSocket() sets s.readable = false (semantically making stdin "write-only").

However, the steps above have already occurred—the read loop has already been started.

Then, when the child process executes fs.closeSync(0):

It closes the stdin file descriptor on the child process's side.

For the underlying file descriptor on the parent process's side, the kernel may generate a "closed on the other end" readable-side event (EOF/HUP, etc., depending on the details of the pipe/socketpair).

After the event loop receives the EOF/HUP event, the following calls are triggered: endWritableNT is called within endReadableNT.

Finally, the entire socket is destroyed and the close event is triggered:

endReadableNT
endWritableNT
NET 97229: _final: not ended, call shutdown()
NET 97229: afterShutdown destroyed=false
NET 97229: destroy
NET 97229: close
NET 97229: close handle
NET 97229: emit close

This is why cp.stdin.on('close') was triggered.

But after v8.11.4, read(0) will no longer be called. As a result, the event loop will not receive EOF/HUP events, nor will it trigger calls to endWritableNT and endReadableNT. Consequently, cp.stdin.on('close') will no longer be triggered.

[Tseian]

I have an idea, but I'm not sure if it's feasible: add two methods to pipe_wrap.cc.

• WatchPeerClose, in the method call the uv_read_start method to inject a listener for the EOF/HUP event. Once the event is detected, a JS callback is triggered to execute, which in turn triggers a method to destroy the socket.

• UnWatchPeerClose, in the method call the uv_read_stop method to unregister the listener.

Is there any other elegant solution?