net: support sending net.BoundSocket to threads and child processes

This adds support for transferring net.BoundSocket instances to other
threads via the worker_threads postMessage() transfer list, and for
sending them to child processes as the sendHandle argument of
subprocess.send(), following on from the BoundSocket introduction.

A BoundSocket reserves a port synchronously at construction time.
Making it transferable means a port can be reserved on one thread or
process and the bound (but not yet listening or connected) TCP handle
handed off to another to listen or connect on, without racing on the
bind.

For threads, BoundSocket implements kTransfer/kTransferList/
kDeserialize, moving the underlying TCP handle with the same mechanism
used for net.Socket and net.Server transfer. For child processes, the
handleConversion entry reuses the same transfer protocol on the
sending side and the same _TransferredBoundSocket deserialization path
on the receiving side; the underlying transport is that of cluster's
shared-handle scheduling: SCM_RIGHTS on Unix and WSADuplicateSocket on
Windows, both of which carry bind state.

In both cases the source instance is left in the adopted state:
address(), fd() and close() throw ERR_SOCKET_HANDLE_ADOPTED. Transfer
requires an un-adopted, open TCP handle, otherwise
ERR_WORKER_HANDLE_NOT_TRANSFERABLE is thrown; pipe (path) binds are
not transferable and throw ERR_INVALID_HANDLE_TYPE when sent over IPC.
On the receiving side the local address is re-derived from the handle
rather than trusted from serialized state.

Signed-off-by: Guy Bedford <guybedford@gmail.com>
PR-URL: #64725
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>

Author: guybedford

doc/api/child_process.md

@@ -1581,7 +1581,7 @@ added: v0.5.9
 
 * `message` {Object} A parsed JSON object or primitive value.
 * `sendHandle` {Handle|undefined} `undefined` or a [`net.Socket`][],
-  [`net.Server`][], or [`dgram.Socket`][] object.
+  [`net.Server`][], [`net.BoundSocket`][], or [`dgram.Socket`][] object.
 
 The `'message'` event is triggered when a child process uses
 [`process.send()`][] to send messages.
@@ -1891,6 +1891,9 @@ subprocess.ref();
 <!-- YAML
 added: v0.5.9
 changes:
+  - version: REPLACEME
+    pr-url: https://github.com/nodejs/node/pull/64725
+    description: '`net.BoundSocket` instances can now be sent.'
   - version: v5.8.0
     pr-url: https://github.com/nodejs/node/pull/5283
     description: The `options` parameter, and the `keepOpen` option
@@ -1905,7 +1908,7 @@ changes:
 
 * `message` {Object}
 * `sendHandle` {Handle|undefined} `undefined`, or a [`net.Socket`][],
-  [`net.Server`][], or [`dgram.Socket`][] object.
+  [`net.Server`][], [`net.BoundSocket`][], or [`dgram.Socket`][] object.
 * `options` {Object} The `options` argument, if present, is an object used to
   parameterize the sending of certain types of handles. `options` supports
   the following properties:
@@ -1972,12 +1975,18 @@ Applications should avoid using such messages or listening for
 `'internalMessage'` events as it is subject to change without notice.
 
 The optional `sendHandle` argument that may be passed to `subprocess.send()` is
-for passing a TCP server or socket object to the child process. The child process will
+for passing a TCP server, socket or [`net.BoundSocket`][] object to the child
+process. The child process will
 receive the object as the second argument passed to the callback function
 registered on the [`'message'`][] event. Any data that is received
 and buffered in the socket will not be sent to the child. Sending IPC sockets is
 not supported on Windows.
 
+Sending a `net.BoundSocket` moves its underlying TCP handle to the child
+process, leaving the source instance in the adopted state as if it had been
+adopted by a server or socket. The bound socket must not have been adopted or
+closed, and pipe (`path`) binds cannot be sent.
+
 The optional `callback` is a function that is invoked after the message is
 sent but before the child process may have received it. The function is called with a
 single argument: `null` on success, or an [`Error`][] object on failure.
@@ -2405,6 +2414,7 @@ or [`child_process.fork()`][].
 [`child_process.spawnSync()`]: #child_processspawnsynccommand-args-options
 [`dgram.Socket`]: dgram.md#class-dgramsocket
 [`maxBuffer` and Unicode]: #maxbuffer-and-unicode
+[`net.BoundSocket`]: net.md#class-netboundsocket
 [`net.Server`]: net.md#class-netserver
 [`net.Socket`]: net.md#class-netsocket
 [`options.detached`]: #optionsdetached

doc/api/errors.md

@@ -2998,8 +2998,9 @@ A call was made and the UDP subsystem was not running.
 ### `ERR_SOCKET_HANDLE_ADOPTED`
 
 An operation was attempted on a [`BoundSocket`][] that had already been adopted
-by a [`net.Server`][] or [`net.Socket`][]. Once a bound socket is adopted, its
-`address()` and `close()` methods can no longer be used.
+by a [`net.Server`][] or [`net.Socket`][], or transferred to another thread.
+Once a bound socket is adopted or transferred, its `address()` and `close()`
+methods can no longer be used.
 
 <a id="ERR_SOURCE_MAP_CORRUPT"></a>
 
@@ -3567,9 +3568,10 @@ The `Response` that has been passed to `WebAssembly.compileStreaming` or to
 
 ### `ERR_WORKER_HANDLE_NOT_TRANSFERABLE`
 
-An attempt was made to transfer a `net.Socket` or `net.Server` to another thread
-via a `worker_threads` `postMessage()` call while it was not in a transferable
-state, for example because it had already started reading or had buffered data.
+An attempt was made to transfer a `net.Socket`, `net.Server` or
+`net.BoundSocket` to another thread via a `worker_threads` `postMessage()` call
+while it was not in a transferable state, for example because it had already
+started reading, had buffered data, or had already been adopted.
 
 <a id="ERR_WORKER_INIT_FAILED"></a>
 

doc/api/net.md

@@ -940,6 +940,13 @@ server.listen(8000);
 A listening [`net.Server`][] can be transferred the same way, which moves the
 listening socket itself (and its pending accept queue) to the receiving thread.
 
+An un-adopted TCP [`BoundSocket`][] can also be transferred, which moves the
+bound (but not yet listening or connected) socket. This allows a port to be
+reserved synchronously on one thread and adopted by a server or outgoing
+connection on another. Pipe binds are not transferable. After the transfer, the
+source `BoundSocket` behaves as if it had been adopted: `address()`, `fd()` and
+`close()` throw [`ERR_SOCKET_HANDLE_ADOPTED`][].
+
 ### `new net.Socket([options])`
 
 <!-- YAML
@@ -1863,6 +1870,13 @@ file system entry; abstract and TCP binds have none to remove.
 When a pipe `BoundSocket` bound to a source `path` is adopted as a client, that
 path is reported as the socket's `localAddress` once it connects.
 
+An un-adopted TCP `BoundSocket` can be moved to another thread by listing it in
+the `transferList` of a [`worker_threads`][] `postMessage()` call, see
+[Transferring TCP handles to other threads][]. It can likewise be sent to a
+child process as the `sendHandle` argument of [`subprocess.send()`][]. In both
+cases the source is left in the adopted state. Pipe binds cannot be moved
+either way.
+
 When an adopted `BoundSocket` connects to a numeric IP literal, `connect(2)` is
 issued synchronously, so [`socket.localAddress`][] is resolved once
 [`socket.connect()`][] returns. Connection failures are still reported via a
@@ -2493,6 +2507,7 @@ net.isIPv6('fhqwhgads'); // returns false
 [`socket.setTimeout()`]: #socketsettimeouttimeout-callback
 [`socket.setTimeout(timeout)`]: #socketsettimeouttimeout-callback
 [`stream.getDefaultHighWaterMark()`]: stream.md#streamgetdefaulthighwatermarkobjectmode
+[`subprocess.send()`]: child_process.md#subprocesssendmessage-sendhandle-options-callback
 [`worker_threads`]: worker_threads.md
 [`writable.destroy()`]: stream.md#writabledestroyerror
 [`writable.destroyed`]: stream.md#writabledestroyed

doc/api/worker_threads.md

@@ -1236,7 +1236,7 @@ port2.postMessage(circularData);
 ```
 
 `transferList` may be a list of {ArrayBuffer}, [`MessagePort`][],
-[`FileHandle`][], {net.Server}, and {net.Socket} objects.
+[`FileHandle`][], {net.Server}, {net.Socket}, and {net.BoundSocket} objects.
 After transferring, they are not usable on the sending side of the channel
 anymore (even if they are not contained in `value`).
 
@@ -1247,6 +1247,8 @@ freshly accepted or created TCP connection that has not yet started reading and
 has no buffered data, otherwise `postMessage()` throws
 `ERR_WORKER_HANDLE_NOT_TRANSFERABLE`. This makes it possible to accept
 connections on one thread and distribute them across a pool of worker threads.
+Transferring a {net.BoundSocket} moves an un-adopted pre-bound socket, so a
+port can be reserved synchronously on one thread and adopted on another.
 Only TCP handles are supported.
 
 If `value` contains {SharedArrayBuffer} instances, those are accessible

lib/internal/child_process.js

@@ -55,6 +55,10 @@ const { TCP } = internalBinding('tcp_wrap');
 const { TTY } = internalBinding('tty_wrap');
 const { UDP } = internalBinding('udp_wrap');
 const SocketList = require('internal/socket_list');
+const {
+  kDeserialize,
+  kTransfer,
+} = require('internal/worker/js_transferable');
 const { owner_symbol } = require('internal/async_hooks').symbols;
 const { convertToValidSignal, deprecate } = require('internal/util');
 const { isArrayBufferView } = require('internal/util/types');
@@ -87,6 +91,24 @@ const kChannelHandle = Symbol('kChannelHandle');
 const kIsUsedAsStdio = Symbol('kIsUsedAsStdio');
 const kPendingMessages = Symbol('kPendingMessages');
 
+// Store the handle after successfully sending it, so it can be closed when
+// the NODE_HANDLE_ACK is received. If the handle could not be sent, just
+// close it.
+function closeHandleAfterAck(message, handle, options, callback, target) {
+  if (handle && !options.keepOpen) {
+    if (target) {
+      // There can only be one _pendingMessage as passing handles are
+      // processed one at a time: handles are stored in _handleQueue while
+      // waiting for the NODE_HANDLE_ACK of the current passing handle.
+      assert(!target._pendingMessage);
+      target._pendingMessage =
+          { callback, message, handle, options, retransmissions: 0 };
+    } else {
+      handle.close();
+    }
+  }
+}
+
 // This object contain function to convert TCP objects to native handle objects
 // and back again.
 const handleConversion = {
@@ -117,6 +139,29 @@ const handleConversion = {
     },
   },
 
+  'net.BoundSocket': {
+    simultaneousAccepts: true,
+
+    send(message, boundSocket, options) {
+      // Pipe (path) binds cannot be sent over the IPC channel.
+      if (boundSocket.isPipe)
+        throw new ERR_INVALID_HANDLE_TYPE();
+
+      // Reuse the worker_threads transfer protocol: detaches the handle,
+      // leaving the source in the adopted state, and throws if the bound
+      // socket is not in a transferable state.
+      return boundSocket[kTransfer]().data.handle;
+    },
+
+    postSend: closeHandleAfterAck,
+
+    got(message, handle, emit) {
+      const boundSocket = new net._TransferredBoundSocket();
+      boundSocket[kDeserialize]({ handle });
+      emit(boundSocket);
+    },
+  },
+
   'net.Socket': {
     send(message, socket, options) {
       if (!socket._handle)
@@ -166,23 +211,7 @@ const handleConversion = {
       return handle;
     },
 
-    postSend(message, handle, options, callback, target) {
-      // Store the handle after successfully sending it, so it can be closed
-      // when the NODE_HANDLE_ACK is received. If the handle could not be sent,
-      // just close it.
-      if (handle && !options.keepOpen) {
-        if (target) {
-          // There can only be one _pendingMessage as passing handles are
-          // processed one at a time: handles are stored in _handleQueue while
-          // waiting for the NODE_HANDLE_ACK of the current passing handle.
-          assert(!target._pendingMessage);
-          target._pendingMessage =
-              { callback, message, handle, options, retransmissions: 0 };
-        } else {
-          handle.close();
-        }
-      }
-    },
+    postSend: closeHandleAfterAck,
 
     got(message, handle, emit) {
       const socket = new net.Socket({
@@ -838,6 +867,8 @@ function setupChannel(target, channel, serializationMode) {
         message.type = 'net.Socket';
       } else if (handle instanceof net.Server) {
         message.type = 'net.Server';
+      } else if (handle instanceof net.BoundSocket) {
+        message.type = 'net.BoundSocket';
       } else if (handle instanceof TCP || handle instanceof Pipe) {
         message.type = 'net.Native';
       } else if (handle instanceof dgram.Socket) {

lib/net.js

@@ -388,6 +388,10 @@ const kBoundPath = Symbol('kBoundPath');
 
 const isLinux = process.platform === 'linux';
 
+// Internal: construct an empty BoundSocket shell during postMessage()
+// deserialization; the transferred handle is installed by [kDeserialize].
+const kBoundSocketDeserialize = Symbol('kBoundSocketDeserialize');
+
 // A role-neutral wrapper over a synchronously bound libuv handle: bound to a
 // local address (a numeric IP literal for TCP, or a filesystem/abstract path
 // for a unix-domain socket via { path }) but neither listening nor connecting
@@ -396,11 +400,20 @@ const isLinux = process.platform === 'linux';
 // handle must be closed by the caller. bind(2) is non-blocking, so binding
 // happens inline and errors throw synchronously. No DNS is performed.
 class BoundSocket {
-  #handle;
+  #handle = null;
   #address = {};
   #path;
 
   constructor(options = kEmptyObject) {
+    // An un-adopted BoundSocket can be moved to another thread by listing it
+    // in the transferList of a worker_threads postMessage() call. See
+    // [kTransfer]().
+    markTransferMode(this, false, true);
+
+    if (options === kBoundSocketDeserialize) {
+      return;
+    }
+
     validateObject(options, 'options');
 
     if (options.path !== undefined) {
@@ -544,7 +557,56 @@ class BoundSocket {
   get isPipe() {
     return this.#path !== undefined;
   }
+
+  // A BoundSocket can be transferred only while it still owns its handle,
+  // i.e. before it has been adopted, closed or already transferred. Only TCP
+  // binds are transferable; pipe handles cannot move between event loops.
+  #assertTransferable() {
+    if (this.#handle === null || !(this.#handle instanceof TCP)) {
+      throw new ERR_WORKER_HANDLE_NOT_TRANSFERABLE('net.BoundSocket');
+    }
+  }
+
+  [kTransferList]() {
+    this.#assertTransferable();
+    return [this.#handle];
+  }
+
+  [kTransfer]() {
+    this.#assertTransferable();
+    const handle = this.#handle;
+    // Detach the handle; the messaging layer takes ownership of it via
+    // TCPWrap::TransferForMessaging(). Further use on the sending side throws
+    // ERR_SOCKET_HANDLE_ADOPTED, as after adoption.
+    this.#handle = null;
+    return {
+      data: { handle },
+      deserializeInfo: 'net:_TransferredBoundSocket',
+    };
+  }
+
+  [kDeserialize](data) {
+    const handle = data?.handle;
+    if (handle == null || !(handle instanceof TCP)) {
+      throw new ERR_WORKER_HANDLE_NOT_TRANSFERABLE('net.BoundSocket');
+    }
+    // Re-derive the bound address from the transferred handle rather than
+    // trusting serialized state.
+    const err = handle.getsockname(this.#address);
+    if (err) {
+      handle.close();
+      throw new ERR_WORKER_HANDLE_NOT_TRANSFERABLE('net.BoundSocket');
+    }
+    this.#handle = handle;
+  }
+}
+
+// Deserialization target for a transferred BoundSocket: constructs an empty
+// shell without binding a new socket. Internal, not part of the public API.
+function _TransferredBoundSocket() {
+  return new BoundSocket(kBoundSocketDeserialize);
 }
+_TransferredBoundSocket.prototype = BoundSocket.prototype;
 
 function Socket(options) {
   if (!(this instanceof Socket)) return new Socket(options);
@@ -2937,6 +2999,7 @@ Server.prototype.unref = function() {
 module.exports = {
   _createServerHandle: createServerHandle,
   _normalizeArgs: normalizeArgs,
+  _TransferredBoundSocket,
   get BlockList() {
     BlockList ??= require('internal/blocklist').BlockList;
     return BlockList;