Skip to content

Allow for adjustment of stdout buffer size (aka highWaterMark?) of child_process.spawn() #41611

Closed
@AndrewJDR

Description

@AndrewJDR

What is the problem this feature will solve?

It does not appear to be possible [1] to adjust the buffer size (highWaterMark) for the stdout from a child_process.spawn() call. This very likely [2] leads to lower throughput when dealing with large amounts of output from a child process, since more event loop iterations are required to consume it. Increasing the buffer size involves trading away event loop latency, but the latency vs. throughput tradeoff decision should ideally be in the hands of the javascript developer as it is with most other read / write streams in nodejs.

  1. Please see Stack Exchange answer from @mmomtchev - https://stackoverflow.com/a/70652730/9129863 and unanswered node help question - How to adjust the the stdout buffer size (aka highWaterMark) of child_process.spawn()? help#3683
  2. While I'd like to provide some performance benchmarks with some large data processing with small vs. large buffer sizes, since the setting is not easily tweakable it's not straightforward

What is the feature you are proposing to solve the problem?

Some way of adjusting the highWaterMark (or equivalent) of the stdout stream of a child_process.spawn() call.

What alternatives have you considered?

As far as I know, there isn't one. You just have to accept the default of 64KB.

Metadata

Metadata

Assignees

No one assigned

    Labels

    child_processIssues and PRs related to the child_process subsystem.feature requestIssues that request new features to be added to Node.js.stalestdioIssues and PRs related to stdio.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions