Closed
Description
What is the problem this feature will solve?
It does not appear to be possible [1] to adjust the buffer size (highWaterMark) for the stdout from a child_process.spawn() call. This very likely [2] leads to lower throughput when dealing with large amounts of output from a child process, since more event loop iterations are required to consume it. Increasing the buffer size involves trading away event loop latency, but the latency vs. throughput tradeoff decision should ideally be in the hands of the javascript developer as it is with most other read / write streams in nodejs.
- Please see Stack Exchange answer from @mmomtchev - https://stackoverflow.com/a/70652730/9129863 and unanswered node help question - How to adjust the the stdout buffer size (aka highWaterMark) of child_process.spawn()? help#3683
- While I'd like to provide some performance benchmarks with some large data processing with small vs. large buffer sizes, since the setting is not easily tweakable it's not straightforward
What is the feature you are proposing to solve the problem?
Some way of adjusting the highWaterMark (or equivalent) of the stdout stream of a child_process.spawn() call.
What alternatives have you considered?
As far as I know, there isn't one. You just have to accept the default of 64KB.