Repository navigation
child_process.spawn() has undocumented 200kb buffer limit #4236
Description
Activity
That depends on with what options you spawned the child process. With
{ stdio: 'pipe' }(the default), the child's stdout and stderr file descriptors are pipes and those have limited (and platform-specific) capacity. If you don't plan on consuming the output from the child, create it with{ stdio: 'ignore' }.It's possible the documentation could be clearer (pull requests welcome) but it's working as intended, it implements flow control the same way
foo | bardoes in the shell.Reacted by cherishedtokenIt's possible the documentation could be clearer
Probably, I've found that option confusing in the past.
I got this problem in 2015 and write a post on medium, maybe I can help to document this :)
@bnoordhuis I see the limit buffer for this cases are stored in
max_buffer_stringas I can see here:and the error comes here:Lines 766 to 770 in ed3d8b1
Lines 596 to 598 in ed3d8b1
I think for better documentation will be nice, if I can provide a value of the max bytes we can write in the std/out/err by default.
it's really a problem, it will cause the process hang-up if you do not handle the stdout or stderr. I think the document should be clearer, or it will be really weird for someone who haven't handled the stdout or stderr.
To fix the docs just need to know whats the real limit as I say in my last comment :(
I think (but I could be wrong), that @bnoordhuis comment "those have limited (and platform-specific) capacity" means that the limits you're looking for come from the operating system, and not values defined in Node/libuv.
Reacted by cherishedtokenYeah, I see this comment @cjihrig, but looking in the code I'm little confused about this. :/ Will try found this information in some place :/
2 remaining items
@nodejs/documentation
- added 5 commits that reference this issue
on Mar 1, 2018 - added a commit that references this issue
on Mar 27, 2018 I am using spawn in CI and for some reason it doesn't redirect to stdout directly while it does on my host.
This result to the following message:
[ERROR] stderr maxBuffer exceededis there a way to increase maxBuffer or to force npx to work the stdout ?
Reacted by Samuel Volin and Penn- added a commit that references this issue
on Dec 3, 2018
The child_process.spawn() function will pause a script's execution if it writes more than 200kb of stdout that isn't captured. I discovered this when my spawned process would suspiciously pause after exactly a certain amount of output.
Adding a listener for stdout that does nothing fixed it and allowed the script to run:
This is not documented in the docs. Is this intentional (and hence should be documented) or a bug?