-
Notifications
You must be signed in to change notification settings - Fork 86
Refactoring of POSIX process logic #208
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a large enough change in code that I'm honestly not that familiar that I can't offer a meaningful review. With tests passing, and based on you as the author, I feel comfortable merging it. I'd probably also pull this into typed-process
and run the test suite there as well. I've added a suggestion to fix the CI failure, and then I'll do a little more testing.
Previously this was a separate argument despite the fact that we already pass a flags argument.
This is a complete rewrite `posix/runProcess.c`. There are a few goals of this rewrite: * fix a long-standing and serious bug in the `execvpe` fallback path, which uses non-reentrant functions after `fork`ing. This is of course undefined behavior and has been causing failures under Darwin's Rosetta binary translation engine (see GHC #19994). * eliminate code duplication in the `fork/exec` implementation. * introduce support for `posix_spawn`, allowing us to unload a significant amount of complexity in some cases. This is particularly desireable as the cost of `fork` has increased considerably in some cases on recent Darwin releases (namely when `MAP_JIT` mappings are used; see [1]) While `posix_spawn` is often a win, there are unfortunately several cases where it cannot be used: * `posix_spawn_file_actions_addchdir_np` is broken on Darwin * `POSIX_SPAWN_SETSID` is only supported on mac 10.15 and later, but doesn't return a proper error code when not supported * the originally-specified semantics of `posix_spawn_file_actions_adddup2` are unsafe and have been amended (see [3]) but not all implementations have caught up (musl has [4], glibc did later [5], Darwin seemingly hasn't) there appears to be no support at all for setuid and setgid * `spawn` is significantly slower than fork on some Darwin releases (see [6]) To address this we first try using `posix_spawn`, falling back on `fork/exec` if we encounter a case which the former cannot handle. [1]: libuv/libuv#3064 [2]: https://www.austingroupbugs.net/view.php?id=411 [3]: rust-lang/rust#80537 [4]: https://git.musl-libc.org/cgit/musl/commit/?id=6fc6ca1a323bc0b6b9e9cdc8fa72221ae18fe206 [5]: https://sourceware.org/bugzilla/show_bug.cgi?id=23640 [6]: https://discuss.python.org/t/multiprocessing-spawn-default-on-macos-since-python-3-8-is-slower-than-fork-method/5910/4
Hi Michael, I believe this is now ready for further testing if you are able. Otherwise, I have also done a fair amount of testing locally (on Darwin and Linux) and I believe the work is sound. |
I've done additional testing on both Windows and Linux, by running both the local test suite and the typed-process test suite. Both are now passing. Merging now, thanks! Do you want me to make a release at this point? |
Cool, release 1.6.13.0 made including the changes from #209. |
Builds fail for
https://matrix.hackage.haskell.org/#/package/process/1.6.13.0 |
This is a complete rewrite of
posix/runProcess.c
. There are a few goalsof this rewrite:
fix a long-standing and serious bug in the
execvpe
fallback path, whichuses non-reentrant functions after
fork
ing. This is of course undefinedbehavior and has been causing failures under Darwin's Rosetta binary
translation engine (see GHC #19994).
eliminate code duplication in the
fork/exec
implementation.introduce support for
posix_spawn
, allowing us to unload a significantamount of complexity in some cases. This is particularly desireable as the
cost of
fork
has increased considerably in some cases on recent Darwinreleases (namely when
MAP_JIT
mappings are used; see 1)While
posix_spawn
is often a win, there are unfortunately several cases whereit cannot be used:
posix_spawn_file_actions_addchdir_np
is broken on DarwinPOSIX_SPAWN_SETSID
is only supported on mac 10.15 and later, but doesn'treturn a proper error code when not supported
the originally-specified semantics of
posix_spawn_file_actions_adddup2
areunsafe and have been amended (see 3) but not all implementations have
caught up (musl has 4, glibc did later 5, Darwin seemingly hasn't) there appears
to be no support at all for setuid and setgid
spawn
is significantly slower than fork on some Darwin releases (see 6)To address this we first try using
posix_spawn
, falling back onfork/exec
if we encounter a case which the former cannot handle.