bpo-35537: subprocess can now use os.posix_spawnp #11579
Conversation
The subprocess module can now use the os.posix_spawnp() function if it is available to locate the program in the PATH.
| @@ -1456,7 +1460,10 @@ def _posix_spawn(self, args, executable, env, restore_signals): | |||
| sigset.append(signum) | |||
| kwargs['setsigdef'] = sigset | |||
|
|
|||
| self.pid = os.posix_spawn(executable, args, env, **kwargs) | |||
| if _HAVE_POSIX_SPAWNP: | |||
| self.pid = os.posix_spawnp(executable, args, env, **kwargs) | |||
There was a problem hiding this comment.
Please keep in mind that _posixsubprocess is likely to use a different algorithm for searching in PATH compared to posix_spawnp implementations (and, moreover, posix_spawnp implementations differ between libcs and even between versions of the same libc). For example, musl uses execvpe as a "backend" for posix_spawnp, which skips only 3 kinds of errors, while _posixsubprocess skips all errors. Also, EACCES is treated in a special way by both glibc and musl.
Also, the default PATH if the environment variable is not set is likely to differ (e.g., glibc uses /bin:/usr/bin, musl uses /usr/local/bin:/bin:/usr/bin, and Python uses :/bin:/usr/bin).
There was a problem hiding this comment.
"... glibc uses /bin:/usr/bin ... and Python uses :/bin:/usr/bin"
Oh. ":" at the start of posixpath.defpath is surprising. Is it a bug? What's the point of putting an empty string in the list of default search paths? It doesn't help to locate an executable...
There was a problem hiding this comment.
It's not a bug: empty entries in PATH mean "the current directory". See also NOTES in man execvp about glibc dropping this empty entry.
There was a problem hiding this comment.
I created https://bugs.python.org/issue35755 to propose to remove it :-)
|
I'm not sure that the exact list of errors catched or nor by subprocess when locating the executable is a major blocker issue, but I didn't notice that subprocess build a list of executable using the PATH computed from its env argument. Using the env argument is a major difference. The devil is in the details... I'm no longer sure that using posix_spawnp() in subprocess is appropriate. In case of doubt, I prefer to revert this change to have more time to decide what is the best option. I wrote PR #11582 to revert this change. |
The subprocess module can now use the os.posix_spawnp() function if
it is available to locate the program in the PATH.
https://bugs.python.org/issue35537