Add parallel test target, using pytest-xdist#8972
Add parallel test target, using pytest-xdist#8972wiredfool merged 3 commits intopython-pillow:mainfrom
Conversation
|
That's a huge improvement! Shall we just make it the default? Then it'll be used through make, tox, and direct pytest. --- a/pyproject.toml
+++ b/pyproject.toml
[tool.pytest.ini_options]
-addopts = "-ra --color=yes"
+addopts = "-ra --color=yes --numprocesses=auto"Although it does change the (pytest and tox) output from: To: One option is to add pytest-sugar to |
|
I think the format change is the One interesting thing -- if we do this, we're going to be running parallel tests on freethread python, which would definitely be a good thing. Edit -- looks like #8492 is actually referring to pytest-run-parallel, which is threads. |
|
The only thing that we'd have to be careful of is the valgrind tests can only run in one process |
|
The formatting change also happens when running plain |
|
I don't care that much about what the format is, other than I want fast tests when I'm iterating. Shorter CI logs aren't a bad thing, but there's a lot less context when you're tailing them. |
| "pytest", | ||
| "pytest-cov", | ||
| "pytest-timeout", | ||
| "pytest-xdist", |
There was a problem hiding this comment.
If the Makefile is going to install pytest-xdist by itself, and we're not using it anywhere else, I don't think this needs to be in pyproject.toml?
There was a problem hiding this comment.
Depends on if we decide to make it the default or not, really.
|
Looking at the parallel by default issue --
|
|
Since this is basically makefile only, merging now. We can look at setting this as default in another PR |
Changes proposed in this pull request:
This seems to be running 4 runners on an 8 core machine, for about a 2.5x speedup. Note that this is on the standard with GIL python, but I'm seeing 1.5x the user time.
-n 8on this machine doesn't speed up the wall clock time much (2-4%, but does significantly add to user time.(note, these numbers are a 8 core intel server in the cloud, probably 5 years old at this point)
On an M4 Air