-
Notifications
You must be signed in to change notification settings - Fork 252
Description
According to https://pytest-xdist.readthedocs.io/en/stable/distribution.html
--dist loadgroup: Tests are grouped by the xdist_group mark. Groups are distributed to available workers as whole units. This guarantees that all tests with same xdist_group name run in the same worker. If a test has multiple groups, they will be joined together into a new group, the order of the marks doesn’t matter. This works along with marks from fixtures and from the pytestmark global variable.
I'm seeing issues when attempting to use --dist loadgroup with pytestmark
Have a simple reproduction
.
├── bar
│ ├── __init__.py
│ └── tests
│ ├── conftest.py
│ └── test_bar.py
├── conftest.py
├── foo
│ ├── __init__.py
│ └── tests
│ ├── conftest.py
│ └── test_foo.py
├── main.py
├── pyproject.toml
├── README.md
└── uv.lock
The root conftest.py
import os
import logging
def pytest_configure(config):
worker_id = os.environ.get("PYTEST_XDIST_WORKER")
if worker_id is not None:
logging.basicConfig(
format=config.getini("log_file_format"),
filename=f"tests_{worker_id}.log",
level=config.getini("log_file_level"),
)
foo/tests/conftest.py looks like
import logging
import pytest
from flask import Flask
pytestmark = pytest.mark.xdist_group(name="foo")
logger = logging.getLogger(__name__)
@pytest.fixture(scope="package")
def app():
logger.warning("foo setup")
yield Flask("test")
logger.warning("foo teardown")
bar/tests/conftest.py looks like
import logging
import pytest
from flask import Flask
pytestmark = pytest.mark.xdist_group(name="foo")
logger = logging.getLogger(__name__)
@pytest.fixture(scope="package")
def app():
logger.warning("foo setup")
yield Flask("test")
logger.warning("foo teardown")
test_foo.py and test_bar.py contain x2 empty tests respectively
Bug
When I run
uv run pytest -n 2 --dist loadgroup
I get the following
====================================================== test session starts =======================================================
platform darwin -- Python 3.13.3, pytest-9.0.2, pluggy-1.6.0 -- /Users/jack/code/hehe/.venv/bin/python
cachedir: .pytest_cache
rootdir: /Users/jack/code/hehe
configfile: pyproject.toml
plugins: xdist-3.8.0
2 workers [4 items]
scheduling tests via LoadGroupScheduling
bar/tests/test_bar.py::test_bar_b
bar/tests/test_bar.py::test_bar_a
[gw1] [ 25%] PASSED bar/tests/test_bar.py::test_bar_b
[gw0] [ 50%] PASSED bar/tests/test_bar.py::test_bar_a
foo/tests/test_foo.py::test_foo_b
foo/tests/test_foo.py::test_foo_a
[gw0] [ 75%] PASSED foo/tests/test_foo.py::test_foo_a
[gw1] [100%] PASSED foo/tests/test_foo.py::test_foo_b
======================================================= 4 passed in 0.28s ========================================================
it looks like gw0 and gw1 and running tests in both foo and bar.
This is more obvious looking in the logs
───────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ File: tests_gw0.log
───────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
1 │ bar setup
2 │ foo setup
3 │ foo teardown
4 │ bar teardown
───────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
───────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ File: tests_gw1.log
───────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
1 │ bar setup
2 │ foo setup
3 │ foo teardown
4 │ bar teardown
───────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Given I have x2 workers and x2 groups, I'd expect gw0 to run foo completley and gw1 to run bar completely with no overlap.