-
Notifications
You must be signed in to change notification settings - Fork 72
Open
Description
Bug Description
When using --testmon --testmon-nocollect with pytest-xdist (-n auto or -n <num>), the test run fails intermittently with:
sqlite3.OperationalError: database is locked
This is especially common on CI systems with multiple parallel workers.
Root Cause
The TestmonXdistSync plugin is only registered inside the if should_collect: block in register_plugins():
def register_plugins(config, should_select, should_collect, cov_plugin):
if should_select or should_collect:
config.pluginmanager.register(TestmonSelect(...))
if should_collect: # BUG: TestmonXdistSync only registered here!
config.pluginmanager.register(TestmonCollect(...))
if config.pluginmanager.hasplugin("xdist"):
config.pluginmanager.register(TestmonXdistSync())When --testmon-nocollect is used:
should_collect = FalseTestmonXdistSyncis not registered- The
pytest_configure_nodehook doesn't run - Workers don't receive
testmon_exec_idfrom the controller - Each worker calls
TestmonData.for_local_run()instead ofTestmonData.for_worker() for_local_run()→initiate_execution()→fetch_or_create_environment()fetch_or_create_environment()usesBEGIN IMMEDIATE TRANSACTION(write lock)- Multiple workers race for the write lock → database locked error
Steps to Reproduce
# Create a simple test file
echo 'def test_one(): pass' > test_sample.py
# First run to create .testmondata
pytest --testmon test_sample.py
# Second run with nocollect + xdist - this causes database lock
pytest --testmon --testmon-nocollect -n 4 test_sample.pyExpected Behavior
The --testmon-nocollect mode should work seamlessly with pytest-xdist. Workers should receive the exec_id from the controller and open the database in read-only mode.
Proposed Fix
- Register
TestmonXdistSyncwhenever testmon is active (not just when collecting):
if should_select or should_collect:
config.pluginmanager.register(TestmonSelect(...))
# Register xdist sync even in nocollect mode
if config.pluginmanager.hasplugin("xdist"):
config.pluginmanager.register(TestmonXdistSync(should_collect=should_collect))- Skip database sync in
pytest_xdist_node_collection_finishedwhen in nocollect mode:
class TestmonXdistSync:
def __init__(self, should_collect=True):
self._should_collect = should_collect
def pytest_xdist_node_collection_finished(self, node, ids):
self.await_nodes += -1
if self.await_nodes == 0 and self._should_collect:
node.config.testmon_data.sync_db_fs_tests(retain=set(ids))Environment
- pytest-testmon version: 2.2.0
- pytest version: 7.x / 8.x
- pytest-xdist version: 3.x
- Python version: 3.10+
- OS: Linux (GitHub Actions runners), but reproducible on any OS
Impact
This bug affects all users who:
- Use
--testmon-nocollectfor CI pipelines (read-only mode) - Use pytest-xdist for parallel test execution
- Run on systems with multiple workers
Metadata
Metadata
Assignees
Labels
No labels