Description
I've noticed a confusing situation that occurs when I write a test that is decorated with a marker that has the same name as an argvalue I pass in to the parametrize marker for that test when run with -m <marker>. Hard to explain, but super easy to reliably demonstrate. I'm seeing this on pytest 3.0.7 (Python 3.5.2, Mac OS X 10.12.4):
# test_thing.py
import pytest
@pytest.mark.cluster
@pytest.mark.parametrize('mode', ('cluster', 'standalone'))
def test_one(mode):
assert True
Running pytest -vs --collect-only shows both tests selected. On the other hand, running pytest -vs --collect-only -m cluster will deselect the test_one[cluster] item and leave me only with the test_one[standalone].
To try to debug what's happening, I used the pytest_collection_modifyitems hook in a local conftest.py to print the keywords.keys() attribute for each item collected. That is,
# conftest.py
def pytest_collection_modifyitems(items):
for item in items:
print(item.keywords.keys())
The output when re-running my test (whether with or without the -m cluster shows the following keys:
['ex', 'parametrize', 'test_thing.py', 'cluster', 'test_one[cluster]']
['standalone', 'test_one[standalone]', 'parametrize', 'cluster', 'ex', 'test_thing.py']
In other words, it looks as if duplicate marker/parametrize ids are being deleted at some point, which causes the test to be deselected.
I've tracked down that this is happening at the marker and id level as the following snippet works around the error:
@pytest.mark.parametrize('mode', ('cluster', 'standalone'), ids=('cluster_', 'standalone')).