Duplicated test case statuses reported with pytest-rerunfailures, retries section is empty #735
Closed
Description
I'm submitting a ...
- bug report
- feature request
- support request => Please do not submit support request here, see note at the top of this template.
What is the current behavior?
Tests that are re-run with pytset-rerunfailures are reported as multiple failures, while the "Retries" tab remains empty - as visible on the attached screenshot.
If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem
The contents of the test script, in this case test_me_2.py
:
def test_fail():
assert False
Execute pytest --reruns=2 --alluredir=<allure_dir> test_me_2.py
And then display the allure report.
The environment (venv) contents:
allure-pytest==2.13.0
allure-python-commons==2.13.0
attrs==22.2.0
exceptiongroup==1.1.0
iniconfig==2.0.0
packaging==23.0
pluggy==1.0.0
pytest==7.2.2
pytest-rerunfailures==11.1.2
six==1.16.0
tomli==2.0.1
What is the expected behavior?
Tests are reported once, in this case it is obviously a failure by design, the re-runs end up in the "Retries" section as on the screenshot below. This is the report I'm seeing when using allure-pytest-2.12.0:
What is the motivation / use case for changing the behavior?
The plugin pytest-rerunfailures is a common pytest plugin to make an attempt at re-running flaky tests. Allure used to work perfectly with it, now it doesn't.
Please tell us about your environment:
- Allure version: 2.10.0
- Test framework: pytest@7.2.2
- Allure adaptor: allure-pytest@2.13.0