Skip to content

Duplicated test case statuses reported with pytest-rerunfailures, retries section is empty #735

Closed
@plume-aostrowski

Description

I'm submitting a ...

  • bug report
  • feature request
  • support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

Tests that are re-run with pytset-rerunfailures are reported as multiple failures, while the "Retries" tab remains empty - as visible on the attached screenshot.
Screenshot from 2023-03-09 14-17-34

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

The contents of the test script, in this case test_me_2.py:

def test_fail():
    assert False

Execute pytest --reruns=2 --alluredir=<allure_dir> test_me_2.py
And then display the allure report.

The environment (venv) contents:

allure-pytest==2.13.0
allure-python-commons==2.13.0
attrs==22.2.0
exceptiongroup==1.1.0
iniconfig==2.0.0
packaging==23.0
pluggy==1.0.0
pytest==7.2.2
pytest-rerunfailures==11.1.2
six==1.16.0
tomli==2.0.1

What is the expected behavior?

Tests are reported once, in this case it is obviously a failure by design, the re-runs end up in the "Retries" section as on the screenshot below. This is the report I'm seeing when using allure-pytest-2.12.0:
Screenshot from 2023-03-09 14-19-24

What is the motivation / use case for changing the behavior?

The plugin pytest-rerunfailures is a common pytest plugin to make an attempt at re-running flaky tests. Allure used to work perfectly with it, now it doesn't.

Please tell us about your environment:

  • Allure version: 2.10.0
  • Test framework: pytest@7.2.2
  • Allure adaptor: allure-pytest@2.13.0

Other information

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions