Does not work with parametrized nodeids

pytest tests/test_database.py::TestDatabaseFixtures::test_access\[db\] works, but adding --count 10 fails:

collecting 0 items
ERROR: not found: …/Vcs/pytest-django/tests/test_database.py::TestDatabaseFixtures::test_access[db]
(no name '…/Vcs/pytest-django/tests/test_database.py::TestDatabaseFixtures::test_access[db]' in any of [<Instance '()'>])

pytest version 3.6.3.dev5+gb7b9c54d

does this happen in older pytest versions ?

Fix pytest-dev#21: allow using repeat(1) marker to disable repeat

It happens in older pytest versions as well (at least until 2.8)
I opened an issue on pytest tracker pytest-dev/pytest#4142 providing a unit test reproducing the bug.

Any progress on this?

Hey, looks like this happens on the latest pytest version
Do we have any updates on this one?

This is happening for me with pytest 7.1.0 and pytest-repeat 0.9.1.

Is it related to the way test items are renamed and collected, in that pytest-repeat modifies the name with extra parameters, and this affects name matching?

For example, with a parametrized test that has one parameter of value 256, this works (runs once):

 $ pytest test.py::test_something[256]

And this works (runs 1000 times but for each parameter, which may be far too many tests):

 $ pytest test.py::test_something --count 1000

But to run for a single parameter value, this does not work:

 $ pytest test.py::test_something[256] --count 1000
(no name 'test.py::test_something[256]' in any of [<Module test.py>])

And this runs once, but does not repeat:

 $ pytest test.py::test_something[256-1-1000] --count 1000

Is there a known workaround, aside from running pytest over and over in a shell script loop?

I'm curious if there's been any thinking from the contributors as to how this might work, if we could get it to work?

I come across this so often I'd be happy to help fix it, if I could get in sync with the designers' thoughts...

Currently no design work on this was done

I believe major internals Changes in pytest itself are necessary to enable this

Seems to work fine now, with pytest 7.4.2 and pytest-repeat 0.9.2

import pytest

@pytest.mark.parametrize('a', (10, 20))
def test_foo(a):
    ...

test run:

(pytest-repeat) $ pytest test_param.py -v --count 3
================================= test session starts =================================
collected 6 items                                                                     

test_param.py::test_foo[10-1-3] PASSED                                          [ 16%]
test_param.py::test_foo[10-2-3] PASSED                                          [ 33%]
test_param.py::test_foo[10-3-3] PASSED                                          [ 50%]
test_param.py::test_foo[20-1-3] PASSED                                          [ 66%]
test_param.py::test_foo[20-2-3] PASSED                                          [ 83%]
test_param.py::test_foo[20-3-3] PASSED                                          [100%]

================================== 6 passed in 0.01s ==================================

@okken

@okken I think the issue remains and it’s that you can’t do this:

 $ pytest test_param.py::test_foo[10] -v --count 3

This one is unfixable without a major change in pytest

The act of using --count 3 changes the nodeids, so with --count 3, test_foo[10] no longer exists.

There is a workaround, selecting parameters by keyword instead of nodeid:

(pytest-repeat) $ pytest test_param.py -v -k 'test_foo[10' --count 3
=========================== test session starts ============================
collected 6 items / 3 deselected / 3 selected                              

test_param.py::test_foo[10-1-3] PASSED                               [ 33%]
test_param.py::test_foo[10-2-3] PASSED                               [ 66%]
test_param.py::test_foo[10-3-3] PASSED                               [100%]

===================== 3 passed, 3 deselected in 0.01s ======================

We also cannot select individual counts using nodeid:

(pytest-repeat) $ pytest 'test_param.py::test_foo[10-2-3]'              
=========================== test session starts ============================
collected 0 items                                                          

========================== no tests ran in 0.00s ===========================
ERROR: not found: /Users/okken/projects/pytest-repeat/test_param.py::test_foo[10-2-3]
(no name '/Users/okken/projects/pytest-repeat/test_param.py::test_foo[10-2-3]' in any of [<Module test_param.py>])

I don't think this is a bug, it's an affect of when the counts are added to the nodeid.

And there is the same workaround, use keywords.

(pytest-repeat) $ pytest test_param.py -v -k 'test_foo[10-2-3' --count 3
=========================== test session starts ============================
collected 6 items / 5 deselected / 1 selected                              

test_param.py::test_foo[10-2-3] PASSED                               [100%]

===================== 1 passed, 5 deselected in 0.00s ======================

@RonnyPfannschmidt

@RonnyPfannschmidt this brings me to a question of "why leave this issue open?"

We know desired syntax isn't realistic due to the implementation of both pytest-repeat and more importantly, pytest.
So unless pytest changes, it's not fixable.
There is a workaround, use keywords.

So why keep this open?

Also just swapped "bug" for "enhancement", because there is a way to do the requested action, just not with the desired syntax.

I'd like to turn this into a feature request for pytest

I personally wouldn't know how to describe the feature request. Partly because I don't fully understand the limitations of pytest that are the problem. But I agree.

I do think, at the very least, this issue should remain open until the workaround is documented and easy to find.

@DavidAntliff

@DavidAntliff Does the -k workaround work for your use case?

@DavidAntliff

@DavidAntliff Does the -k workaround work for your use case?

Yes, that works for me - thank you. I've been looking for a way to do this for a while.

Navigation Menu

Repository navigation

Does not work with parametrized nodeids #21

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Does not work with parametrized nodeids #21

Description

Activity

RonnyPfannschmidt commented on Jul 3, 2018

vbarbaresi commented on Oct 14, 2018

blueyed commented on Oct 14, 2018

blueyed commented on Oct 14, 2018

1Mark commented on Jul 17, 2021

vyahello commented on Aug 17, 2021

DavidAntliff commented on Feb 20, 2023

DavidAntliff commented on Aug 1, 2023

RonnyPfannschmidt commented on Aug 1, 2023

okken commented on Oct 5, 2023

DavidAntliff commented on Oct 5, 2023

RonnyPfannschmidt commented on Oct 5, 2023

okken commented on Oct 5, 2023

okken commented on Oct 5, 2023

okken commented on Oct 5, 2023

okken commented on Oct 5, 2023

RonnyPfannschmidt commented on Oct 5, 2023

okken commented on Oct 5, 2023

okken commented on Oct 5, 2023

DavidAntliff commented on Oct 6, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions