Test case naming function (alternative to ids=)

[radeklat]

After migration to pytest, I am frequently missing feature with regard to test parametrization that I used in the parameterized library:

Ability to generate test ID based on the entire set of values.

In parameterized, this has been implemented as an testcase_func_name argument, which is similar to the ids argument. With one notable exception:

• ids operates on individual values
• testcase_func_name operates on the entire set of values

The latter is much more flexible when it comes to dropping, reordering or combining values.

At the moment, I'm forced to use utility functions instead that take take a list of example values and map it to pytest.param:

def named_test_cases(func: Callable[..., str], params: List[Tuple]) -> List[ParameterSet]:
    return [pytest.param(*param, id=func(*param)) for param in params]

and use it as:

class TestInOperator:
    @staticmethod
    @pytest.mark.parametrize(
        'collection,value',
        named_test_cases(
            lambda collection, value: f"{value} in {collection.__class__.__name__}",
            [([1], 1), ({1: 1}, 1)]
        )
    )
    def test_should_work_on(collection, value):
        assert value in collection

(artificial example for shortness, I'm sure you can imagine more creative ways of combining the values)

Or even simpler, allowing a string that is treated as an input for str.format():

def named_test_cases(format_string: str, params: List[Tuple]) -> List[ParameterSet]:
    return [pytest.param(*param, id=format_string.format(*param)) for param in params]

and use it as:

...
named_test_cases(
    "{1} in {0.__class__.__name__}",
    [([1], 1), ({1: 1}, 1)]
)
...

You can see this is a short and elegant way of providing well readable names for test cases.

I searched the existing issues and the closest one on similar topic was #6335. I agree with not modifying the existing ids argument that is already quite complex. However, would it be an option to implement functionality similar to above using new argument(s)? (testcase_func_name, testcase_format_name, or anything else)

The end results could look something like:

class TestInOperator:
    @staticmethod
    @pytest.mark.parametrize(
        'collection,value',
        [([1], 1), ({1: 1}, 1)],
        testcase_format_name="{1} in {0.__class__.__name__}",
    )
    def test_should_work_on(collection, value):
        assert value in collection

[blueyed]

Related #759 (search for idsetfn), and some (stalled) WIP: blueyed#104

[Zac-HD]

Personally I'm -1 on this change; while I can see that it's useful for you the parametrize interface is already more complicated than I'd like it to be, and you've also demonstrated that it's reasonably easy to define this functionality in your own code.

[RonnyPfannschmidt]

i believe we should investigate adding a general format function

as a sibling to idfn we might want to have idfmt="{value} in {collection.__class__.__name__}" as a easy way to spell those

[Zac-HD]

Closing as duplicate of #6185