No warning when CLI options are used multiple times

[ondrejmirtes]

Hi, right now it seems from my testing that passing --filter multiple times rewrites the previous occurrence so the first --filter option is silently discarded.

vendor/bin/phpunit --filter 'PHPStan\\Type\\UnionTypeTest::testSorting' --filter 'PHPStan\\Type\\UnionTypeTest::testAccepts#7' --verbose --debug
PHPUnit 9.5.23 #StandWithUkraine

Runtime:       PHP 8.3.1
Configuration: /Users/ondrej/Development/phpstan/phpunit.xml
Warning:       No code coverage driver available

Test 'PHPStan\Type\UnionTypeTest::testAccepts with data set #7 (PHPStan\Type\UnionType Object (...), PHPStan\Type\ObjectType Object (...), PHPStan\TrinaryLogic Object (...))' started
Test 'PHPStan\Type\UnionTypeTest::testAccepts with data set #7 (PHPStan\Type\UnionType Object (...), PHPStan\Type\ObjectType Object (...), PHPStan\TrinaryLogic Object (...))' ended

Time: 00:00.117, Memory: 570.00 MB

OK (1 test, 1 assertion)

I'd like to use --filter to be interpreted in a way - "I want to run tests that pass this filter and also tests that pass this other filter". So in this case PHPUnit should run the two tests I specified.

Even if you don't want to interpret multiple --filter options that way, I think something could be done about the silent discarding.

Thank you.

[sebastianbergmann]

May I kindly ask you to check whether PHPUnit 10.5 and PHPUnit 11.0 also silently ignore multiple --filter options? Thanks!

[sebastianbergmann]

Nevermind, I can confirm that --filter (as well as --exclude-filter) can be used only once and that when it is used multiple times the "last one parsed wins", silently ignoring the previously parsed ones. I consider this a bug that should be fixed for PHPUnit 10 and PHPUnit 11 (PHPUnit 9 no longer receives bug fixes).

I do not, however, think that multiple --filter options should be allowed.

[ondrejmirtes]

I do not, however, think that multiple --filter options should be allowed.

Yeah, I got a little bit obsessed recently with passing the same option multiple times after watching https://www.youtube.com/watch?v=aolI_Rz0ZqY (great talk btw) because git blame does different things whether you pass -C once, twice, or thrice. So I thought it could apply here but of course I'm fine if it doesn't do anything useful.

[sebastianbergmann]

All long options supported by PHPUnit's CLI test runner can only be used once. For instance, when --filter foo --filter bar is used then only --filter bar is used as it overwrites any --filter option that was given before.

As a first step, PHPUnit's CLI test runner will be changed (for PHPUnit >= 10.5) to emit a warning when an option is used multiple times. Future work will be done (for PHPUnit >= 11.1) to allow multiple --group and --exclude-group options, for example.

[ondrejmirtes]

Awesome, thanks!