Skip to content

Use @<data-set-name> format (compatible with --filter CLI option) in defect messages #6272

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 12, 2025
Merged

Use @<data-set-name> format (compatible with --filter CLI option) in defect messages #6272

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
2ae35dd
Simplify `TestCase->dataSetAsString()`
staabm Jul 12, 2025
43469ae
Refactor: extract `dataSetAsFilterString`
staabm Jul 12, 2025
59ee1ef
Use "@<data-set-name>" --filter CLI option compatible error format
staabm Jul 12, 2025
32769ea
added more tests
staabm Jul 12, 2025
c5e600a
naming
staabm Jul 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 25 additions & 8 deletions src/Framework/TestCase.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -843,17 +843,15 @@ final public function dataName(): int|string
*/
final public function dataSetAsString(): string
Copy link
Contributor Author

@staabm staabm Jul 12, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since TestDox formatter does not use the className::methodName <dataset> format, I did not change the format in this method itself but created a new dataSetAsFilterString for use only in the default-printer

{
$buffer = '';

if ($this->data !== []) {
if (is_int($this->dataName)) {
$buffer .= sprintf(' with data set #%d', $this->dataName);
} else {
$buffer .= sprintf(' with data set "%s"', $this->dataName);
return sprintf(' with data set %s', $this->dataSetAsFilterString());
}

return sprintf(' with data set "%s"', $this->dataSetAsFilterString());
}

return $buffer;
return '';
}

/**
Expand All @@ -865,8 +863,9 @@ final public function dataSetAsStringWithData(): string
return '';
}

return $this->dataSetAsString() . sprintf(
' (%s)',
return sprintf(
'%s with data (%s)',
$this->dataSetAsFilterString(),
Exporter::shortenedRecursiveExport($this->data),
);
}
Expand Down Expand Up @@ -1268,6 +1267,24 @@ protected function onNotSuccessfulTest(Throwable $t): never
throw $t;
}

/**
* Returns the data set as a string compatible with the --filter CLI option.
*
* @internal This method is not covered by the backward compatibility promise for PHPUnit
*/
private function dataSetAsFilterString(): string
{
if ($this->data !== []) {
if (is_int($this->dataName)) {
return sprintf('#%d', $this->dataName);
}

return sprintf('@%s', $this->dataName);
}

return '';
}

/**
* @throws AssertionFailedError
* @throws Exception
Expand Down
2 changes: 1 addition & 1 deletion src/Runner/Filter/NameFilterIterator.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ private function prepareFilter(string $filter): array
// * testDetermineJsonError@JSON.*
elseif (preg_match('/^(.*?)@(.+)$/', $filter, $matches)) {
$filter = sprintf(
'%s.*with data set "%s"$',
'%s.*with data set "@%s"$',
$matches[1],
$matches[2],
);
Expand Down
2 changes: 1 addition & 1 deletion tests/end-to-end/data-provider/dependency-result.phpt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Time: %s, Memory: %s

There was 1 error:

1) PHPUnit\TestFixture\DataProviderDependencyResultTest::testAdd with data set #2 (2, 0)
1) PHPUnit\TestFixture\DataProviderDependencyResultTest::testAdd#2 with data (2, 0)
Error: Cannot use positional argument after named argument during unpacking

ERRORS!
Expand Down
2 changes: 1 addition & 1 deletion tests/end-to-end/logging/teamcity/teamcity-data-provider.phpt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Time: %s, Memory: %s

There was 1 failure:

1) PHPUnit\TestFixture\TeamCity\DataProviderTest::testOne with data set #1 (false)
1) PHPUnit\TestFixture\TeamCity\DataProviderTest::testOne#1 with data (false)
Failed asserting that false is true.

%sDataProviderTest.php:28
Expand Down
2 changes: 1 addition & 1 deletion tests/end-to-end/regression/5616.phpt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Time: %s, Memory: %s MB

There was 1 failure:

1) PHPUnit\TestFixture\Issue5616\Issue5616Test::testOne with data set #0 (1, '2', 3.0, true)
1) PHPUnit\TestFixture\Issue5616\Issue5616Test::testOne#0 with data (1, '2', 3.0, true)
Failed asserting that false is true.

%sIssue5616Test.php:%d
Expand Down
6 changes: 3 additions & 3 deletions tests/end-to-end/regression/6222.phpt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,17 +26,17 @@ Failed asserting that false is true.

%sIssue6222Test.php:%d

2) PHPUnit\TestFixture\Issue6222\Issue6222Test::testOneCasePassing with data set #1 (2)
2) PHPUnit\TestFixture\Issue6222\Issue6222Test::testOneCasePassing#1 with data (2)
Failed asserting that 2 is identical to 1.

%sIssue6222Test.php:%d

3) PHPUnit\TestFixture\Issue6222\Issue6222Test::testZeroCasesPassing with data set #0 (1)
3) PHPUnit\TestFixture\Issue6222\Issue6222Test::testZeroCasesPassing#0 with data (1)
Failed asserting that 1 is identical to 3.

%sIssue6222Test.php:%d

4) PHPUnit\TestFixture\Issue6222\Issue6222Test::testZeroCasesPassing with data set #1 (2)
4) PHPUnit\TestFixture\Issue6222\Issue6222Test::testZeroCasesPassing#1 with data (2)
Failed asserting that 2 is identical to 3.

%sIssue6222Test.php:%d
Expand Down
4 changes: 2 additions & 2 deletions tests/end-to-end/testdox/data-provider-with-string-data-set-name.phpt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ Runtime: %s
Time: %s, Memory: %s

Data Provider With String Data Set Name (PHPUnit\TestFixture\TestDox\DataProviderWithStringDataSetName)
✔ Something that works with data set "data set name"
✘ Something that does not work with data set "data set name"
✔ Something that works with data set "@data set name"
✘ Something that does not work with data set "@data set name"
│
│ Failed asserting that false is true.
│
Expand Down
2 changes: 1 addition & 1 deletion tests/end-to-end/testdox/formatter-method-does-not-exist.phpt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Formatter Method Does Not Exist (PHPUnit\TestFixture\TestDox\FormatterMethodDoes

There was 1 PHPUnit error:

1) PHPUnit\TestFixture\TestDox\FormatterMethodDoesNotExistTest::testOne with data set #0 ('string')
1) PHPUnit\TestFixture\TestDox\FormatterMethodDoesNotExistTest::testOne#0 with data ('string')
Method PHPUnit\TestFixture\TestDox\FormatterMethodDoesNotExistTest::formatter() cannot be used as a TestDox formatter because it does not exist

%sFormatterMethodDoesNotExistTest.php:%d
Expand Down
2 changes: 1 addition & 1 deletion tests/end-to-end/testdox/formatter-method-is-not-public.phpt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Formatter Method Is Not Public (PHPUnit\TestFixture\TestDox\FormatterMethodIsNot

There was 1 PHPUnit error:

1) PHPUnit\TestFixture\TestDox\FormatterMethodIsNotPublicTest::testOne with data set #0 ('string')
1) PHPUnit\TestFixture\TestDox\FormatterMethodIsNotPublicTest::testOne#0 with data ('string')
Method PHPUnit\TestFixture\TestDox\FormatterMethodIsNotPublicTest::formatter() cannot be used as a TestDox formatter because it is not public

%sFormatterMethodIsNotPublicTest.php:%d
Expand Down
2 changes: 1 addition & 1 deletion tests/end-to-end/testdox/formatter-method-is-not-static.phpt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Formatter Method Is Not Static (PHPUnit\TestFixture\TestDox\FormatterMethodIsNot

There was 1 PHPUnit error:

1) PHPUnit\TestFixture\TestDox\FormatterMethodIsNotStaticTest::testOne with data set #0 ('string')
1) PHPUnit\TestFixture\TestDox\FormatterMethodIsNotStaticTest::testOne#0 with data ('string')
Method PHPUnit\TestFixture\TestDox\FormatterMethodIsNotStaticTest::formatter() cannot be used as a TestDox formatter because it is not static

%sFormatterMethodIsNotStaticTest.php:%d
Expand Down
2 changes: 1 addition & 1 deletion tests/end-to-end/testdox/formatter-method-throws-exception.phpt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Formatter Method Throws Exception (PHPUnit\TestFixture\TestDox\FormatterMethodTh

There was 1 PHPUnit error:

1) PHPUnit\TestFixture\TestDox\FormatterMethodThrowsExceptionTest::testOne with data set #0 ('string')
1) PHPUnit\TestFixture\TestDox\FormatterMethodThrowsExceptionTest::testOne#0 with data ('string')
TestDox formatter PHPUnit\TestFixture\TestDox\FormatterMethodThrowsExceptionTest::formatter() triggered an error: message
%sFormatterMethodThrowsExceptionTest.php:%d

Expand Down
4 changes: 2 additions & 2 deletions tests/end-to-end/testdox/metadata-data-provider-with-string-data-set-name.phpt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ Runtime: %s
Time: %s, Memory: %s

Text from class-level TestDox metadata
✔ Text from method-level TestDox metadata for successful test with data set "data set name"
✘ Text from method-level TestDox metadata for failing test with data set "data set name"
✔ Text from method-level TestDox metadata for successful test with data set "@data set name"
✘ Text from method-level TestDox metadata for failing test with data set "@data set name"
│
│ Failed asserting that false is true.
│
Expand Down
19 changes: 19 additions & 0 deletions tests/unit/Framework/TestCaseTest.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@

use function sprintf;
use PHPUnit\Framework\Attributes\CoversClass;
use PHPUnit\Framework\Attributes\DataProvider;
use PHPUnit\Framework\Attributes\ExcludeGlobalVariableFromBackup;
use PHPUnit\TestFixture\TestWithDifferentNames;

Expand All @@ -21,6 +22,15 @@ class TestCaseTest extends TestCase
{
protected static int $testStatic = 456;

public static function provideDataSetAsStringWithDataProvider(): iterable
{
yield ['', 'dataSet', []];

yield ["#0 with data ('foo', 'bar')", 0, ['foo', 'bar']];

yield ["@dataSet with data ('foo', 'bar')", 'dataSet', ['foo', 'bar']];
}

public static function setUpBeforeClass(): void
{
$GLOBALS['a'] = 'a';
Expand Down Expand Up @@ -83,4 +93,13 @@ public function testGetNameReturnsMethodName(): void

$this->assertSame($methodName, $testCase->nameWithDataSet());
}

#[DataProvider('provideDataSetAsStringWithDataProvider')]
public function testDataSetAsStringWithData(string $expectedData, int|string $dataName, array $data): void
{
$testCase = new TestWithDifferentNames('testWithName');
$testCase->setData($dataName, $data);

$this->assertSame($expectedData, $testCase->dataSetAsStringWithData());
}
}
2 changes: 1 addition & 1 deletion tests/unit/Logging/TestDox/NamePrettifierTest.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ public function __toString(): string
false,
],
[
'Two with data set "a"',
'Two with data set "@a"',
$testWithDataWithStringKey,
false,
],
Expand Down
58 changes: 29 additions & 29 deletions tests/unit/Runner/TestSuiteSorterTest.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -448,19 +448,19 @@ public static function defectsSorterWithDataProviderProvider(): array
TestSuiteSorter::ORDER_DEFAULT,
self::IGNORE_DEPENDENCIES,
[
'testOne' => ['state' => TestStatus::success(), 'time' => 1],
'testWithProvider with data set "good1"' => ['state' => TestStatus::success(), 'time' => 1],
'testWithProvider with data set "good2"' => ['state' => TestStatus::success(), 'time' => 1],
'testWithProvider with data set "good3"' => ['state' => TestStatus::success(), 'time' => 1],
'testWithProvider with data set "fail1"' => ['state' => TestStatus::success(), 'time' => 1],
'testWithProvider with data set "fail2"' => ['state' => TestStatus::success(), 'time' => 1],
],
[
FaillingDataProviderTest::class . '::testWithProvider with data set "good1"',
FaillingDataProviderTest::class . '::testWithProvider with data set "good2"',
FaillingDataProviderTest::class . '::testWithProvider with data set "fail1"',
FaillingDataProviderTest::class . '::testWithProvider with data set "good3"',
FaillingDataProviderTest::class . '::testWithProvider with data set "fail2"',
'testOne' => ['state' => TestStatus::success(), 'time' => 1],
'testWithProvider with data set "@good1"' => ['state' => TestStatus::success(), 'time' => 1],
'testWithProvider with data set "@good2"' => ['state' => TestStatus::success(), 'time' => 1],
'testWithProvider with data set "@good3"' => ['state' => TestStatus::success(), 'time' => 1],
'testWithProvider with data set "@fail1"' => ['state' => TestStatus::success(), 'time' => 1],
'testWithProvider with data set "@fail2"' => ['state' => TestStatus::success(), 'time' => 1],
],
[
FaillingDataProviderTest::class . '::testWithProvider with data set "@good1"',
FaillingDataProviderTest::class . '::testWithProvider with data set "@good2"',
FaillingDataProviderTest::class . '::testWithProvider with data set "@fail1"',
FaillingDataProviderTest::class . '::testWithProvider with data set "@good3"',
FaillingDataProviderTest::class . '::testWithProvider with data set "@fail2"',
FaillingDataProviderTest::class . '::testOne',
],
],
Expand All @@ -473,11 +473,11 @@ public static function defectsSorterWithDataProviderProvider(): array
// empty result cache
],
[
FaillingDataProviderTest::class . '::testWithProvider with data set "good1"',
FaillingDataProviderTest::class . '::testWithProvider with data set "good2"',
FaillingDataProviderTest::class . '::testWithProvider with data set "fail1"',
FaillingDataProviderTest::class . '::testWithProvider with data set "good3"',
FaillingDataProviderTest::class . '::testWithProvider with data set "fail2"',
FaillingDataProviderTest::class . '::testWithProvider with data set "@good1"',
FaillingDataProviderTest::class . '::testWithProvider with data set "@good2"',
FaillingDataProviderTest::class . '::testWithProvider with data set "@fail1"',
FaillingDataProviderTest::class . '::testWithProvider with data set "@good3"',
FaillingDataProviderTest::class . '::testWithProvider with data set "@fail2"',
FaillingDataProviderTest::class . '::testOne',
],
],
Expand All @@ -486,19 +486,19 @@ public static function defectsSorterWithDataProviderProvider(): array
TestSuiteSorter::ORDER_DEFAULT,
self::IGNORE_DEPENDENCIES,
[
'testOne' => ['state' => TestStatus::success(), 'time' => 1],
'testWithProvider with data set "good1"' => ['state' => TestStatus::success(), 'time' => 1],
'testWithProvider with data set "good2"' => ['state' => TestStatus::success(), 'time' => 1],
'testWithProvider with data set "good3"' => ['state' => TestStatus::success(), 'time' => 1],
'testWithProvider with data set "fail1"' => ['state' => TestStatus::error(), 'time' => 1],
'testWithProvider with data set "fail2"' => ['state' => TestStatus::error(), 'time' => 1],
'testOne' => ['state' => TestStatus::success(), 'time' => 1],
'testWithProvider with data set "@good1"' => ['state' => TestStatus::success(), 'time' => 1],
'testWithProvider with data set "@good2"' => ['state' => TestStatus::success(), 'time' => 1],
'testWithProvider with data set "@good3"' => ['state' => TestStatus::success(), 'time' => 1],
'testWithProvider with data set "@fail1"' => ['state' => TestStatus::error(), 'time' => 1],
'testWithProvider with data set "@fail2"' => ['state' => TestStatus::error(), 'time' => 1],
],
[
FaillingDataProviderTest::class . '::testWithProvider with data set "fail1"',
FaillingDataProviderTest::class . '::testWithProvider with data set "fail2"',
FaillingDataProviderTest::class . '::testWithProvider with data set "good1"',
FaillingDataProviderTest::class . '::testWithProvider with data set "good2"',
FaillingDataProviderTest::class . '::testWithProvider with data set "good3"',
FaillingDataProviderTest::class . '::testWithProvider with data set "@fail1"',
FaillingDataProviderTest::class . '::testWithProvider with data set "@fail2"',
FaillingDataProviderTest::class . '::testWithProvider with data set "@good1"',
FaillingDataProviderTest::class . '::testWithProvider with data set "@good2"',
FaillingDataProviderTest::class . '::testWithProvider with data set "@good3"',
FaillingDataProviderTest::class . '::testOne',
],
],
Expand Down