Output from test run in separate process is printed twice

[yurii-stickee-2023]

Q	A
PHPUnit version	11.5.3
PHP version	8.3.16
Installation Method	Composer

Summary

Recently I upgraded from PHPUnit 9.x and refactored test with @runInSeparateProcess into #[\PHPUnit\Framework\Attributes\RunInSeparateProcess] .
By accident I saw that test is executed twice now under PHPUnit 11.x (I am testing debug_headers(), and first test run has no headers, while second test run has expected headers).

Current behavior

Test with #[\PHPUnit\Framework\Attributes\RunInSeparateProcess] or @runInSeparateProcess is executed twice.

How to reproduce

1. create test case at tests/TestAnnotation.php

<?php

/**
 * @coversNothing
 */
class TestAnnotation extends \PHPUnit\Framework\TestCase
{
    public static int $calls = 0;

    public function test_case_1_set()
    {
        $GLOBALS['asd'] = 123;
        $this->assertTrue(true);
    }

    /**
     * @runInSeparateProcess
     * @preserveGlobalState disabled
     */
    public function test_case_2_check()
    {
        var_dump(++self::$calls);
        $this->assertTrue(true);
        $this->assertFalse(isset($GLOBALS['asd']));
    }
}

2. install PHPUnit
3. run test as ./vendor/bin/phpunit tests/TestAnnotation.php

I run it with 9.x (under PHP 7.4), 10.x and 11.x versions.
Here is my output

./vendor/bin/phpunit tests/TestAnnotation.php 
PHPUnit 9.6.22 by Sebastian Bergmann and contributors.

..                                                                  2 / 2 (100%)/var/www/test-phpunit/tests/TestAnnotation.php:25:
int(1)


Time: 00:00.052, Memory: 4.00 MB

OK (2 tests, 3 assertions)

./vendor/bin/phpunit tests/TestAnnotation.php 
PHPUnit 10.5.41 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.16

./var/www/test-phpunit/tests/TestAnnotation.php:22:
int(1)
.                                                                  2 / 2 (100%)/var/www/test-phpunit/tests/TestAnnotation.php:22:
int(1)


Time: 00:00.059, Memory: 8.00 MB

OK (2 tests, 3 assertions)

./vendor/bin/phpunit tests/TestAnnotation.php 
PHPUnit 11.5.3 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.16

.int(1)
.                                                                  2 / 2 (100%)int(1)


Time: 00:00.066, Memory: 8.00 MB

OK, but there were issues!
Tests: 2, Assertions: 3, PHPUnit Deprecations: 4.

Expected behavior

Since PHPUnit 10 test with runInSeparateProcess (annotation or PHP attribute) is executed twice.
My expectation was that it should execute only once in separate process, as it was in PHPUnit 9.x

[sebastianbergmann]

As far as I can see, the test is only executed once, but its output is printed twice (which is a bug, of course):

Issue6103Test.php

<?php declare(strict_types=1);
namespace PHPUnit\TestFixture\Issue6103;

use PHPUnit\Framework\Attributes\RunInSeparateProcess;
use PHPUnit\Framework\TestCase;

final class Issue6103Test extends TestCase
{
    #[RunInSeparateProcess]
    public function testOne(): void
    {
        print '*';

        file_put_contents('/tmp/debug', '*', FILE_APPEND | LOCK_EX);

        $this->assertTrue(true);
    }
}

phpunit --debug Issue6103Test.php

❯ rm /tmp/debug

❯ phpunit --debug Issue6103Test.php
PHPUnit Started (PHPUnit 11.5.3-8-g09ce2ee174 using PHP 8.4.3 (cli) on Linux)
Test Runner Configured
Event Facade Sealed
Test Suite Loaded (1 test)
Test Runner Started
Test Suite Sorted
Test Runner Execution Started (1 test)
Test Suite Started (PHPUnit\TestFixture\Issue6103\Issue6103Test, 1 test)
Child Process Started
Test Preparation Started (PHPUnit\TestFixture\Issue6103\Issue6103Test::testOne)
Test Prepared (PHPUnit\TestFixture\Issue6103\Issue6103Test::testOne)
Test Passed (PHPUnit\TestFixture\Issue6103\Issue6103Test::testOne)
Test Printed Unexpected Output
*
Test Finished (PHPUnit\TestFixture\Issue6103\Issue6103Test::testOne)
Child Process Finished
*Test Suite Finished (PHPUnit\TestFixture\Issue6103\Issue6103Test, 1 test)
Test Runner Execution Finished
Test Runner Finished
PHPUnit Finished (Shell Exit Code: 0)

❯ cat /tmp/debug
*

If the test were executed twice then /tmp/debug would contain ** instead of *.

phpunit Issue6103Test.php

PHPUnit 11.5.3-8-g09ce2ee174 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.4.3

*.                                                                   1 / 1 (100%)*

Time: 00:00.065, Memory: 8.00 MB

OK (1 test, 1 assertion)