Code Coverage does not work for tests run in separate process(es)

[sebastianbergmann]

phpunit.xml

<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/10.0/phpunit.xsd"
         bootstrap="src/Email.php"
         cacheDirectory=".phpunit.cache"
         executionOrder="depends,defects"
         requireCoverageMetadata="true"
         beStrictAboutCoverageMetadata="true"
         beStrictAboutOutputDuringTests="true"
         failOnRisky="true"
         failOnWarning="true">
    <testsuites>
        <testsuite name="default">
            <directory>tests</directory>
        </testsuite>
    </testsuites>

    <coverage>
        <include>
            <directory suffix=".php">src</directory>
        </include>
    </coverage>
</phpunit>

src/Email.php

<?php declare(strict_types=1);
final class Email
{
    private $email;

    private function __construct(string $email)
    {
        $this->ensureIsValidEmail($email);

        $this->email = $email;
    }

    public static function fromString(string $email): self
    {
        return new self($email);
    }

    public function __toString(): string
    {
        return $this->email;
    }

    private function ensureIsValidEmail(string $email): void
    {
        if (!filter_var($email, FILTER_VALIDATE_EMAIL)) {
            throw new InvalidArgumentException(
                sprintf(
                    '"%s" is not a valid email address',
                    $email
                )
            );
        }
    }
}

tests/EmailTest.php

<?php declare(strict_types=1);
use PHPUnit\Framework\Attributes\CoversClass;
use PHPUnit\Framework\TestCase;

#[CoversClass(Email::class)]
final class EmailTest extends TestCase
{
    public function testCanBeCreatedFromValidEmailAddress(): void
    {
        $this->assertInstanceOf(
            Email::class,
            Email::fromString('[email protected]')
        );
    }

    public function testCannotBeCreatedFromInvalidEmailAddress(): void
    {
        $this->expectException(InvalidArgumentException::class);

        Email::fromString('invalid');
    }

    public function testCanBeUsedAsString(): void
    {
        $this->assertEquals(
            '[email protected]',
            Email::fromString('[email protected]')
        );
    }
}

Process Isolation

./vendor/bin/phpunit --coverage-html /tmp/coverage --process-isolation
PHPUnit 10.0.9 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.2.3 with PCOV 1.0.11
Configuration: /home/sb/5197/phpunit.xml

...                                                                 3 / 3 (100%)

Time: 00:00.214, Memory: 6.00 MB

OK (3 tests, 3 assertions)

Generating code coverage report in HTML format ... done [00:00.013]

No Process Isolation

./vendor/bin/phpunit --coverage-html /tmp/coverage                    
PHPUnit 10.0.9 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.2.3 with PCOV 1.0.11
Configuration: /home/sb/5197/phpunit.xml

...                                                                 3 / 3 (100%)

Time: 00:00.047, Memory: 8.00 MB

OK (3 tests, 3 assertions)

Generating code coverage report in HTML format ... done [00:00.005]

[sebastianbergmann]

This problem was introduced between PHPUnit 10.0.5 and PHPUnit 10.0.6:

PHPUnit 10.0.5

phpunit --process-isolation --coverage-html /tmp/coverage
PHPUnit 10.0.5 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.2.3 with PCOV 1.0.11
Configuration: /home/sb/5197/phpunit.xml

...                                                                 3 / 3 (100%)

Time: 00:00.258, Memory: 6.00 MB

OK (3 tests, 3 assertions)

Generating code coverage report in HTML format ... done [00:00.016]

PHPUnit 10.0.6

PHPUnit 10.0.6 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.2.3 with PCOV 1.0.11
Configuration: /home/sb/5197/phpunit.xml

...                                                                 3 / 3 (100%)

Time: 00:00.232, Memory: 6.00 MB

OK (3 tests, 3 assertions)

Generating code coverage report in HTML format ... done [00:00.008]

[sebastianbergmann]

@Slamdunk It looks like your refactoring(s) between PHPUnit 10.0.5 and PHPUnit 10.0.6 cause this issue. Can you have a look, please? Thanks!

[kukulich]

Oh no - I've just spent two hours of debugging to find the same bug 😅

[sebastianbergmann]

Oh no - I've just spent two hours of debugging to find the same bug 😅

Sorry :(

[Slamdunk]

@sebastianbergmann sure, stay tuned 📻

[kukulich]

Oh no - I've just spent two hours of debugging to find the same bug 😅

Sorry :(

Never mind :) I'm happy we finally find the cause :)