AssertionFailedError loses stacktrace in process isolation when chaining with a previous exception

[mondrake]

Q	A
PHPUnit version	9.5.24
PHP version	n/a
Installation Method	Composer

Summary

Identified in #4983.

When a AssertionFailedError is thrown as a re-throw of a previous exception (for instance as part of a onNotSuccessfulTest() callback), and the previous exception is chained in, the failure reported by PHPUnit traces to the part where AssertionFailedError was thrown, and the original trace is lost. So there is no information about where the test failed.

Current behavior

PHPUnit 9.5.24 #StandWithUkraine

Runtime:       PHP 8.1.10
Configuration: /home/runner/work/d8-unit/d8-unit/core/phpunit.xml.dist

Testing Drupal\Tests\node\Functional\NodeTitleTest
F                                                                   1 / 1 (100%)

Time: 00:06.500, Memory: 10.00 MB

There was 1 failure:

1) Drupal\Tests\node\Functional\NodeTitleTest::testNodeTitle
Title found

/home/runner/work/d8-unit/d8-unit/core/tests/Drupal/Tests/BrowserTestBase.php:441
/home/runner/work/d8-unit/d8-unit/core/tests/Drupal/Tests/Traits/UnsuccessfulTestHandlerTrait.php:16
/home/runner/work/d8-unit/d8-unit/vendor/phpunit/phpunit/src/Framework/TestResult.php:726

FAILURES!
Tests: 1, Assertions: 17, Failures: 1

How to reproduce

Implement a onNotSuccessfulTest() callback in a test, and add logic so that a Throwable is re-thrown as a AssertionFailedError with the Throwable chained in. For example,

    if ($t instanceof MinkException) {
      throw new AssertionFailedError($t->getMessage(), $t->getCode(), $t);
    }

Expected behavior

PHPUnit 9.5.24 #StandWithUkraine

Runtime:       PHP 8.1.10
Configuration: /home/runner/work/d8-unit/d8-unit/core/phpunit.xml.dist

Testing Drupal\Tests\node\Functional\NodeTitleTest
F                                                                   1 / 1 (100%)

Time: 00:06.500, Memory: 10.00 MB

There was 1 failure:

1) Drupal\Tests\node\Functional\NodeTitleTest::testNodeTitle
Title found

/home/runner/work/d8-unit/d8-unit/core/tests/Drupal/Tests/BrowserTestBase.php:441
/home/runner/work/d8-unit/d8-unit/core/tests/Drupal/Tests/Traits/UnsuccessfulTestHandlerTrait.php:16
/home/runner/work/d8-unit/d8-unit/vendor/phpunit/phpunit/src/Framework/TestResult.php:726

Caused by Behat\Mink\Exception\ExpectationException: Title found
/home/runner/work/d8-unit/d8-unit/core/tests/Drupal/Tests/WebAssert.php:538
/home/runner/work/d8-unit/d8-unit/core/tests/Drupal/Tests/WebAssert.php:281
/home/runner/work/d8-unit/d8-unit/core/modules/node/tests/src/Functional/NodeTitleTest.php:94
/home/runner/work/d8-unit/d8-unit/vendor/phpunit/phpunit/src/Framework/TestResult.php:726

FAILURES!
Tests: 1, Assertions: 17, Failures: 1.

[mondrake]

Will file a PR soon.

[mondrake]

Running tests, I realized this occurs only when the test is executed in a separate process. Which is the case of Drupal functional tests where we found this issue. When executed within the same process, the stack trace is printed. Thanks @alexpott for pointing this out.

[sebastianbergmann]

I will leave this ticket open for now, but please note that I will not work on this as I do not think that this edge case is worth pursuing.

I strongly recommend to change your implementation of onNotSuccessfulTest() to not rely on exception chaining. AFACIS, all that needs to be done is mapping the information from Mink's ExpectationException to PHPUnit's AssertionFailedError.

[mondrake]

Let's not waste time, then.