Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Incorrect exit code with --fail-on-incomplete #1348

Open
wants to merge 1 commit into
base: 3.x
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 18 additions & 26 deletions src/Result.php
Original file line number Diff line number Diff line change
Expand Up @@ -40,44 +40,36 @@ public static function ok(Configuration $configuration, TestResult $result): boo
*/
public static function exitCode(Configuration $configuration, TestResult $result): int
{
if ($result->wasSuccessfulIgnoringPhpunitWarnings()) {
if ($configuration->failOnWarning()) {
$warnings = $result->numberOfTestsWithTestTriggeredPhpunitWarningEvents()
+ count($result->warnings())
+ count($result->phpWarnings());

if ($warnings > 0) {
return self::FAILURE_EXIT;
}
}

if (! $result->hasTestTriggeredPhpunitWarningEvents()) {
return self::SUCCESS_EXIT;
}
if ($result->hasTestErroredEvents()) {
return self::EXCEPTION_EXIT;
}

if ($configuration->failOnEmptyTestSuite() && ResultReflection::numberOfTests($result) === 0) {
return self::FAILURE_EXIT;
}

if ($result->wasSuccessfulIgnoringPhpunitWarnings()) {
if ($configuration->failOnRisky() && $result->hasTestConsideredRiskyEvents()) {
$returnCode = self::FAILURE_EXIT;
}
if ($configuration->failOnWarning()) {
$warnings = $result->numberOfTestsWithTestTriggeredPhpunitWarningEvents()
+ count($result->warnings())
+ count($result->phpWarnings());

if ($configuration->failOnIncomplete() && $result->hasTestMarkedIncompleteEvents()) {
$returnCode = self::FAILURE_EXIT;
if ($warnings > 0) {
return self::FAILURE_EXIT;
}
}

if ($configuration->failOnSkipped() && $result->hasTestSkippedEvents()) {
$returnCode = self::FAILURE_EXIT;
}
if ($configuration->failOnRisky() && $result->hasTestConsideredRiskyEvents()) {
return self::FAILURE_EXIT;
}

if ($result->hasTestErroredEvents()) {
return self::EXCEPTION_EXIT;
if ($configuration->failOnIncomplete() && $result->hasTestMarkedIncompleteEvents()) {
return self::FAILURE_EXIT;
}

if ($configuration->failOnSkipped() && $result->hasTestSkippedEvents()) {
return self::FAILURE_EXIT;
}

return self::FAILURE_EXIT;
return self::SUCCESS_EXIT;
}
}
6 changes: 5 additions & 1 deletion tests/.snapshots/success.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1679,6 +1679,10 @@
PASS Tests\Visual\Help
✓ visual snapshot of help command output

WARN Tests\Visual\Incompleted
- incompleted test with fail-on-incomplete returns non-zero exit code
- incompleted test with fail-on-incomplete returns non-zero exit code parallel

WARN Tests\Visual\JUnit
✓ junit output
- junit with parallel → Not working yet
Expand Down Expand Up @@ -1708,4 +1712,4 @@
WARN Tests\Visual\Version
- visual snapshot of help command output

Tests: 2 deprecated, 4 warnings, 5 incomplete, 2 notices, 38 todos, 33 skipped, 1152 passed (2744 assertions)
Tests: 2 deprecated, 4 warnings, 5 incomplete, 2 notices, 38 todos, 35 skipped, 1152 passed (2744 assertions)
31 changes: 31 additions & 0 deletions tests/Visual/Incompleted.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
<?php

use Symfony\Component\Process\Process;

$run = function (bool $parallel) {
$process = new Process(
['php', 'bin/pest', 'tests/Features/Incompleted.php', $parallel ? '--parallel' : '', '--fail-on-incomplete'],
dirname(__DIR__, 2),
['COLLISION_PRINTER' => 'DefaultPrinter', 'COLLISION_IGNORE_DURATION' => 'true'],
);

$process->run();

expect($process->getExitCode())->toBe(1);

return removeAnsiEscapeSequences($process->getOutput());
};

test('incompleted test with fail-on-incomplete returns non-zero exit code', function () use ($run) {
expect($run(false))
->toContain('Tests: 5 incomplete');
})
->skipOnWindows()
->skip(! getenv('REBUILD_SNAPSHOTS') && getenv('EXCLUDE'));

test('incompleted test with fail-on-incomplete returns non-zero exit code parallel', function () use ($run) {
expect($run(true))
->toContain('Tests: 5 incomplete');
})
->skipOnWindows()
->skip(! getenv('REBUILD_SNAPSHOTS') && getenv('EXCLUDE'));