sf force lightning lwc test run returns exit code 0 when tests fail

[jamessimone]

Summary

Using latest CLI version (@salesforce/cli/2.55.6 win32-x64 node-v20.16.0), these two commands differ when encountering failing LWC jest tests:

• npx sfdx-lwc-jest --skipApiVersionCheck - returns exit code 1
• sf force lightning lwc test run - returns exit code 0

This improperly suppresses failures when relying on exit codes (such as in pipeline runs)

Steps To Reproduce

describe('sample failure', () => {
  it('fails', () => {
    expect(1).toBe(2);
  })
})

Expected result

CLI should return exit code 1 when using sf force lightning lwc test run with any test failure

Actual result

CLI returns exit code 0

[github-actions]

Hello @jamessimone 👋 It looks like you didn't include the full Salesforce CLI version information in your issue.
Please provide the output of version --verbose --json for the CLI you're using (sf or sfdx).

A few more things to check:

• Make sure you've provided detailed steps to reproduce your issue.
  • A repository that clearly demonstrates the bug is ideal.
• Make sure you've installed the latest version of Salesforce CLI. (docs)
  • Better yet, try the rc or nightly versions. (docs)
• Try running the doctor command to diagnose common issues.
• Search GitHub for existing related issues.

Thank you!

[github-actions]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[github-actions]

This issue has not received a response in 7 days. It will auto-close in 7 days unless a response is posted.

[jamessimone]

This issue should not be closed

[cristiand391]

hey, sorry for the late reply.

the command should return the same exit code that sfdx-lwc-test does:
https://github.com/salesforcecli/plugin-lwc-test/blob/024530c2ac27f3097f4edb075ccb1cc3b14cabf4/src/commands/force/lightning/lwc/test/run.ts#L60

but looking at the e2e test it seems that hasn't been the case:
https://github.com/salesforcecli/plugin-lwc-test/blob/024530c2ac27f3097f4edb075ccb1cc3b14cabf4/test/commands/lwc/test/run.nut.ts#L84

we'll discuss with the owners of sfdx-lwc-test about the current state of the lwc commands and get back.

[github-actions]

This issue has not received a response in 7 days. It will auto-close in 7 days unless a response is posted.

[jamessimone]

Bump

[github-actions]

This issue has not received a response in 7 days. It will auto-close in 7 days unless a response is posted.

[jamessimone]

Bump

[github-actions]

This issue has not received a response in 7 days. It will auto-close in 7 days unless a response is posted.

[jamessimone]

Bump

[VivekMChawla]

@jamessimone - Thanks for keeping this bumped.

My team connected with the team that owns sfdx-lwc-jest

We confirmed that the behavior of sf force lightning lwc test run is incorrect. The command should exit with a status code of 1 if any tests fail.

The CLI team will explore a fix for this issue, but I can't provide a timeline.

[git2gus]

This issue has been linked to a new work item: W-16884223

[jamessimone]

@VivekMChawla thanks for keeping us updated, it's much appreciated

[jshackell-sfdc]

This issue has been fixed in CLI version 2.62.6 (October 16, 2024).

[jamessimone]

@jshackell-sfdc thanks!