Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Addon Test: Fix error reporting for vitest crashes #29751

Merged

Conversation

ndelangen
Copy link
Member

@ndelangen ndelangen commented Nov 29, 2024

Closes #29721

What I did

When vitest fails to initialize, we should report this problem properly to the UI.

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

  • stories
  • unit tests
  • integration tests
  • end-to-end tests

Manual testing

This section is mandatory for all contributions. If you believe no manual test is necessary, please state so explicitly. Thanks!

Documentation

  • Add or update documentation reflecting your changes
  • If you are deprecating/removing a feature, make sure to update
    MIGRATION.MD

Checklist for Maintainers

  • When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli-storybook/src/sandbox-templates.ts

  • Make sure this PR contains one of the labels below:

    Available labels
    • bug: Internal changes that fixes incorrect behavior.
    • maintenance: User-facing maintenance tasks.
    • dependencies: Upgrading (sometimes downgrading) dependencies.
    • build: Internal-facing build tooling & test updates. Will not show up in release changelog.
    • cleanup: Minor cleanup style change. Will not show up in release changelog.
    • documentation: Documentation only changes. Will not show up in release changelog.
    • feature request: Introducing a new feature.
    • BREAKING CHANGE: Changes that break compatibility in some way with current major version.
    • other: Changes that don't fit in the above categories.

🦋 Canary release

This PR does not have a canary release associated. You can request a canary release of this pull request by mentioning the @storybookjs/core team here.

core team members can create a canary release here or locally with gh workflow run --repo storybookjs/storybook canary-release-pr.yml --field pr=<PR_NUMBER>

name before after diff z %
createSize 0 B 0 B 0 B - -
generateSize 77.7 MB 77.7 MB 2.43 kB 1.5 0%
initSize 130 MB 130 MB 16.3 kB -1.1 0%
diffSize 52.4 MB 52.4 MB 13.9 kB -1.1 0%
buildSize 6.83 MB 6.83 MB 43 B 1.19 0%
buildSbAddonsSize 1.51 MB 1.51 MB 0 B -0.03 0%
buildSbCommonSize 195 kB 195 kB 0 B - 0%
buildSbManagerSize 1.86 MB 1.86 MB 43 B 1.29 0%
buildSbPreviewSize 271 kB 271 kB 0 B -0.2 0%
buildStaticSize 0 B 0 B 0 B - -
buildPrebuildSize 3.84 MB 3.84 MB 43 B 1.16 0%
buildPreviewSize 3 MB 3 MB 0 B 1.1 0%
testBuildSize 0 B 0 B 0 B - -
testBuildSbAddonsSize 0 B 0 B 0 B - -
testBuildSbCommonSize 0 B 0 B 0 B - -
testBuildSbManagerSize 0 B 0 B 0 B - -
testBuildSbPreviewSize 0 B 0 B 0 B - -
testBuildStaticSize 0 B 0 B 0 B - -
testBuildPrebuildSize 0 B 0 B 0 B - -
testBuildPreviewSize 0 B 0 B 0 B - -
name before after diff z %
createTime 6.5s 7.3s 782ms -0.96 10.7%
generateTime 19.6s 19.7s 159ms -0.64 0.8%
initTime 13.1s 13.8s 619ms -0.48 4.5%
buildTime 10.6s 7.8s -2s -888ms -1.29 🔰-37%
testBuildTime 0ms 0ms 0ms - -
devPreviewResponsive 4.7s 5.4s 652ms 0.01 12%
devManagerResponsive 3.5s 4.1s 621ms 0.81 14.9%
devManagerHeaderVisible 511ms 566ms 55ms -0.1 9.7%
devManagerIndexVisible 591ms 648ms 57ms 0.05 8.8%
devStoryVisibleUncached 1.6s 1.7s 150ms 0.51 8.4%
devStoryVisible 540ms 647ms 107ms 0.11 16.5%
devAutodocsVisible 449ms 509ms 60ms -0.15 11.8%
devMDXVisible 341ms 474ms 133ms -0.29 28.1%
buildManagerHeaderVisible 536ms 501ms -35ms -0.37 -7%
buildManagerIndexVisible 548ms 515ms -33ms -0.37 -6.4%
buildStoryVisible 535ms 503ms -32ms -0.34 -6.4%
buildAutodocsVisible 430ms 439ms 9ms -0.21 2.1%
buildMDXVisible 401ms 409ms 8ms -0.35 2%

Greptile Summary

Based on the provided files and context, here's a concise summary of the changes:

Improves error handling and reporting for Vitest crashes in Storybook's testing module, particularly focusing on initialization failures and coverage dependency issues.

  • Added try-catch block in vitest-manager.ts to capture initialization failures and report them through testManager.reportFatalError()
  • Modified crash report handling in SidebarBottom.tsx to standardize error format with { name: 'Crashed!', message: details.error.message }
  • Enhanced error reporting UI feedback when Vitest fails to initialize or when coverage dependencies are missing
  • Improved state management for test provider crashes through updateTestProviderState API

The changes provide better visibility and more graceful handling of Vitest-related errors, particularly addressing issue #29721 regarding coverage dependency requirements.

@ndelangen ndelangen changed the title error reporting for vitest crashes AddonTest: Error reporting for vitest crashes Nov 29, 2024
@ndelangen ndelangen self-assigned this Nov 29, 2024
Copy link

nx-cloud bot commented Nov 29, 2024

☁️ Nx Cloud Report

CI is running/has finished running commands for commit 35714f8. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this CI Pipeline Execution


✅ Successfully ran 2 targets

Sent with 💌 from NxCloud.

Base automatically changed from jeppe/vitest-coverage-backend to next November 29, 2024 14:27
@ndelangen ndelangen marked this pull request as ready for review December 2, 2024 09:11
@ndelangen ndelangen merged commit 0fe1043 into next Dec 2, 2024
61 of 65 checks passed
@ndelangen ndelangen deleted the norbert/vitest-coverage-error-reporting-improvements branch December 2, 2024 09:11
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 file(s) reviewed, 2 comment(s)
Edit PR Review Bot Settings | Greptile

Comment on lines +83 to +87
try {
await this.vitest.init();
} catch (e) {
this.testManager.reportFatalError('Failed to init Vitest', e);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: vitest instance may be in an invalid state after init failure. Consider setting this.vitest = null in catch block to prevent subsequent operations on failed instance.

Comment on lines 124 to 126
api.updateTestProviderState(providerId, {
details,
error: { name: 'Crashed!', message: details.error.message },
running: false,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: Consider preserving the original error name from details.error.name instead of hardcoding 'Crashed!'

@github-actions github-actions bot mentioned this pull request Dec 2, 2024
11 tasks
@shilman shilman changed the title AddonTest: Error reporting for vitest crashes Addon Test: Error reporting for vitest crashes Dec 2, 2024
@shilman shilman changed the title Addon Test: Error reporting for vitest crashes Addon Test: Fix error reporting for vitest crashes Dec 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Gracefully handle coverage dependencies on @vitest/coverage-v8/@vitest/coverage-istanbul
2 participants