Addon Test: Wait for 2 seconds before showing result mismatch warning

[ghengeveld]

Closes #29703

What I did

Waits for 2 seconds for results from browser and CLI to settle before showing discrepancy warning.

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

• stories
• unit tests
• integration tests
• end-to-end tests

Manual testing

Open a story with a play function and an assertion. Enable watch mode for Storybook component testing. Change the assertion to fail. Verify that the test fails in both CLI and browser, and no warning message is (however briefly) shown. Change back the assertion to pass and verify the same (no warning). Now add this to the play function:

// eslint-disable-next-line no-underscore-dangle
if (globalThis.__vitest_browser__) {
  throw new Error('Oops');
}

Verify that the warning does show with this code (failure in CLI, pass in browser).

Documentation

• Add or update documentation reflecting your changes
• If you are deprecating/removing a feature, make sure to update
  MIGRATION.MD

Checklist for Maintainers

• When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli-storybook/src/sandbox-templates.ts

• Make sure this PR contains one of the labels below:

  Available labels

  • bug: Internal changes that fixes incorrect behavior.
  • maintenance: User-facing maintenance tasks.
  • dependencies: Upgrading (sometimes downgrading) dependencies.
  • build: Internal-facing build tooling & test updates. Will not show up in release changelog.
  • cleanup: Minor cleanup style change. Will not show up in release changelog.
  • documentation: Documentation only changes. Will not show up in release changelog.
  • feature request: Introducing a new feature.
  • BREAKING CHANGE: Changes that break compatibility in some way with current major version.
  • other: Changes that don't fit in the above categories.

🦋 Canary release

This PR does not have a canary release associated. You can request a canary release of this pull request by mentioning the @storybookjs/core team here.

core team members can create a canary release here or locally with gh workflow run --repo storybookjs/storybook canary-release-pr.yml --field pr=<PR_NUMBER>

name	before	after	diff	z	%
createSize	0 B	0 B	0 B	-	-
generateSize	77.7 MB	77.7 MB	0 B	0.78	0%
initSize	133 MB	133 MB	0 B	2.09	0%
diffSize	55.3 MB	55.3 MB	0 B	2.17	0%
buildSize	6.87 MB	6.87 MB	0 B	-1.74	0%
buildSbAddonsSize	1.51 MB	1.51 MB	0 B	-1.73	0%
buildSbCommonSize	195 kB	195 kB	0 B	-	0%
buildSbManagerSize	1.86 MB	1.86 MB	0 B	-2	0%
buildSbPreviewSize	0 B	0 B	0 B	-	-
buildStaticSize	0 B	0 B	0 B	-	-
buildPrebuildSize	3.57 MB	3.57 MB	0 B	-1.74	0%
buildPreviewSize	3.3 MB	3.3 MB	0 B	-2	0%
testBuildSize	0 B	0 B	0 B	-	-
testBuildSbAddonsSize	0 B	0 B	0 B	-	-
testBuildSbCommonSize	0 B	0 B	0 B	-	-
testBuildSbManagerSize	0 B	0 B	0 B	-	-
testBuildSbPreviewSize	0 B	0 B	0 B	-	-
testBuildStaticSize	0 B	0 B	0 B	-	-
testBuildPrebuildSize	0 B	0 B	0 B	-	-
testBuildPreviewSize	0 B	0 B	0 B	-	-

name	before	after	diff	z	%
createTime	6.3s	6.9s	540ms	-0.77	7.8%
generateTime	19.3s	20.9s	1.6s	-0.29	7.7%
initTime	13.4s	13.7s	353ms	-0.39	2.6%
buildTime	8.3s	8.1s	-232ms	-1.84	-2.8%
testBuildTime	0ms	0ms	0ms	-	-
devPreviewResponsive	5.3s	5.7s	377ms	0.99	6.6%
devManagerResponsive	4.1s	4.2s	93ms	1.13	2.2%
devManagerHeaderVisible	528ms	619ms	91ms	0.54	14.7%
devManagerIndexVisible	557ms	659ms	102ms	0.41	15.5%
devStoryVisibleUncached	1.8s	2s	196ms	0.46	9.7%
devStoryVisible	558ms	656ms	98ms	0.42	14.9%
devAutodocsVisible	470ms	542ms	72ms	0.31	13.3%
devMDXVisible	466ms	517ms	51ms	-0.2	9.9%
buildManagerHeaderVisible	508ms	653ms	145ms	0.73	22.2%
buildManagerIndexVisible	611ms	736ms	125ms	0.65	17%
buildStoryVisible	471ms	588ms	117ms	0.82	19.9%
buildAutodocsVisible	393ms	549ms	156ms	1.48	🔺28.4%
buildMDXVisible	419ms	504ms	85ms	1.06	16.9%

Greptile Summary

This PR adds a 2-second delay mechanism before displaying result mismatch warnings in the test panel to prevent false positives during test execution in watch mode.

• Added timeout handling in code/addons/test/src/components/Panel.tsx to delay mismatch warnings
• Replaced useMemo with useEffect hook to manage timeout state and cleanup
• Introduced hasResultMismatch state to track discrepancies between browser and story test status
• Added proper timeout cleanup on component unmount or condition changes
• Fixed metrics reliability by preventing premature mismatch reporting

[greptile-apps]

_{1 file(s) reviewed, 1 comment(s)
Edit PR Review Bot Settings | Greptile}

[nx-cloud]

☁️ Nx Cloud Report

CI is running/has finished running commands for commit 30b6933. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this CI Pipeline Execution

---

✅ Successfully ran 1 target

• nx run-many -t build --parallel=3

---

Sent with 💌 from NxCloud.

[storybook-pr-benchmarking]

Package Benchmarks

^{Commit: 30b6933, ran on 12 December 2024 at 08:59:29 UTC}

No significant changes detected, all good. 👏