Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[frontend] Flaky tests in CompareV2 resulting in false failures in PRs #10811

Closed
Tracked by #11153
HumairAK opened this issue May 9, 2024 · 5 comments
Closed
Tracked by #11153

[frontend] Flaky tests in CompareV2 resulting in false failures in PRs #10811

HumairAK opened this issue May 9, 2024 · 5 comments
Assignees
@ElayAharoni
Labels
area/frontend kind/bug

Comments

@HumairAK
Copy link
Collaborator

HumairAK commented May 9, 2024 •
edited
Loading

Tests in CompareV2.test.tsx seem to be very Flaky, and keep resulting in random failures in PRs, I constantly have to re-run these tests to get it fully passing. Some examples of failure errors I've seen:

failures.1 
Summary of all failing tests

FAIL src/pages/CompareV2.test.tsx (91.424 s)

● CompareV2 › Parameters and Scalar metrics tab initially enabled with loading then error, and switch tabs

thrown: "Exceeded timeout of 5000 ms for a test.

Use jest.setTimeout(newTimeout) to increase the timeout value, if this is a long-running test."

410 | });

411 |

> 412 | it('Parameters and Scalar metrics tab initially enabled with loading then error, and switch tabs', async () => {

| ^

413 | const getRunSpy = jest.spyOn(Apis.runServiceApiV2, 'getRun');

414 | runs = [newMockRun(MOCK_RUN_1_ID), newMockRun(MOCK_RUN_2_ID), newMockRun(MOCK_RUN_3_ID)];

415 | getRunSpy.mockImplementation((id: string) => runs.find(r => r.run_id === id));

at src/pages/CompareV2.test.tsx:412:3

at Object.<anonymous> (src/pages/CompareV2.test.tsx:34:1)

Test Suites: 1 failed, 113 passed, 114 total

Tests: 1 failed, 1611 passed, 1612 total

Snapshots: 427 passed, 427 total

Time: 238.994 s
failures.2 
● CompareV2 › Parameters and Scalar metrics tab initially enabled with loading then error, and switch tabs

Unable to find an element with the text: An error is preventing the Scalar Metrics from being displayed.. This could be because the text is broken up by multiple elements. In this case, you can provide a function for your text matcher to make your matcher more flexible.

<body>
....
OMITTED
....

426 | screen.getByText('There are no Parameters available on the selected runs.');

427 | screen.getByText('An error is preventing the Scalar Metrics from being displayed.');

428 |

at waitForWrapper (node_modules/@testing-library/react/node_modules/@testing-library/dom/dist/wait-for.js:173:27)

at Object.<anonymous> (src/pages/CompareV2.test.tsx:425:18)

Examples:

Love this idea? Give it a 👍.
@HumairAK HumairAK changed the title [frontend] Flakey tests in CompareV2 resulting in false failures in PRs [frontend] Flaky tests in CompareV2 resulting in false failures in PRs May 9, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 9, 2024

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the lifecycle/stale The issue / pull request is stale, any activities remove this label. label Jul 9, 2024
@HumairAK
Copy link
Collaborator Author

HumairAK commented Jul 9, 2024

/remove-lifecycle stale

@google-oss-prow google-oss-prow bot removed the lifecycle/stale The issue / pull request is stale, any activities remove this label. label Jul 9, 2024
@hbelmiro hbelmiro mentioned this issue Aug 29, 2024
Closed
8 tasks
@ElayAharoni
Copy link
Contributor

/assign

ElayAharoni pushed a commit to ElayAharoni/pipelines that referenced this issue Sep 5, 2024
fix: Flaky tests in CompareV2 resulting in false failures in PRs kube…
54f04e8 
…flow#10811

Signed-off-by: Elay Aharoni (EXT-Nokia) <[email protected]>
@ElayAharoni
Copy link
Contributor

tried to reproduce the flaky tests with no success.
3 times via pushing to github and running the test, and each time all the tests passed.
and 5 more times locally also did not see the flaky tests.

might be not-reproduceable

@ElayAharoni
Copy link
Contributor

@HumairAK after closing #11170
we might be able to close this issue?

@ElayAharoni ElayAharoni mentioned this issue Sep 17, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ElayAharoni ElayAharoni

Labels
area/frontend kind/bug
Projects
None yet
Milestone
No milestone
2 participants
@HumairAK @ElayAharoni