-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Security Solution][Endpoint] Fix and unskip flaky test #149841
[Security Solution][Endpoint] Fix and unskip flaky test #149841
Conversation
Pinging @elastic/security-defend-workflows (Team:Defend Workflows) |
afterEach(() => { | ||
useUserPrivilegesMock.mockImplementation(getUserPrivilegesMockDefaultValue); | ||
useUserPrivilegesMock.mockReset(); | ||
}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder why this solves the issue? The mock is setup in all of the flaky test cases - how can this have an effect on those?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe the mockReset
is the right way to reset and remove all mocked return values.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with you on this, I just don't see how this is connected to the flaky tests. Do you see the connection?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My best guess is that it's because we're using useCanSeeHostIsolationExceptionsMenu
in 8.6
tests and that clashes with the mockImplementation of useUserPrivilegesMock
. I notice now that we're actually missing a test for HIE page visibility when useCanSeeHostIsolationExceptionsMenu
is true
in 8.6
. Moreover we're not using the useCanSeeHostIsolationExceptionsMenu
in those 8.6
tests. I'll create a PR for 8.6
with that test. Also another one for main that uses the canReadHostIsolationExceptions
for HIE page visibility.
Looks like the test was flaky again on the last test after this PR was merged to main.
💚 Build Succeeded
Metrics [docs]
History
To update your PR or re-run it, just comment with: cc @ashokaditya |
💔 All backports failed
Manual backportTo create the backport manually run:
Questions ?Please refer to the Backport tool documentation |
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
> **Note** > **Merge after elastic/pull/149839** ## Summary Fixes flaky test elastic/issues/145204 flaky test runners - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1817 x 50 (failed on a single [unrelated ](https://github.com/ashokaditya/kibana/blob/92cb000a2f5116fc7408f52794cea06ad40de4bb/x-pack/test/security_solution_endpoint/apps/endpoint/artifact_entries_list.ts#L75)flaky test) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1826 x 150 (failed on a single run for an [unrelated](https://github.com/ashokaditya/kibana/blob/92cb000a2f5116fc7408f52794cea06ad40de4bb/x-pack/test/security_solution_endpoint/apps/endpoint/artifact_entries_list.ts#L87) flaky test) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1834 x 200 (successful on all runs) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1847 x 100 (successful on all runs) ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios (cherry picked from commit 9b85acb) # Conflicts: # x-pack/plugins/security_solution/public/management/pages/integration_tests/index.test.tsx
… (#150070) # Backport This will backport the following commits from `main` to `8.6`: - [[Security Solution][Endpoint] Fix and unskip flaky test (#149841)](#149841) <!--- Backport version: 8.9.7 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Ashokaditya","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-02-01T14:57:05Z","message":"[Security Solution][Endpoint] Fix and unskip flaky test (#149841)\n\n> **Note**\r\n> **Merge after /pull/149839**\r\n\r\n## Summary\r\n\r\nFixes flaky test /issues/145204\r\n\r\nflaky test runners \r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1817\r\nx 50 (failed on a single [unrelated\r\n](https://github.com/ashokaditya/kibana/blob/92cb000a2f5116fc7408f52794cea06ad40de4bb/x-pack/test/security_solution_endpoint/apps/endpoint/artifact_entries_list.ts#L75)flaky\r\ntest)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1826\r\nx 150 (failed on a single run for an\r\n[unrelated](https://github.com/ashokaditya/kibana/blob/92cb000a2f5116fc7408f52794cea06ad40de4bb/x-pack/test/security_solution_endpoint/apps/endpoint/artifact_entries_list.ts#L87)\r\nflaky test)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1834\r\nx 200 (successful on all runs)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1847\r\nx 100 (successful on all runs)\r\n\r\n### Checklist\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios","sha":"9b85acb49e821f16b886c608787b285d813198b6","branchLabelMapping":{"^v8.7.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-failure-flaky","Team:Defend Workflows","OLM Sprint","v8.6.0","v8.7.0"],"number":149841,"url":"https://github.com/elastic/kibana/pull/149841","mergeCommit":{"message":"[Security Solution][Endpoint] Fix and unskip flaky test (#149841)\n\n> **Note**\r\n> **Merge after /pull/149839**\r\n\r\n## Summary\r\n\r\nFixes flaky test /issues/145204\r\n\r\nflaky test runners \r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1817\r\nx 50 (failed on a single [unrelated\r\n](https://github.com/ashokaditya/kibana/blob/92cb000a2f5116fc7408f52794cea06ad40de4bb/x-pack/test/security_solution_endpoint/apps/endpoint/artifact_entries_list.ts#L75)flaky\r\ntest)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1826\r\nx 150 (failed on a single run for an\r\n[unrelated](https://github.com/ashokaditya/kibana/blob/92cb000a2f5116fc7408f52794cea06ad40de4bb/x-pack/test/security_solution_endpoint/apps/endpoint/artifact_entries_list.ts#L87)\r\nflaky test)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1834\r\nx 200 (successful on all runs)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1847\r\nx 100 (successful on all runs)\r\n\r\n### Checklist\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios","sha":"9b85acb49e821f16b886c608787b285d813198b6"}},"sourceBranch":"main","suggestedTargetBranches":["8.6"],"targetPullRequestStates":[{"branch":"8.6","label":"v8.6.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.7.0","labelRegex":"^v8.7.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/149841","number":149841,"mergeCommit":{"message":"[Security Solution][Endpoint] Fix and unskip flaky test (#149841)\n\n> **Note**\r\n> **Merge after /pull/149839**\r\n\r\n## Summary\r\n\r\nFixes flaky test /issues/145204\r\n\r\nflaky test runners \r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1817\r\nx 50 (failed on a single [unrelated\r\n](https://github.com/ashokaditya/kibana/blob/92cb000a2f5116fc7408f52794cea06ad40de4bb/x-pack/test/security_solution_endpoint/apps/endpoint/artifact_entries_list.ts#L75)flaky\r\ntest)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1826\r\nx 150 (failed on a single run for an\r\n[unrelated](https://github.com/ashokaditya/kibana/blob/92cb000a2f5116fc7408f52794cea06ad40de4bb/x-pack/test/security_solution_endpoint/apps/endpoint/artifact_entries_list.ts#L87)\r\nflaky test)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1834\r\nx 200 (successful on all runs)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1847\r\nx 100 (successful on all runs)\r\n\r\n### Checklist\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios","sha":"9b85acb49e821f16b886c608787b285d813198b6"}}]}] BACKPORT-->
> **Note** > **Merge after elastic/pull/149839** ## Summary Fixes flaky test elastic/issues/145204 flaky test runners - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1817 x 50 (failed on a single [unrelated ](https://github.com/ashokaditya/kibana/blob/92cb000a2f5116fc7408f52794cea06ad40de4bb/x-pack/test/security_solution_endpoint/apps/endpoint/artifact_entries_list.ts#L75)flaky test) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1826 x 150 (failed on a single run for an [unrelated](https://github.com/ashokaditya/kibana/blob/92cb000a2f5116fc7408f52794cea06ad40de4bb/x-pack/test/security_solution_endpoint/apps/endpoint/artifact_entries_list.ts#L87) flaky test) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1834 x 200 (successful on all runs) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1847 x 100 (successful on all runs) ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
Summary
Fixes flaky test /issues/145204
flaky test runners
Checklist