Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution][Endpoint] Fix and unskip flaky test #149841

Merged
merged 8 commits into from
Feb 1, 2023

Conversation

ashokaditya
Copy link
Member

@ashokaditya ashokaditya commented Jan 30, 2023

Note
Merge after /pull/149839

Summary

Fixes flaky test /issues/145204

flaky test runners

Checklist

Sorry, something went wrong.

@ashokaditya ashokaditya self-assigned this Jan 30, 2023
@ashokaditya ashokaditya added release_note:skip Skip the PR/issue when compiling release notes test-failure-flaky Team:Defend Workflows “EDR Workflows” sub-team of Security Solution OLM Sprint v8.7.0 labels Jan 30, 2023
@ashokaditya ashokaditya changed the title [Security Solution][Endpoint] Fix flaky test #145204 [Security Solution][Endpoint] Fix and unskip flaky test Jan 31, 2023
@ashokaditya ashokaditya marked this pull request as ready for review January 31, 2023 15:09
@ashokaditya ashokaditya requested a review from a team as a code owner January 31, 2023 15:09
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-defend-workflows (Team:Defend Workflows)

Comment on lines 31 to 33
afterEach(() => {
useUserPrivilegesMock.mockImplementation(getUserPrivilegesMockDefaultValue);
useUserPrivilegesMock.mockReset();
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why this solves the issue? The mock is setup in all of the flaky test cases - how can this have an effect on those?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the mockReset is the right way to reset and remove all mocked return values.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with you on this, I just don't see how this is connected to the flaky tests. Do you see the connection?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My best guess is that it's because we're using useCanSeeHostIsolationExceptionsMenu in 8.6 tests and that clashes with the mockImplementation of useUserPrivilegesMock. I notice now that we're actually missing a test for HIE page visibility when useCanSeeHostIsolationExceptionsMenu is true in 8.6. Moreover we're not using the useCanSeeHostIsolationExceptionsMenu in those 8.6 tests. I'll create a PR for 8.6 with that test. Also another one for main that uses the canReadHostIsolationExceptions for HIE page visibility.

Looks like the test was flaky again on the last test after this PR was merged to main.

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @ashokaditya

@ashokaditya ashokaditya merged commit 9b85acb into elastic:main Feb 1, 2023
@ashokaditya ashokaditya deleted the task/dw-fix-flaky-test-145204 branch February 1, 2023 14:57
@kibanamachine
Copy link
Contributor

💔 All backports failed

Status Branch Result
8.6 Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 149841

Questions ?

Please refer to the Backport tool documentation

@ashokaditya
Copy link
Member Author

💚 All backports created successfully

Status Branch Result
8.6

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

ashokaditya added a commit to ashokaditya/kibana that referenced this pull request Feb 1, 2023
ashokaditya added a commit that referenced this pull request Feb 1, 2023
… (#150070)

# Backport

This will backport the following commits from `main` to `8.6`:
- [[Security Solution][Endpoint] Fix and unskip flaky test
(#149841)](#149841)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT
[{"author":{"name":"Ashokaditya","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-02-01T14:57:05Z","message":"[Security
Solution][Endpoint] Fix and unskip flaky test (#149841)\n\n>
**Note**\r\n> **Merge after /pull/149839**\r\n\r\n##
Summary\r\n\r\nFixes flaky test
/issues/145204\r\n\r\nflaky test runners
\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1817\r\nx
50 (failed on a single
[unrelated\r\n](https://github.com/ashokaditya/kibana/blob/92cb000a2f5116fc7408f52794cea06ad40de4bb/x-pack/test/security_solution_endpoint/apps/endpoint/artifact_entries_list.ts#L75)flaky\r\ntest)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1826\r\nx
150 (failed on a single run for
an\r\n[unrelated](https://github.com/ashokaditya/kibana/blob/92cb000a2f5116fc7408f52794cea06ad40de4bb/x-pack/test/security_solution_endpoint/apps/endpoint/artifact_entries_list.ts#L87)\r\nflaky
test)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1834\r\nx
200 (successful on all
runs)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1847\r\nx
100 (successful on all runs)\r\n\r\n### Checklist\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios","sha":"9b85acb49e821f16b886c608787b285d813198b6","branchLabelMapping":{"^v8.7.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-failure-flaky","Team:Defend
Workflows","OLM
Sprint","v8.6.0","v8.7.0"],"number":149841,"url":"https://github.com/elastic/kibana/pull/149841","mergeCommit":{"message":"[Security
Solution][Endpoint] Fix and unskip flaky test (#149841)\n\n>
**Note**\r\n> **Merge after /pull/149839**\r\n\r\n##
Summary\r\n\r\nFixes flaky test
/issues/145204\r\n\r\nflaky test runners
\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1817\r\nx
50 (failed on a single
[unrelated\r\n](https://github.com/ashokaditya/kibana/blob/92cb000a2f5116fc7408f52794cea06ad40de4bb/x-pack/test/security_solution_endpoint/apps/endpoint/artifact_entries_list.ts#L75)flaky\r\ntest)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1826\r\nx
150 (failed on a single run for
an\r\n[unrelated](https://github.com/ashokaditya/kibana/blob/92cb000a2f5116fc7408f52794cea06ad40de4bb/x-pack/test/security_solution_endpoint/apps/endpoint/artifact_entries_list.ts#L87)\r\nflaky
test)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1834\r\nx
200 (successful on all
runs)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1847\r\nx
100 (successful on all runs)\r\n\r\n### Checklist\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios","sha":"9b85acb49e821f16b886c608787b285d813198b6"}},"sourceBranch":"main","suggestedTargetBranches":["8.6"],"targetPullRequestStates":[{"branch":"8.6","label":"v8.6.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.7.0","labelRegex":"^v8.7.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/149841","number":149841,"mergeCommit":{"message":"[Security
Solution][Endpoint] Fix and unskip flaky test (#149841)\n\n>
**Note**\r\n> **Merge after /pull/149839**\r\n\r\n##
Summary\r\n\r\nFixes flaky test
/issues/145204\r\n\r\nflaky test runners
\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1817\r\nx
50 (failed on a single
[unrelated\r\n](https://github.com/ashokaditya/kibana/blob/92cb000a2f5116fc7408f52794cea06ad40de4bb/x-pack/test/security_solution_endpoint/apps/endpoint/artifact_entries_list.ts#L75)flaky\r\ntest)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1826\r\nx
150 (failed on a single run for
an\r\n[unrelated](https://github.com/ashokaditya/kibana/blob/92cb000a2f5116fc7408f52794cea06ad40de4bb/x-pack/test/security_solution_endpoint/apps/endpoint/artifact_entries_list.ts#L87)\r\nflaky
test)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1834\r\nx
200 (successful on all
runs)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1847\r\nx
100 (successful on all runs)\r\n\r\n### Checklist\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios","sha":"9b85acb49e821f16b886c608787b285d813198b6"}}]}]
BACKPORT-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OLM Sprint release_note:skip Skip the PR/issue when compiling release notes Team:Defend Workflows “EDR Workflows” sub-team of Security Solution test-failure-flaky v8.6.0 v8.6.2 v8.7.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants