-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Security Solution][Endpoint] Unskip flaky tests (document signing & response console - isolate) #169539
Merged
ashokaditya
merged 23 commits into
elastic:main
from
ashokaditya:task/unskip-document-signing-flaky-test
Nov 1, 2023
Merged
[Security Solution][Endpoint] Unskip flaky tests (document signing & response console - isolate) #169539
ashokaditya
merged 23 commits into
elastic:main
from
ashokaditya:task/unskip-document-signing-flaky-test
Nov 1, 2023
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ashokaditya
added
release_note:skip
Skip the PR/issue when compiling release notes
Team:Defend Workflows
“EDR Workflows” sub-team of Security Solution
OLM Sprint
v8.12.0
labels
Oct 23, 2023
Pinging @elastic/security-defend-workflows (Team:Defend Workflows) |
the agent id to use for tampered data must be an agent with elastic agent installed.
ashokaditya
changed the title
[Security Solution][Endpoint] Unskip document signing flaky test
[Security Solution][Endpoint] Unskip flaky tests (document signing & response console - isolate)
Oct 30, 2023
pzl
approved these changes
Oct 30, 2023
kevinlog
approved these changes
Oct 30, 2023
The test was flaky as it depends on the previous `isolate` test to isolate the host. If it doesn't then the release test can not assert that the host is isolated to start with when doing to `release` test. This makes the test independent from the previous isolate.
ashokaditya
force-pushed
the
task/unskip-document-signing-flaky-test
branch
from
November 1, 2023 08:42
14270a6
to
473a98a
Compare
ashokaditya
force-pushed
the
task/unskip-document-signing-flaky-test
branch
from
November 1, 2023 09:09
a9babde
to
3d56416
Compare
💚 Build Succeeded
Metrics [docs]
History
To update your PR or re-run it, just comment with: cc @ashokaditya |
💔 All backports failed
Manual backportTo create the backport manually run:
Questions ?Please refer to the Backport tool documentation |
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
ashokaditya
added a commit
to ashokaditya/kibana
that referenced
this pull request
Nov 2, 2023
…response console - isolate) (elastic#169539) ## Summary Split test to avoid long running times. Fixes elastic#168296 elastic#168390 **Flaky test runner** - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3754 x 150 (3 failures for document signing: 7, 101, 141) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3809 x 50 (all pass for document_signing.cy.ts) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3835 x 50 (2 fails for document signing and 1 fail for response console release due to timeout on response delay) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3842 x 25 (all pass for document_singing.cy.ts and response_console_actions.cy.ts) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3843 x 25 (3 fails for document_signing.cy.ts related to timeout for action response) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3844 x 100 (all pass for release, 3 failures for document signing due to time out waiting for action response) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3848 x 50 ( all pass, 1 unrelated failure related to `endpoints_list_response_console.cy·ts`) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3849 x 50 (all pass) ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios (cherry picked from commit a3d71f2) # Conflicts: # x-pack/plugins/security_solution/public/management/cypress/e2e/response_actions/response_console_actions.cy.ts
ashokaditya
added a commit
that referenced
this pull request
Nov 2, 2023
…ning & response console - isolate) (#169539) (#170381) # Backport This will backport the following commits from `main` to `8.11`: - [[Security Solution][Endpoint] Unskip flaky tests (document signing & response console - isolate) (#169539)](#169539) <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Ash","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-11-01T20:33:36Z","message":"[Security Solution][Endpoint] Unskip flaky tests (document signing & response console - isolate) (#169539)\n\n## Summary\r\n\r\nSplit test to avoid long running times.\r\n\r\nFixes\r\nhttps://github.com//issues/168296\r\nhttps://github.com//issues/168390\r\n\r\n**Flaky test runner**\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3754\r\nx 150 (3 failures for document signing: 7, 101, 141)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3809\r\nx 50 (all pass for document_signing.cy.ts)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3835\r\nx 50 (2 fails for document signing and 1 fail for response console\r\nrelease due to timeout on response delay)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3842\r\nx 25 (all pass for document_singing.cy.ts and\r\nresponse_console_actions.cy.ts)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3843\r\nx 25 (3 fails for document_signing.cy.ts related to timeout for action\r\nresponse)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3844\r\nx 100 (all pass for release, 3 failures for document signing due to time\r\nout waiting for action response)\r\n\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3848\r\nx 50 ( all pass, 1 unrelated failure related to\r\n`endpoints_list_response_console.cy·ts`)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3849\r\nx 50 (all pass)\r\n### Checklist\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios","sha":"a3d71f2534a56dd070982d6d8c60e56aa43ae2fa","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Defend Workflows","OLM Sprint","v8.11.0","v8.12.0"],"number":169539,"url":"https://github.com/elastic/kibana/pull/169539","mergeCommit":{"message":"[Security Solution][Endpoint] Unskip flaky tests (document signing & response console - isolate) (#169539)\n\n## Summary\r\n\r\nSplit test to avoid long running times.\r\n\r\nFixes\r\nhttps://github.com//issues/168296\r\nhttps://github.com//issues/168390\r\n\r\n**Flaky test runner**\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3754\r\nx 150 (3 failures for document signing: 7, 101, 141)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3809\r\nx 50 (all pass for document_signing.cy.ts)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3835\r\nx 50 (2 fails for document signing and 1 fail for response console\r\nrelease due to timeout on response delay)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3842\r\nx 25 (all pass for document_singing.cy.ts and\r\nresponse_console_actions.cy.ts)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3843\r\nx 25 (3 fails for document_signing.cy.ts related to timeout for action\r\nresponse)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3844\r\nx 100 (all pass for release, 3 failures for document signing due to time\r\nout waiting for action response)\r\n\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3848\r\nx 50 ( all pass, 1 unrelated failure related to\r\n`endpoints_list_response_console.cy·ts`)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3849\r\nx 50 (all pass)\r\n### Checklist\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios","sha":"a3d71f2534a56dd070982d6d8c60e56aa43ae2fa"}},"sourceBranch":"main","suggestedTargetBranches":["8.11"],"targetPullRequestStates":[{"branch":"8.11","label":"v8.11.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/169539","number":169539,"mergeCommit":{"message":"[Security Solution][Endpoint] Unskip flaky tests (document signing & response console - isolate) (#169539)\n\n## Summary\r\n\r\nSplit test to avoid long running times.\r\n\r\nFixes\r\nhttps://github.com//issues/168296\r\nhttps://github.com//issues/168390\r\n\r\n**Flaky test runner**\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3754\r\nx 150 (3 failures for document signing: 7, 101, 141)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3809\r\nx 50 (all pass for document_signing.cy.ts)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3835\r\nx 50 (2 fails for document signing and 1 fail for response console\r\nrelease due to timeout on response delay)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3842\r\nx 25 (all pass for document_singing.cy.ts and\r\nresponse_console_actions.cy.ts)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3843\r\nx 25 (3 fails for document_signing.cy.ts related to timeout for action\r\nresponse)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3844\r\nx 100 (all pass for release, 3 failures for document signing due to time\r\nout waiting for action response)\r\n\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3848\r\nx 50 ( all pass, 1 unrelated failure related to\r\n`endpoints_list_response_console.cy·ts`)\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3849\r\nx 50 (all pass)\r\n### Checklist\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios","sha":"a3d71f2534a56dd070982d6d8c60e56aa43ae2fa"}}]}] BACKPORT-->
delanni
pushed a commit
to delanni/kibana
that referenced
this pull request
Nov 6, 2023
…response console - isolate) (elastic#169539) ## Summary Split test to avoid long running times. Fixes elastic#168296 elastic#168390 **Flaky test runner** - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3754 x 150 (3 failures for document signing: 7, 101, 141) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3809 x 50 (all pass for document_signing.cy.ts) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3835 x 50 (2 fails for document signing and 1 fail for response console release due to timeout on response delay) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3842 x 25 (all pass for document_singing.cy.ts and response_console_actions.cy.ts) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3843 x 25 (3 fails for document_signing.cy.ts related to timeout for action response) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3844 x 100 (all pass for release, 3 failures for document signing due to time out waiting for action response) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3848 x 50 ( all pass, 1 unrelated failure related to `endpoints_list_response_console.cy·ts`) - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3849 x 50 (all pass) ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
OLM Sprint
release_note:skip
Skip the PR/issue when compiling release notes
Team:Defend Workflows
“EDR Workflows” sub-team of Security Solution
v8.11.0
v8.12.0
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Split test to avoid long running times.
Fixes
#168296
#168390
Flaky test runner
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3754 x 150 (3 failures for document signing: 7, 101, 141)
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3809 x 50 (all pass for document_signing.cy.ts)
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3835 x 50 (2 fails for document signing and 1 fail for response console release due to timeout on response delay)
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3842 x 25 (all pass for document_singing.cy.ts and response_console_actions.cy.ts)
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3843 x 25 (3 fails for document_signing.cy.ts related to timeout for action response)
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3844 x 100 (all pass for release, 3 failures for document signing due to time out waiting for action response)
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3848 x 50 ( all pass, 1 unrelated failure related to
endpoints_list_response_console.cy·ts
)https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3849 x 50 (all pass)
Checklist