Failing test: Security Solution Cypress.x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists/value_lists·cy·ts - value lists management modal export list types exports a "ip" list from an uploaded file exports a "ip" list from an uploaded file #164056
Assignees
Labels
failed-test
A test failure on a tracked branch, potentially flaky-test
Team:Detection Engine
Security Solution Detection Engine Area
Team:Detections and Resp
Security Detection Response Team
Team: SecuritySolution
Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.
triage_needed
Comments
kibanamachine
added
the
failed-test
kibanamachine
added
the
Team:Detection Engine
banderror
added
triage_needed
Team:Detections and Resp
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
maximpn
added a commit
that referenced
this issue
Aug 21, 2023
**Fixes:** #164056 ## Summary This PR fixes [value_lists.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists/value_lists.cy.ts) tests flakiness. ## The flakiness reason Value list items are processed in a bulk via bulk creation and `refresh=wait_for` is [used](https://github.com/elastic/kibana/blob/main/x-pack/plugins/lists/server/services/items/create_list_items_bulk.ts#L87). The problem it returns sometimes earlier than data is available. [Bulk API docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#bulk-refresh) say the following > Only the shards that receive the bulk request will be affected by refresh. Imagine a _bulk?refresh=wait_for request with three documents in it that happen to be routed to different shards in an index with five shards. The request will only wait for those three shards to refresh. The other two shards that make up the index do not participate in the _bulk request at all. While (it seems) only one shard is used in tests but it still cause issues (approx. 1 test per 50 fails) so adding explicit index refresh helps to get rid of flakiness. ## Flaky test runner [value_lists.cy.ts (150 runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2924) 🟢
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Aug 21, 2023
**Fixes:** elastic#164056 ## Summary This PR fixes [value_lists.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists/value_lists.cy.ts) tests flakiness. ## The flakiness reason Value list items are processed in a bulk via bulk creation and `refresh=wait_for` is [used](https://github.com/elastic/kibana/blob/main/x-pack/plugins/lists/server/services/items/create_list_items_bulk.ts#L87). The problem it returns sometimes earlier than data is available. [Bulk API docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#bulk-refresh) say the following > Only the shards that receive the bulk request will be affected by refresh. Imagine a _bulk?refresh=wait_for request with three documents in it that happen to be routed to different shards in an index with five shards. The request will only wait for those three shards to refresh. The other two shards that make up the index do not participate in the _bulk request at all. While (it seems) only one shard is used in tests but it still cause issues (approx. 1 test per 50 fails) so adding explicit index refresh helps to get rid of flakiness. ## Flaky test runner [value_lists.cy.ts (150 runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2924) 🟢 (cherry picked from commit d34c845)
kibanamachine
referenced
this issue
Aug 21, 2023
…164324) # Backport This will backport the following commits from `main` to `8.10`: - [[Security Solution] Fix value lists tests flakiness (#164253)](#164253) <!--- Backport version: 8.9.7 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-08-21T14:31:21Z","message":"[Security Solution] Fix value lists tests flakiness (#164253)\n\n**Fixes:** https://github.com/elastic/kibana/issues/164056\r\n\r\n## Summary\r\n\r\nThis PR fixes [value_lists.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists/value_lists.cy.ts) tests flakiness.\r\n\r\n## The flakiness reason\r\n\r\nValue list items are processed in a bulk via bulk creation and `refresh=wait_for` is [used](https://github.com/elastic/kibana/blob/main/x-pack/plugins/lists/server/services/items/create_list_items_bulk.ts#L87). The problem it returns sometimes earlier than data is available. [Bulk API docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#bulk-refresh) say the following\r\n\r\n> Only the shards that receive the bulk request will be affected by refresh. Imagine a _bulk?refresh=wait_for request with three documents in it that happen to be routed to different shards in an index with five shards. The request will only wait for those three shards to refresh. The other two shards that make up the index do not participate in the _bulk request at all.\r\n\r\nWhile (it seems) only one shard is used in tests but it still cause issues (approx. 1 test per 50 fails) so adding explicit index refresh helps to get rid of flakiness.\r\n\r\n## Flaky test runner\r\n\r\n[value_lists.cy.ts (150 runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2924) 🟢","sha":"d34c845955e2cb1cd6bae630ac3d0d551544f916","branchLabelMapping":{"^v8.11.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","Team:Detections and Resp","Team: SecuritySolution","Feature:Rule Value Lists","Team:Detection Rule Management","v8.10.0","v8.11.0"],"number":164253,"url":"https://github.com/elastic/kibana/pull/164253","mergeCommit":{"message":"[Security Solution] Fix value lists tests flakiness (#164253)\n\n**Fixes:** https://github.com/elastic/kibana/issues/164056\r\n\r\n## Summary\r\n\r\nThis PR fixes [value_lists.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists/value_lists.cy.ts) tests flakiness.\r\n\r\n## The flakiness reason\r\n\r\nValue list items are processed in a bulk via bulk creation and `refresh=wait_for` is [used](https://github.com/elastic/kibana/blob/main/x-pack/plugins/lists/server/services/items/create_list_items_bulk.ts#L87). The problem it returns sometimes earlier than data is available. [Bulk API docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#bulk-refresh) say the following\r\n\r\n> Only the shards that receive the bulk request will be affected by refresh. Imagine a _bulk?refresh=wait_for request with three documents in it that happen to be routed to different shards in an index with five shards. The request will only wait for those three shards to refresh. The other two shards that make up the index do not participate in the _bulk request at all.\r\n\r\nWhile (it seems) only one shard is used in tests but it still cause issues (approx. 1 test per 50 fails) so adding explicit index refresh helps to get rid of flakiness.\r\n\r\n## Flaky test runner\r\n\r\n[value_lists.cy.ts (150 runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2924) 🟢","sha":"d34c845955e2cb1cd6bae630ac3d0d551544f916"}},"sourceBranch":"main","suggestedTargetBranches":["8.10"],"targetPullRequestStates":[{"branch":"8.10","label":"v8.10.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.11.0","labelRegex":"^v8.11.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/164253","number":164253,"mergeCommit":{"message":"[Security Solution] Fix value lists tests flakiness (#164253)\n\n**Fixes:** https://github.com/elastic/kibana/issues/164056\r\n\r\n## Summary\r\n\r\nThis PR fixes [value_lists.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/value_lists/value_lists.cy.ts) tests flakiness.\r\n\r\n## The flakiness reason\r\n\r\nValue list items are processed in a bulk via bulk creation and `refresh=wait_for` is [used](https://github.com/elastic/kibana/blob/main/x-pack/plugins/lists/server/services/items/create_list_items_bulk.ts#L87). The problem it returns sometimes earlier than data is available. [Bulk API docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#bulk-refresh) say the following\r\n\r\n> Only the shards that receive the bulk request will be affected by refresh. Imagine a _bulk?refresh=wait_for request with three documents in it that happen to be routed to different shards in an index with five shards. The request will only wait for those three shards to refresh. The other two shards that make up the index do not participate in the _bulk request at all.\r\n\r\nWhile (it seems) only one shard is used in tests but it still cause issues (approx. 1 test per 50 fails) so adding explicit index refresh helps to get rid of flakiness.\r\n\r\n## Flaky test runner\r\n\r\n[value_lists.cy.ts (150 runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2924) 🟢","sha":"d34c845955e2cb1cd6bae630ac3d0d551544f916"}}]}] BACKPORT--> Co-authored-by: Maxim Palenov <[email protected]>
|
Fixed by #164253. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
A test failed on a tracked branch
First failure: CI Build - main
The text was updated successfully, but these errors were encountered: