Failing test: Rules Management - Prebuilt Rules Management Integration Tests - ESS Env - Trial License.x-pack/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/management/trial_license_complete_tier/bootstrap_prebuilt_rules·ts - Rules Management - Prebuilt Rules - Prebuilt Rules Management @ess @serverless @skipInServerlessMKI Bootstrap Prebuilt Rules should skip installing fleet packages if they are already installed #197108

kibanamachine · 2024-10-21T17:37:18Z

A test failed on a tracked branch

Error: expected 200 "OK", got 500 "Internal Server Error"
    at Context.apply (bootstrap_prebuilt_rules.ts:46:58)
    at Object.apply (wrap_function.js:74:30)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
----
    at Test._assertStatus (node_modules/supertest/lib/test.js:252:14)
    at /opt/buildkite-agent/builds/bk-agent-prod-gcp-1729530268606186624/elastic/kibana-on-merge/kibana/node_modules/supertest/lib/test.js:308:13
    at Test._assertFunction (node_modules/supertest/lib/test.js:285:13)
    at Test.assert (node_modules/supertest/lib/test.js:164:23)
    at localAssert (node_modules/supertest/lib/test.js:120:14)
    at fn (node_modules/supertest/lib/test.js:125:7)
    at Test.callback (node_modules/superagent/src/node/index.js:899:3)
    at fn (node_modules/superagent/src/node/index.js:1165:18)
    at IncomingMessage.<anonymous> (node_modules/superagent/src/node/parsers/json.js:19:7)
    at IncomingMessage.emit (node:events:531:35)
    at endReadableNT (node:internal/streams/readable:1696:12)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)

First failure: kibana-on-merge - main

The text was updated successfully, but these errors were encountered:

elasticmachine · 2024-10-21T17:37:36Z

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

elasticmachine · 2024-10-22T09:23:57Z

Pinging @elastic/security-detections-response (Team:Detections and Resp)

elasticmachine · 2024-10-22T09:23:57Z

Pinging @elastic/security-solution (Team: SecuritySolution)

banderror · 2024-11-15T18:16:43Z

No repeated failures for 3 weeks, closing.

kibanamachine · 2024-12-10T19:32:36Z

New failure: kibana-on-merge - 8.17

kibanamachine · 2024-12-10T21:45:49Z

New failure: kibana-on-merge - 8.17

kibanamachine · 2024-12-11T05:27:41Z

Skipped

8.17: c84defd

@ess

…point (elastic#205106) **Fixes: elastic#197108 **Fixes: elastic#202037 **Fixes: elastic#203632 ## Summary Builds on top of elastic#203799 and: - simplifies the package deletion code (1 [API call](https://www.elastic.co/docs/api/doc/kibana/operation/operation-delete-fleet-epm-packages-pkgname-pkgversion) instead of 2, per package) - adds retry logic - refactors the testing utils a bit Flaky test runs: - [ESS](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7632) - 190 runs - [Serverless](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7633) - 100 runs ## Details Here's how the test logs look like: ``` info Starting tests └-: Rules Management - Prebuilt Rules - Prebuilt Rules Management └-> "before all" hook: beforeTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" └-: @ess @serverless @skipInServerlessMKI Bootstrap Prebuilt Rules └-> "before all" hook: beforeTestSuite.trigger for "should install fleet packages required for detection engine to function" └-> should install fleet packages required for detection engine to function └-> "before each" hook: global before each for "should install fleet packages required for detection engine to function" └-> "before each" hook for "should install fleet packages required for detection engine to function" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'security_detection_engine is not installed' │ } │ debg Deleting endpoint package │ debg Deleted endpoint package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'endpoint is not installed' │ } └- ✓ pass (21.6s) └-> should skip installing fleet packages if they are already installed └-> "before each" hook: global before each for "should skip installing fleet packages if they are already installed" └-> "before each" hook for "should skip installing fleet packages if they are already installed" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was installed) │ debg Deleting endpoint package │ debg Deleted endpoint package (was installed) └- ✓ pass (22.4s) └-> "after all" hook: afterTestSuite.trigger for "should skip installing fleet packages if they are already installed" └-> "after all" hook: afterTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" 2 passing (1.0m) ``` ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed (cherry picked from commit 71144ed) # Conflicts: # x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rules/prebuilt_rules/delete_endpoint_fleet_package.ts # x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rules/prebuilt_rules/delete_prebuilt_rules_fleet_package.ts

@ess

…point (elastic#205106) **Fixes: elastic#197108 **Fixes: elastic#202037 **Fixes: elastic#203632 ## Summary Builds on top of elastic#203799 and: - simplifies the package deletion code (1 [API call](https://www.elastic.co/docs/api/doc/kibana/operation/operation-delete-fleet-epm-packages-pkgname-pkgversion) instead of 2, per package) - adds retry logic - refactors the testing utils a bit Flaky test runs: - [ESS](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7632) - 190 runs - [Serverless](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7633) - 100 runs ## Details Here's how the test logs look like: ``` info Starting tests └-: Rules Management - Prebuilt Rules - Prebuilt Rules Management └-> "before all" hook: beforeTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" └-: @ess @serverless @skipInServerlessMKI Bootstrap Prebuilt Rules └-> "before all" hook: beforeTestSuite.trigger for "should install fleet packages required for detection engine to function" └-> should install fleet packages required for detection engine to function └-> "before each" hook: global before each for "should install fleet packages required for detection engine to function" └-> "before each" hook for "should install fleet packages required for detection engine to function" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'security_detection_engine is not installed' │ } │ debg Deleting endpoint package │ debg Deleted endpoint package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'endpoint is not installed' │ } └- ✓ pass (21.6s) └-> should skip installing fleet packages if they are already installed └-> "before each" hook: global before each for "should skip installing fleet packages if they are already installed" └-> "before each" hook for "should skip installing fleet packages if they are already installed" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was installed) │ debg Deleting endpoint package │ debg Deleted endpoint package (was installed) └- ✓ pass (22.4s) └-> "after all" hook: afterTestSuite.trigger for "should skip installing fleet packages if they are already installed" └-> "after all" hook: afterTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" 2 passing (1.0m) ``` ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed (cherry picked from commit 71144ed) # Conflicts: # x-pack/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/management/trial_license_complete_tier/bootstrap_prebuilt_rules.ts # x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rules/prebuilt_rules/delete_endpoint_fleet_package.ts # x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rules/prebuilt_rules/delete_prebuilt_rules_fleet_package.ts

@ess

…point (elastic#205106) **Fixes: elastic#197108 **Fixes: elastic#202037 **Fixes: elastic#203632 ## Summary Builds on top of elastic#203799 and: - simplifies the package deletion code (1 [API call](https://www.elastic.co/docs/api/doc/kibana/operation/operation-delete-fleet-epm-packages-pkgname-pkgversion) instead of 2, per package) - adds retry logic - refactors the testing utils a bit Flaky test runs: - [ESS](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7632) - 190 runs - [Serverless](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7633) - 100 runs ## Details Here's how the test logs look like: ``` info Starting tests └-: Rules Management - Prebuilt Rules - Prebuilt Rules Management └-> "before all" hook: beforeTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" └-: @ess @serverless @skipInServerlessMKI Bootstrap Prebuilt Rules └-> "before all" hook: beforeTestSuite.trigger for "should install fleet packages required for detection engine to function" └-> should install fleet packages required for detection engine to function └-> "before each" hook: global before each for "should install fleet packages required for detection engine to function" └-> "before each" hook for "should install fleet packages required for detection engine to function" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'security_detection_engine is not installed' │ } │ debg Deleting endpoint package │ debg Deleted endpoint package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'endpoint is not installed' │ } └- ✓ pass (21.6s) └-> should skip installing fleet packages if they are already installed └-> "before each" hook: global before each for "should skip installing fleet packages if they are already installed" └-> "before each" hook for "should skip installing fleet packages if they are already installed" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was installed) │ debg Deleting endpoint package │ debg Deleted endpoint package (was installed) └- ✓ pass (22.4s) └-> "after all" hook: afterTestSuite.trigger for "should skip installing fleet packages if they are already installed" └-> "after all" hook: afterTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" 2 passing (1.0m) ``` ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed (cherry picked from commit 71144ed) # Conflicts: # x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rules/prebuilt_rules/delete_endpoint_fleet_package.ts # x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rules/prebuilt_rules/delete_prebuilt_rules_fleet_package.ts

kibanamachine · 2024-12-28T15:15:14Z

New failure: kibana-on-merge - main

xcrzx · 2024-12-30T11:38:21Z

Checked the logs of the most recent test failure.

The socket hung up error occurs during a call to the bootstrap endpoint.
From the Kibana logs, it appears that the security_detection_engine package installation doesn't start. The failure happens earlier during the endpoint package installation. This suggests the failure could be due to the endpoint installation never completing or occurring before Kibana logs the start of the security_detection_engine package installation. Could it be a hung response from the registry?
There are also status: 503 Failed to execute phase [query] errors logged by Elasticsearch, along with log entries such as "Cluster health status changed from [YELLOW] to [GREEN]". This points to the possibility of an unresponsive Elasticsearch node.

Still unsure about the root cause of the failure, but regardless it might be worth trying to use retries when calling bootstrapPrebuiltRules, similar to how it’s done in deleteFleetPackage.

@ess

…point (elastic#205106) **Fixes: elastic#197108 **Fixes: elastic#202037 **Fixes: elastic#203632 ## Summary Builds on top of elastic#203799 and: - simplifies the package deletion code (1 [API call](https://www.elastic.co/docs/api/doc/kibana/operation/operation-delete-fleet-epm-packages-pkgname-pkgversion) instead of 2, per package) - adds retry logic - refactors the testing utils a bit Flaky test runs: - [ESS](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7632) - 190 runs - [Serverless](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7633) - 100 runs ## Details Here's how the test logs look like: ``` info Starting tests └-: Rules Management - Prebuilt Rules - Prebuilt Rules Management └-> "before all" hook: beforeTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" └-: @ess @serverless @skipInServerlessMKI Bootstrap Prebuilt Rules └-> "before all" hook: beforeTestSuite.trigger for "should install fleet packages required for detection engine to function" └-> should install fleet packages required for detection engine to function └-> "before each" hook: global before each for "should install fleet packages required for detection engine to function" └-> "before each" hook for "should install fleet packages required for detection engine to function" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'security_detection_engine is not installed' │ } │ debg Deleting endpoint package │ debg Deleted endpoint package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'endpoint is not installed' │ } └- ✓ pass (21.6s) └-> should skip installing fleet packages if they are already installed └-> "before each" hook: global before each for "should skip installing fleet packages if they are already installed" └-> "before each" hook for "should skip installing fleet packages if they are already installed" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was installed) │ debg Deleting endpoint package │ debg Deleted endpoint package (was installed) └- ✓ pass (22.4s) └-> "after all" hook: afterTestSuite.trigger for "should skip installing fleet packages if they are already installed" └-> "after all" hook: afterTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" 2 passing (1.0m) ``` ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed

@ess

…point (elastic#205106) **Fixes: elastic#197108 **Fixes: elastic#202037 **Fixes: elastic#203632 ## Summary Builds on top of elastic#203799 and: - simplifies the package deletion code (1 [API call](https://www.elastic.co/docs/api/doc/kibana/operation/operation-delete-fleet-epm-packages-pkgname-pkgversion) instead of 2, per package) - adds retry logic - refactors the testing utils a bit Flaky test runs: - [ESS](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7632) - 190 runs - [Serverless](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7633) - 100 runs ## Details Here's how the test logs look like: ``` info Starting tests └-: Rules Management - Prebuilt Rules - Prebuilt Rules Management └-> "before all" hook: beforeTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" └-: @ess @serverless @skipInServerlessMKI Bootstrap Prebuilt Rules └-> "before all" hook: beforeTestSuite.trigger for "should install fleet packages required for detection engine to function" └-> should install fleet packages required for detection engine to function └-> "before each" hook: global before each for "should install fleet packages required for detection engine to function" └-> "before each" hook for "should install fleet packages required for detection engine to function" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'security_detection_engine is not installed' │ } │ debg Deleting endpoint package │ debg Deleted endpoint package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'endpoint is not installed' │ } └- ✓ pass (21.6s) └-> should skip installing fleet packages if they are already installed └-> "before each" hook: global before each for "should skip installing fleet packages if they are already installed" └-> "before each" hook for "should skip installing fleet packages if they are already installed" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was installed) │ debg Deleting endpoint package │ debg Deleted endpoint package (was installed) └- ✓ pass (22.4s) └-> "after all" hook: afterTestSuite.trigger for "should skip installing fleet packages if they are already installed" └-> "after all" hook: afterTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" 2 passing (1.0m) ``` ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed

kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Oct 21, 2024

botelastic bot added the needs-team Issues missing a team label label Oct 21, 2024

kibanamachine added the Team:Detection Rule Management Security Detection Rule Management Team label Oct 21, 2024

botelastic bot removed the needs-team Issues missing a team label label Oct 21, 2024

banderror added Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules area 8.17 candidate labels Oct 22, 2024

banderror assigned xcrzx Oct 22, 2024

banderror closed this as not planned Won't fix, can't repro, duplicate, stale Nov 15, 2024

kibanamachine reopened this Dec 10, 2024

kibanamachine added blocker skipped-test v8.17.0 labels Dec 11, 2024

kibanamachine added a commit that referenced this issue Dec 11, 2024

skip failing test suite (#197108)

c84defd

banderror added 8.18 candidate and removed 8.17 candidate labels Dec 20, 2024

banderror self-assigned this Dec 23, 2024

banderror mentioned this issue Dec 23, 2024

[Security Solution] Unskip tests for the prebuilt rules bootstrap endpoint #205106

Merged

2 tasks

banderror closed this as completed in #205106 Dec 24, 2024

banderror closed this as completed in 71144ed Dec 24, 2024

kibanamachine removed blocker labels Dec 24, 2024

kibanamachine removed skipped-test v8.17.0 labels Dec 24, 2024

kibanamachine reopened this Dec 28, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

kibanamachine commented Oct 21, 2024 •

edited

Loading

elasticmachine commented Oct 21, 2024

elasticmachine commented Oct 22, 2024

elasticmachine commented Oct 22, 2024

banderror commented Nov 15, 2024 •

edited

Loading

kibanamachine commented Dec 10, 2024

kibanamachine commented Dec 10, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 28, 2024

xcrzx commented Dec 30, 2024

Comments

kibanamachine commented Oct 21, 2024 • edited Loading

elasticmachine commented Oct 21, 2024

elasticmachine commented Oct 22, 2024

elasticmachine commented Oct 22, 2024

banderror commented Nov 15, 2024 • edited Loading

kibanamachine commented Dec 10, 2024

kibanamachine commented Dec 10, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 28, 2024

xcrzx commented Dec 30, 2024

kibanamachine commented Oct 21, 2024 •

edited

Loading

banderror commented Nov 15, 2024 •

edited

Loading