Failing test: Rules Management - Prebuilt Rules Management Integration Tests - ESS Env - Trial License.x-pack/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/management/trial_license_complete_tier/bootstrap_prebuilt_rules·ts - Rules Management - Prebuilt Rules - Prebuilt Rules Management @ess @serverless @skipInServerlessMKI Bootstrap Prebuilt Rules should install fleet packages required for detection engine to function #203632

kibanamachine · 2024-12-10T16:07:08Z

A test failed on a tracked branch

Error: socket hang up
    at Socket.socketOnEnd (node:_http_client:524:23)
    at Socket.emit (node:events:531:35)
    at endReadableNT (node:internal/streams/readable:1696:12)
    at processTicksAndRejections (node:internal/process/task_queues:82:21) {
  code: 'ECONNRESET',
  response: undefined
}

First failure: kibana-on-merge - 8.x

The text was updated successfully, but these errors were encountered:

elasticmachine · 2024-12-10T16:07:29Z

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

kibanamachine · 2024-12-11T19:51:32Z

New failure: kibana-on-merge - main

kibanamachine · 2024-12-11T19:57:52Z

New failure: kibana-on-merge - main

kibanamachine · 2024-12-11T19:59:36Z

New failure: kibana-on-merge - main

kibanamachine · 2024-12-11T20:04:13Z

New failure: kibana-on-merge - main

kibanamachine · 2024-12-11T20:05:32Z

New failure: kibana-on-merge - main

kibanamachine · 2024-12-11T20:06:12Z

New failure: kibana-on-merge - main

kibanamachine · 2024-12-11T20:13:02Z

New failure: kibana-on-merge - main

kibanamachine · 2024-12-11T20:13:44Z

New failure: kibana-on-merge - main

kibanamachine · 2024-12-11T20:15:50Z

New failure: kibana-on-merge - 8.x

kibanamachine · 2024-12-11T20:20:51Z

New failure: kibana-on-merge - main

kibanamachine · 2024-12-11T20:23:48Z

New failure: kibana-on-merge - 8.x

kibanamachine · 2024-12-11T21:01:22Z

New failure: kibana-on-merge - main

kibanamachine · 2024-12-11T21:27:25Z

New failure: kibana-on-merge - main

kibanamachine · 2024-12-11T21:35:39Z

New failure: kibana-on-merge - main

kibanamachine · 2024-12-11T22:02:31Z

New failure: kibana-on-merge - main

kibanamachine · 2024-12-11T22:09:10Z

New failure: kibana-on-merge - main

mistic · 2024-12-11T22:09:48Z

Skipped.

main: 854b382
8.x: 43d5c42

@ess

…point (elastic#205106) **Fixes: elastic#197108 **Fixes: elastic#202037 **Fixes: elastic#203632 ## Summary Builds on top of elastic#203799 and: - simplifies the package deletion code (1 [API call](https://www.elastic.co/docs/api/doc/kibana/operation/operation-delete-fleet-epm-packages-pkgname-pkgversion) instead of 2, per package) - adds retry logic - refactors the testing utils a bit Flaky test runs: - [ESS](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7632) - 190 runs - [Serverless](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7633) - 100 runs ## Details Here's how the test logs look like: ``` info Starting tests └-: Rules Management - Prebuilt Rules - Prebuilt Rules Management └-> "before all" hook: beforeTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" └-: @ess @serverless @skipInServerlessMKI Bootstrap Prebuilt Rules └-> "before all" hook: beforeTestSuite.trigger for "should install fleet packages required for detection engine to function" └-> should install fleet packages required for detection engine to function └-> "before each" hook: global before each for "should install fleet packages required for detection engine to function" └-> "before each" hook for "should install fleet packages required for detection engine to function" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'security_detection_engine is not installed' │ } │ debg Deleting endpoint package │ debg Deleted endpoint package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'endpoint is not installed' │ } └- ✓ pass (21.6s) └-> should skip installing fleet packages if they are already installed └-> "before each" hook: global before each for "should skip installing fleet packages if they are already installed" └-> "before each" hook for "should skip installing fleet packages if they are already installed" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was installed) │ debg Deleting endpoint package │ debg Deleted endpoint package (was installed) └- ✓ pass (22.4s) └-> "after all" hook: afterTestSuite.trigger for "should skip installing fleet packages if they are already installed" └-> "after all" hook: afterTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" 2 passing (1.0m) ``` ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed (cherry picked from commit 71144ed) # Conflicts: # x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rules/prebuilt_rules/delete_endpoint_fleet_package.ts # x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rules/prebuilt_rules/delete_prebuilt_rules_fleet_package.ts

@ess

…point (elastic#205106) **Fixes: elastic#197108 **Fixes: elastic#202037 **Fixes: elastic#203632 ## Summary Builds on top of elastic#203799 and: - simplifies the package deletion code (1 [API call](https://www.elastic.co/docs/api/doc/kibana/operation/operation-delete-fleet-epm-packages-pkgname-pkgversion) instead of 2, per package) - adds retry logic - refactors the testing utils a bit Flaky test runs: - [ESS](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7632) - 190 runs - [Serverless](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7633) - 100 runs ## Details Here's how the test logs look like: ``` info Starting tests └-: Rules Management - Prebuilt Rules - Prebuilt Rules Management └-> "before all" hook: beforeTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" └-: @ess @serverless @skipInServerlessMKI Bootstrap Prebuilt Rules └-> "before all" hook: beforeTestSuite.trigger for "should install fleet packages required for detection engine to function" └-> should install fleet packages required for detection engine to function └-> "before each" hook: global before each for "should install fleet packages required for detection engine to function" └-> "before each" hook for "should install fleet packages required for detection engine to function" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'security_detection_engine is not installed' │ } │ debg Deleting endpoint package │ debg Deleted endpoint package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'endpoint is not installed' │ } └- ✓ pass (21.6s) └-> should skip installing fleet packages if they are already installed └-> "before each" hook: global before each for "should skip installing fleet packages if they are already installed" └-> "before each" hook for "should skip installing fleet packages if they are already installed" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was installed) │ debg Deleting endpoint package │ debg Deleted endpoint package (was installed) └- ✓ pass (22.4s) └-> "after all" hook: afterTestSuite.trigger for "should skip installing fleet packages if they are already installed" └-> "after all" hook: afterTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" 2 passing (1.0m) ``` ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed (cherry picked from commit 71144ed) # Conflicts: # x-pack/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/management/trial_license_complete_tier/bootstrap_prebuilt_rules.ts # x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rules/prebuilt_rules/delete_endpoint_fleet_package.ts # x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rules/prebuilt_rules/delete_prebuilt_rules_fleet_package.ts

@ess

…point (elastic#205106) **Fixes: elastic#197108 **Fixes: elastic#202037 **Fixes: elastic#203632 ## Summary Builds on top of elastic#203799 and: - simplifies the package deletion code (1 [API call](https://www.elastic.co/docs/api/doc/kibana/operation/operation-delete-fleet-epm-packages-pkgname-pkgversion) instead of 2, per package) - adds retry logic - refactors the testing utils a bit Flaky test runs: - [ESS](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7632) - 190 runs - [Serverless](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7633) - 100 runs ## Details Here's how the test logs look like: ``` info Starting tests └-: Rules Management - Prebuilt Rules - Prebuilt Rules Management └-> "before all" hook: beforeTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" └-: @ess @serverless @skipInServerlessMKI Bootstrap Prebuilt Rules └-> "before all" hook: beforeTestSuite.trigger for "should install fleet packages required for detection engine to function" └-> should install fleet packages required for detection engine to function └-> "before each" hook: global before each for "should install fleet packages required for detection engine to function" └-> "before each" hook for "should install fleet packages required for detection engine to function" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'security_detection_engine is not installed' │ } │ debg Deleting endpoint package │ debg Deleted endpoint package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'endpoint is not installed' │ } └- ✓ pass (21.6s) └-> should skip installing fleet packages if they are already installed └-> "before each" hook: global before each for "should skip installing fleet packages if they are already installed" └-> "before each" hook for "should skip installing fleet packages if they are already installed" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was installed) │ debg Deleting endpoint package │ debg Deleted endpoint package (was installed) └- ✓ pass (22.4s) └-> "after all" hook: afterTestSuite.trigger for "should skip installing fleet packages if they are already installed" └-> "after all" hook: afterTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" 2 passing (1.0m) ``` ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed (cherry picked from commit 71144ed) # Conflicts: # x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rules/prebuilt_rules/delete_endpoint_fleet_package.ts # x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rules/prebuilt_rules/delete_prebuilt_rules_fleet_package.ts

kibanamachine · 2024-12-27T14:49:28Z

New failure: kibana-on-merge - 8.x

@ess

…point (elastic#205106) **Fixes: elastic#197108 **Fixes: elastic#202037 **Fixes: elastic#203632 ## Summary Builds on top of elastic#203799 and: - simplifies the package deletion code (1 [API call](https://www.elastic.co/docs/api/doc/kibana/operation/operation-delete-fleet-epm-packages-pkgname-pkgversion) instead of 2, per package) - adds retry logic - refactors the testing utils a bit Flaky test runs: - [ESS](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7632) - 190 runs - [Serverless](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7633) - 100 runs ## Details Here's how the test logs look like: ``` info Starting tests └-: Rules Management - Prebuilt Rules - Prebuilt Rules Management └-> "before all" hook: beforeTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" └-: @ess @serverless @skipInServerlessMKI Bootstrap Prebuilt Rules └-> "before all" hook: beforeTestSuite.trigger for "should install fleet packages required for detection engine to function" └-> should install fleet packages required for detection engine to function └-> "before each" hook: global before each for "should install fleet packages required for detection engine to function" └-> "before each" hook for "should install fleet packages required for detection engine to function" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'security_detection_engine is not installed' │ } │ debg Deleting endpoint package │ debg Deleted endpoint package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'endpoint is not installed' │ } └- ✓ pass (21.6s) └-> should skip installing fleet packages if they are already installed └-> "before each" hook: global before each for "should skip installing fleet packages if they are already installed" └-> "before each" hook for "should skip installing fleet packages if they are already installed" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was installed) │ debg Deleting endpoint package │ debg Deleted endpoint package (was installed) └- ✓ pass (22.4s) └-> "after all" hook: afterTestSuite.trigger for "should skip installing fleet packages if they are already installed" └-> "after all" hook: afterTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" 2 passing (1.0m) ``` ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed

banderror · 2025-01-02T10:53:53Z

The build mentioned in #203632 (comment) was green and the test succeeded on the 2nd attempt. I will close the ticket for now, but let's if it happens again. If it will, we'd need to revisit the timeout values and adjust logging so the logs we write from tests become visible on CI.

@ess

…point (elastic#205106) **Fixes: elastic#197108 **Fixes: elastic#202037 **Fixes: elastic#203632 ## Summary Builds on top of elastic#203799 and: - simplifies the package deletion code (1 [API call](https://www.elastic.co/docs/api/doc/kibana/operation/operation-delete-fleet-epm-packages-pkgname-pkgversion) instead of 2, per package) - adds retry logic - refactors the testing utils a bit Flaky test runs: - [ESS](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7632) - 190 runs - [Serverless](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7633) - 100 runs ## Details Here's how the test logs look like: ``` info Starting tests └-: Rules Management - Prebuilt Rules - Prebuilt Rules Management └-> "before all" hook: beforeTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" └-: @ess @serverless @skipInServerlessMKI Bootstrap Prebuilt Rules └-> "before all" hook: beforeTestSuite.trigger for "should install fleet packages required for detection engine to function" └-> should install fleet packages required for detection engine to function └-> "before each" hook: global before each for "should install fleet packages required for detection engine to function" └-> "before each" hook for "should install fleet packages required for detection engine to function" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'security_detection_engine is not installed' │ } │ debg Deleting endpoint package │ debg Deleted endpoint package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'endpoint is not installed' │ } └- ✓ pass (21.6s) └-> should skip installing fleet packages if they are already installed └-> "before each" hook: global before each for "should skip installing fleet packages if they are already installed" └-> "before each" hook for "should skip installing fleet packages if they are already installed" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was installed) │ debg Deleting endpoint package │ debg Deleted endpoint package (was installed) └- ✓ pass (22.4s) └-> "after all" hook: afterTestSuite.trigger for "should skip installing fleet packages if they are already installed" └-> "after all" hook: afterTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" 2 passing (1.0m) ``` ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed

kibanamachine · 2025-01-09T17:09:45Z

New failure: kibana-on-merge - main

maximpn · 2025-01-09T18:05:37Z

Failure mentioned in #203632 (comment) due to socket hang up error. The same as mentioned in #203632 (comment).

It looks like an infra issue happening once per weeks. I'll keep the ticket open to see if the same issue repeats.

@ess

…point (elastic#205106) **Fixes: elastic#197108 **Fixes: elastic#202037 **Fixes: elastic#203632 ## Summary Builds on top of elastic#203799 and: - simplifies the package deletion code (1 [API call](https://www.elastic.co/docs/api/doc/kibana/operation/operation-delete-fleet-epm-packages-pkgname-pkgversion) instead of 2, per package) - adds retry logic - refactors the testing utils a bit Flaky test runs: - [ESS](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7632) - 190 runs - [Serverless](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7633) - 100 runs ## Details Here's how the test logs look like: ``` info Starting tests └-: Rules Management - Prebuilt Rules - Prebuilt Rules Management └-> "before all" hook: beforeTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" └-: @ess @serverless @skipInServerlessMKI Bootstrap Prebuilt Rules └-> "before all" hook: beforeTestSuite.trigger for "should install fleet packages required for detection engine to function" └-> should install fleet packages required for detection engine to function └-> "before each" hook: global before each for "should install fleet packages required for detection engine to function" └-> "before each" hook for "should install fleet packages required for detection engine to function" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'security_detection_engine is not installed' │ } │ debg Deleting endpoint package │ debg Deleted endpoint package (was not installed) { │ statusCode: 400, │ error: 'Bad Request', │ message: 'endpoint is not installed' │ } └- ✓ pass (21.6s) └-> should skip installing fleet packages if they are already installed └-> "before each" hook: global before each for "should skip installing fleet packages if they are already installed" └-> "before each" hook for "should skip installing fleet packages if they are already installed" │ debg Deleting security_detection_engine package │ debg Deleted security_detection_engine package (was installed) │ debg Deleting endpoint package │ debg Deleted endpoint package (was installed) └- ✓ pass (22.4s) └-> "after all" hook: afterTestSuite.trigger for "should skip installing fleet packages if they are already installed" └-> "after all" hook: afterTestSuite.trigger in "Rules Management - Prebuilt Rules - Prebuilt Rules Management" 2 passing (1.0m) ``` ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed

kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Dec 10, 2024

botelastic bot added the needs-team Issues missing a team label label Dec 10, 2024

kibanamachine added the Team:Detection Rule Management Security Detection Rule Management Team label Dec 10, 2024

botelastic bot removed the needs-team Issues missing a team label label Dec 10, 2024

mistic added a commit that referenced this issue Dec 11, 2024

skip flaky suite (#203632)

854b382

mistic added a commit that referenced this issue Dec 11, 2024

skip flaky suite (#203632)

43d5c42

mistic added blocker skipped-test v9.0.0 v8.18.0 labels Dec 11, 2024

CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Dec 12, 2024

skip flaky suite (elastic#203632)

07fc3b5

banderror added the Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules area label Dec 20, 2024

banderror added the 8.18 candidate label Dec 20, 2024

banderror assigned xcrzx and banderror Dec 20, 2024

banderror mentioned this issue Dec 23, 2024

[Security Solution] Unskip tests for the prebuilt rules bootstrap endpoint #205106

Merged

2 tasks

banderror closed this as completed in #205106 Dec 24, 2024

banderror closed this as completed in 71144ed Dec 24, 2024

kibanamachine removed v8.18.0 skipped-test blocker v9.0.0 labels Dec 24, 2024

kibanamachine reopened this Dec 27, 2024

banderror closed this as completed Jan 2, 2025

kibanamachine reopened this Jan 9, 2025

CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Jan 13, 2025

skip flaky suite (elastic#203632)

bc794bb

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

kibanamachine commented Dec 10, 2024 •

edited

Loading

elasticmachine commented Dec 10, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

mistic commented Dec 11, 2024

kibanamachine commented Dec 27, 2024

banderror commented Jan 2, 2025

kibanamachine commented Jan 9, 2025

maximpn commented Jan 9, 2025

Comments

kibanamachine commented Dec 10, 2024 • edited Loading

elasticmachine commented Dec 10, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

kibanamachine commented Dec 11, 2024

mistic commented Dec 11, 2024

kibanamachine commented Dec 27, 2024

banderror commented Jan 2, 2025

kibanamachine commented Jan 9, 2025

maximpn commented Jan 9, 2025

kibanamachine commented Dec 10, 2024 •

edited

Loading