[8.12] [Security Solution] Unskip flaky tests in Prebuilt Rules FTR Integration tests (#173998) #174761
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ion tests (elastic#173998) **Addresses:** elastic#172107 elastic#171380 ## Summary Unskip skipped tests in: 1. `x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/prebuilt_rules/bundled_prebuilt_rules_package/install_latest_bundled_prebuilt_rules.ts` 2. `x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/prebuilt_rules/management/fleet_integration.ts` - Makes the `retryDelay` in the **RetryService** in `packages/kbn-ftr-common-functional-services/services/retry/retry.ts` a configurable parameter - used in our `retry` util to shorten the wait period to 200ms. - Creates `retry` wrapper util for our FTR Integration tests, that wraps `retry.try` from the **RetryService**, to implement maximum attempts. - Uses this `retry` wrapper in test utils that install the `security_detection_engine` Fleet package, and asserts that rules have been succesfully installed - and retries if they haven't. - Creates `refreshSavedObjectIndices` reusable util that cleans cache and refreshes indices. Centralizes comment spread around file into this single file. - Use this new util to clean the cache in util that install the Fleet package, and utils that read the prebuilt rule status. ## Flaky test runner **Before changes:** - For both `bundled_prebuilt_rules_package` and `management`: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4688 🟢 (250 and 250 runs) - `bundled_prebuilt_rules_package`: - ESS: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4805 (500 runs) - Serverless: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4807 (500 runs) - `management` - ESS: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4806 (500 runs) - Serverless: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4808 (500 runs) **After changes:** - `bundled_prebuilt_rules_package`: - ESS: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4825 🟢 (500 runs) - Serverless: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4826 🟢 (500 runs) - `management` - ESS: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4827 🟢 (500 runs) - Serverless: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4828 🟢 (500 runs) ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit 81d6478) # Conflicts: # x-pack/test/security_solution_api_integration/tsconfig.json
1 task
nikitaindik
approved these changes
Jan 12, 2024
💛 Build succeeded, but was flaky
Failed CI StepsMetrics [docs]Public APIs missing comments
To update your PR or re-run it, just comment with: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport
This will backport the following commits from
mainto8.12:Questions ?
Please refer to the Backport tool documentation