Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Cloud Security] Unskipped and fixed flaky agentless test #208844

Merged
merged 1 commit into from
Jan 30, 2025
Merged

[Cloud Security] Unskipped and fixed flaky agentless test #208844

merged 1 commit into from
Jan 30, 2025

Conversation

seanrathier
Copy link
Contributor

@seanrathier seanrathier commented Jan 29, 2025 •
edited by kibanamachine
Loading

Summary

Unskipped and fixed flaky agentless test

-https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7817

Checklist

@seanrathier seanrathier changed the title fixed agentless tests [Cloud Security] Unskipped and fixed flaky agentless test Jan 30, 2025
@seanrathier seanrathier added backport:prev-minor Backport to (9.0) the previous minor version (i.e. one version back from main) Team:Cloud Security Cloud Security team related release_note:skip Skip the PR/issue when compiling release notes labels Jan 30, 2025
@seanrathier seanrathier self-assigned this Jan 30, 2025
@elastic elastic deleted a comment from elasticmachine Jan 30, 2025
@seanrathier seanrathier marked this pull request as ready for review January 30, 2025 14:04
@seanrathier seanrathier requested a review from a team as a code owner January 30, 2025 14:04
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security)

@seanrathier seanrathier enabled auto-merge (squash) January 30, 2025 14:05
@kibanamachine
Copy link
Contributor

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#7817

[✅] x-pack/test/cloud_security_posture_functional/config.agentless.ts: 25/25 tests passed.

see run history

@seanrathier seanrathier merged commit df4a6d7 into elastic:main Jan 30, 2025
9 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/13063298956

opauloh
opauloh requested changes Jan 30, 2025
View reviewed changes
x-pack/test/cloud_security_posture_functional/agentless/create_agent.ts
@@ -128,6 +129,8 @@ export default function ({ getPageObjects, getService }: FtrProviderContext) {
await cisIntegration.clickOptionButton(AWS_SINGLE_ACCOUNT_TEST_ID);

await cisIntegration.inputIntegrationName(integrationPolicyName);
await pageObjects.header.waitUntilLoadingHasFinished();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think checking for "Loading" on the header component is desirable for those scenarios. Relying on the behaviour of the header component seems to make us very prone to flakiness.

For a more flaky-proof solution, we should consider wrapping async side effects with retry.try instead of using pageObjects.header.waitUntilLoadingHasFinished(),

more reference on: https://github.com/elastic/security-team/issues/10975

@elasticmachine
Copy link
Contributor

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #5 / machine learning - anomaly detection anomaly explorer with influencers with farequote based multi metric job Anomaly Charts as embeddable attaches an embeddable to a case

Metrics [docs]

✅ unchanged

History

cc @seanrathier

@kibanamachine
Copy link
Contributor

💔 All backports failed

Status Branch Result
❌ 8.x Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 208844

Questions ?

Please refer to the Backport tool documentation

@kibanamachine kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Jan 31, 2025
@kibanamachine
Copy link
Contributor

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 208844 locally

@seanrathier seanrathier mentioned this pull request Feb 3, 2025
Merged
seanrathier added a commit to seanrathier/kibana that referenced this pull request Feb 3, 2025
@seanrathier
[Cloud Security] Unskipped and fixed flaky agentless test (elastic#20…
5337d6c 
…8844)

(cherry picked from commit df4a6d7)
@seanrathier seanrathier mentioned this pull request Feb 3, 2025
Merged
seanrathier added a commit to seanrathier/kibana that referenced this pull request Feb 3, 2025
@seanrathier
[Cloud Security] Unskipped and fixed flaky agentless test (elastic#20…
0dc0538 
…8844)

(cherry picked from commit df4a6d7)

# Conflicts:
#	x-pack/test/cloud_security_posture_functional/agentless/create_agent.ts
#	x-pack/test/cloud_security_posture_functional/agentless/security_posture.ts
seanrathier added a commit to seanrathier/kibana that referenced this pull request Feb 3, 2025
@seanrathier
[Cloud Security] Unskipped and fixed flaky agentless test (elastic#20…
d20e4a8 
…8844)

(cherry picked from commit df4a6d7)

# Conflicts:
#	x-pack/test/cloud_security_posture_functional/agentless/create_agent.ts
#	x-pack/test/cloud_security_posture_functional/agentless/security_posture.ts
@seanrathier seanrathier mentioned this pull request Feb 3, 2025
Merged
@seanrathier
Copy link
Contributor Author

💚 All backports created successfully

Status Branch Result
✅ 9.0
✅ 8.x
✅ 8.18

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

seanrathier added a commit that referenced this pull request Feb 3, 2025
@seanrathier
[8.18] [Cloud Security] Unskipped and fixed flaky agentless test (#20…
1cae919 
…8844) (#209385)

# Backport

This will backport the following commits from `main` to `8.18`:
- [[Cloud Security] Unskipped and fixed flaky agentless test
(#208844)](#208844)

<!--- Backport version: 9.6.4 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT
[{"author":{"name":"seanrathier","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-01-30T23:36:59Z","message":"[Cloud
Security] Unskipped and fixed flaky agentless test
(#208844)","sha":"df4a6d7ce37bfaaf612190d8e68fc8769d69b235","branchLabelMapping":{"^v9.0.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport
missing","v9.0.0","Team:Cloud
Security","backport:prev-minor"],"title":"[Cloud Security] Unskipped and
fixed flaky agentless
test","number":208844,"url":"https://github.com/elastic/kibana/pull/208844","mergeCommit":{"message":"[Cloud
Security] Unskipped and fixed flaky agentless test
(#208844)","sha":"df4a6d7ce37bfaaf612190d8e68fc8769d69b235"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/208844","number":208844,"mergeCommit":{"message":"[Cloud
Security] Unskipped and fixed flaky agentless test
(#208844)","sha":"df4a6d7ce37bfaaf612190d8e68fc8769d69b235"}}]}]
BACKPORT-->
seanrathier added a commit that referenced this pull request Feb 3, 2025
@seanrathier
[8.x] [Cloud Security] Unskipped and fixed flaky agentless test (#208844
dac6251 
) (#209384)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Cloud Security] Unskipped and fixed flaky agentless test
(#208844)](#208844)

<!--- Backport version: 9.6.4 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT
[{"author":{"name":"seanrathier","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-01-30T23:36:59Z","message":"[Cloud
Security] Unskipped and fixed flaky agentless test
(#208844)","sha":"df4a6d7ce37bfaaf612190d8e68fc8769d69b235","branchLabelMapping":{"^v9.0.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport
missing","v9.0.0","Team:Cloud
Security","backport:prev-minor"],"title":"[Cloud Security] Unskipped and
fixed flaky agentless
test","number":208844,"url":"https://github.com/elastic/kibana/pull/208844","mergeCommit":{"message":"[Cloud
Security] Unskipped and fixed flaky agentless test
(#208844)","sha":"df4a6d7ce37bfaaf612190d8e68fc8769d69b235"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/208844","number":208844,"mergeCommit":{"message":"[Cloud
Security] Unskipped and fixed flaky agentless test
(#208844)","sha":"df4a6d7ce37bfaaf612190d8e68fc8769d69b235"}}]}]
BACKPORT-->
seanrathier added a commit that referenced this pull request Feb 3, 2025
@seanrathier
[9.0] [Cloud Security] Unskipped and fixed flaky agentless test (#208844
3d4f8b0 
) (#209383)

# Backport

This will backport the following commits from `main` to `9.0`:
- [[Cloud Security] Unskipped and fixed flaky agentless test
(#208844)](#208844)

<!--- Backport version: 9.6.4 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT
[{"author":{"name":"seanrathier","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-01-30T23:36:59Z","message":"[Cloud
Security] Unskipped and fixed flaky agentless test
(#208844)","sha":"df4a6d7ce37bfaaf612190d8e68fc8769d69b235","branchLabelMapping":{"^v9.0.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport
missing","v9.0.0","Team:Cloud
Security","backport:prev-minor"],"title":"[Cloud Security] Unskipped and
fixed flaky agentless
test","number":208844,"url":"https://github.com/elastic/kibana/pull/208844","mergeCommit":{"message":"[Cloud
Security] Unskipped and fixed flaky agentless test
(#208844)","sha":"df4a6d7ce37bfaaf612190d8e68fc8769d69b235"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/208844","number":208844,"mergeCommit":{"message":"[Cloud
Security] Unskipped and fixed flaky agentless test
(#208844)","sha":"df4a6d7ce37bfaaf612190d8e68fc8769d69b235"}}]}]
BACKPORT-->
@kibanamachine kibanamachine added v8.19.0 and removed backport missing Added to PRs automatically when the are determined to be missing a backport. labels Feb 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@opauloh opauloh opauloh requested changes

@Omolola-Akinleye Omolola-Akinleye Omolola-Akinleye approved these changes

Assignees

@seanrathier seanrathier

Labels
backport:prev-minor Backport to (9.0) the previous minor version (i.e. one version back from main) release_note:skip Skip the PR/issue when compiling release notes Team:Cloud Security Cloud Security team related v8.18.0 v8.19.0 v9.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@seanrathier @elasticmachine @kibanamachine @Omolola-Akinleye @opauloh