[Security Solution] Adding conditional execution on ci for Security Solution cypress tests #173815
Conversation
MadameSheema
changed the title
|
/ci |
|
To better signal which yaml files under |
|
/ci |
MadameSheema
added
Team: SecuritySolution
MadameSheema
requested review from
spong,
andrew-goldstein,
stephmilovic,
banderror,
yctercero,
michaelolo24 and
jaredburgettelastic
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
|
Pinging @elastic/security-threat-hunting (Team:Threat Hunting) |
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
There was a problem hiding this comment.
Rule Management related changes LGTM, thank you @MadameSheema 👍
| (await doAnyChangesMatch([ | ||
| /^package.json/, | ||
| /^packages\/kbn-securitysolution-.*/, | ||
| /^x-pack\/plugins\/alerting/, | ||
| /^x-pack\/plugins\/data_views\/common/, | ||
| /^x-pack\/plugins\/lists/, | ||
| /^x-pack\/plugins\/rule_registry\/common/, | ||
| /^x-pack\/plugins\/security_solution/, | ||
| /^x-pack\/plugins\/security_solution_ess/, | ||
| /^x-pack\/plugins\/security_solution_serverless/, | ||
| /^x-pack\/plugins\/task_manager/, | ||
| /^x-pack\/plugins\/timelines/, | ||
| /^x-pack\/plugins\/triggers_actions_ui\/public\/application\/sections\/action_connector_form/, | ||
| /^x-pack\/plugins\/triggers_actions_ui\/public\/application\/context\/actions_connectors_context\.tsx/, | ||
| /^x-pack\/plugins\/usage_collection\/public/, | ||
| /^x-pack\/test\/security_solution_cypress/, | ||
| ])) || | ||
| GITHUB_PR_LABELS.includes('ci:all-cypress-suites') |
| /^x-pack\/plugins\/triggers_actions_ui\/public\/application\/sections\/action_connector_form/, | ||
| /^x-pack\/plugins\/triggers_actions_ui\/public\/application\/context\/actions_connectors_context\.tsx/, | ||
| /^x-pack\/plugins\/usage_collection\/public/, | ||
| /^x-pack\/test\/security_solution_cypress/, |
There was a problem hiding this comment.
Do we also need to specify our FTRs here? Right now they are under security_solution_api_integration with only a few left in detection_engine_api_integration.
There was a problem hiding this comment.
Good question. As far as I know, the top priority right now is Cypress. We can do a follow-up PR to add conditional execution for the FTR.
There was a problem hiding this comment.
@yctercero I don't think we need to specify these folders here because our FTR tests run unconditionally in all PRs, as all other FTR tests and as configured here:
kibana/.buildkite/ftr_configs.yml
Lines 461 to 514 in 1a0dcf6
Co-authored-by: Steph Milovic <[email protected]>
Co-authored-by: Steph Milovic <[email protected]>
There was a problem hiding this comment.
We're open to trying it, however:
Operations is going to be much more strict about the on-merge pipeline than pull requests - we need main to always be releasable. If we see reoccurring issues where untested pull request (shared packages, dependencies) changes break on-merge we'll have to look into another approach.
Can we follow this up with a notification to contributors, and a reminder of how to run all cypress tests for global changes?
💚 Build Succeeded
Metrics [docs]
History
To update your PR or re-run it, just comment with: |
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
kibanamachine
added
the
backport missing
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
## Summary After a conversation with @michaelolo24 we found out that: - We were missing a dependency for the investigations team - Not all the dependencies added in #173815 apply to the investigations team In this PR we are adding a new conditional for the investigations team execution with the aim of having more granularity and more accurate dependencies. --------- Co-authored-by: Jon <[email protected]>
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
kibanamachine
removed
the
backport missing
Move the remaining Security Solution Cypress tests behind the GitHub flag `ci:all-cypress-suites`. This is a follow-up to PR #173815. --------- Co-authored-by: Yngrid Coello <[email protected]>
Move the remaining Security Solution Cypress tests behind the GitHub flag `ci:all-cypress-suites`. This is a follow-up to PR elastic#173815. --------- Co-authored-by: Yngrid Coello <[email protected]>
Summary
With the aim of preventing our tests from blocking PRs on teams introducing changes not related with our basecode and saving buildkite time execution, PRs the security solution Cypress tests are going to be executed just when a change is introduced on the listed dependencies in
.buildkite/scripts/pipelines/pull_request/pipeline.ts.The execution of our tests on every change in the
on-mergepipeline is going to be maintained, in that way, we are going to be aware of any possible issue we may miss and our flaky tests are going to continue being tracked in github