Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: move remaining Cypress tests behind GH label #174523

Merged
merged 8 commits into from
Jan 17, 2024
Merged

ci: move remaining Cypress tests behind GH label #174523

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
a2f2f7e
ci: move remaining Cypress tests behind GH label
Jan 9, 2024
4ffc74f
Merge branch 'main' into move-cypress
Jan 11, 2024
d19744c
Merge branch 'main' into move-cypress
yngrdyn Jan 11, 2024
816ec8b
Merge branch 'main' into move-cypress
yngrdyn Jan 12, 2024
3a64668
Address review comments
Jan 15, 2024
db1f4e8
Merge branch 'main' into move-cypress
Jan 15, 2024
62cc241
Rename file
Jan 15, 2024
d0ddbbd
Address review comments
Jan 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 0 additions & 48 deletions .buildkite/pipelines/pull_request/base.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,54 +57,6 @@ steps:
- exit_status: '*'
limit: 1

- command: .buildkite/scripts/steps/functional/threat_intelligence.sh
label: 'Threat Intelligence Cypress Tests'
agents:
queue: n2-4-spot
depends_on: build
timeout_in_minutes: 60
parallelism: 2
retry:
automatic:
- exit_status: '*'
limit: 1

- command: .buildkite/scripts/steps/functional/osquery_cypress.sh
label: 'Osquery Cypress Tests'
agents:
queue: n2-4-spot
depends_on: build
timeout_in_minutes: 60
parallelism: 6
retry:
automatic:
- exit_status: '*'
limit: 1

- command: .buildkite/scripts/steps/functional/security_serverless_osquery.sh
label: 'Serverless Osquery Cypress Tests'
agents:
queue: n2-4-spot
depends_on: build
timeout_in_minutes: 60
parallelism: 6
retry:
automatic:
- exit_status: '*'
limit: 1

# status_exception: Native role management is not enabled in this Elasticsearch instance
# - command: .buildkite/scripts/steps/functional/security_serverless_defend_workflows.sh
# label: 'Serverless Security Defend Workflows Cypress Tests'
# agents:
# queue: n2-4-spot
# depends_on: build
# timeout_in_minutes: 60
# retry:
# automatic:
# - exit_status: '*'
# limit: 1

- command: .buildkite/scripts/steps/lint.sh
label: 'Linting'
agents:
Expand Down
12 changes: 12 additions & 0 deletions .buildkite/pipelines/pull_request/security_solution/defend_workflows.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,15 @@ steps:
automatic:
- exit_status: '*'
limit: 1

# status_exception: Native role management is not enabled in this Elasticsearch instance
# - command: .buildkite/scripts/steps/functional/security_serverless_defend_workflows.sh
# label: 'Serverless Security Defend Workflows Cypress Tests'
# agents:
# queue: n2-4-spot
# depends_on: build
# timeout_in_minutes: 60
# retry:
# automatic:
# - exit_status: '*'
# limit: 1
24 changes: 24 additions & 0 deletions .buildkite/pipelines/pull_request/security_solution/osquery.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
steps:
- command: .buildkite/scripts/steps/functional/osquery_cypress.sh
label: 'Osquery Cypress Tests'
agents:
queue: n2-4-spot
depends_on: build
timeout_in_minutes: 60
parallelism: 6
retry:
automatic:
- exit_status: '*'
limit: 1

- command: .buildkite/scripts/steps/functional/security_serverless_osquery.sh
label: 'Serverless Osquery Cypress Tests'
agents:
queue: n2-4-spot
depends_on: build
timeout_in_minutes: 60
parallelism: 6
retry:
automatic:
- exit_status: '*'
limit: 1
12 changes: 12 additions & 0 deletions .buildkite/pipelines/pull_request/security_solution/threat_intelligence.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
steps:
- command: .buildkite/scripts/steps/functional/threat_intelligence.sh
label: 'Threat Intelligence Cypress Tests'
agents:
queue: n2-4-spot
depends_on: build
timeout_in_minutes: 60
parallelism: 2
retry:
automatic:
- exit_status: '*'
limit: 1
28 changes: 28 additions & 0 deletions .buildkite/scripts/pipelines/pull_request/pipeline.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -265,6 +265,34 @@ const uploadPipeline = (pipelineContent: string | object) => {
);
}

if (
(await doAnyChangesMatch([
/^package.json/,
/^packages\/kbn-securitysolution-.*/,
/^x-pack\/plugins\/threat_intelligence/,
/^x-pack\/packages\/security-solution/,
/^x-pack\/test\/security_solution_cypress/,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/^x-pack\/test\/security_solution_cypress/
should be replaced by
/^x-pack\/test\/threat_intelligence_cypress/

Currently all the Cypress tests haven’t been migrated to the x-pack/test folder are still in the x-pack/plugins/threat_intelligence folder unfortunately.
I had started to migrated things over before the holidays (actually I was trying to merge them within the security_solution_cypress folder directly for simplicity and code reusability) but we had to jump on reducing flakiness instead. Not sure when I’ll have the time to get back to that…

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, looking at the code, here are a few that I think we should add

other x-pack plugins:

  • /x-pack\/plugins\/cases/
  • /x-pack\/plugins\/timelines/
  • /x-pack\/plugins\/triggers_actions_ui/
  • /x-pack\/plugins\/rule_registry/

other packages:

  • /packages\/kbn-es-query/
  • /packages\/kbn-securitysolution-io-ts-list-types/
  • /packages\/kbn-i18n-react/
  • /packages\/kbn-i18n/
  • /packages\/shared-ux/
  • /packages\/kbn-doc-links/
  • /packages\/kbn-securitysolution-io-ts-list-types/

other src plugins:

  • /src\/plugins\/data/
  • /src\/plugins\/kibana_utils/
  • /src\/plugins\/inspector/
  • /src\/plugins\/data_views/
  • /src\/core/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done 👍

])) ||
GITHUB_PR_LABELS.includes('ci:all-cypress-suites')
) {
pipeline.push(
getPipeline('.buildkite/pipelines/pull_request/security_solution/threat_intelligence.yml')
);
}

if (
(await doAnyChangesMatch([
/^package.json/,
/^packages\/kbn-securitysolution-.*/,
/^x-pack\/plugins\/osquery/,
/^x-pack\/packages\/security-solution/,
/^x-pack\/test\/security_solution_cypress/,
])) ||
GITHUB_PR_LABELS.includes('ci:all-cypress-suites')
) {
pipeline.push(getPipeline('.buildkite/pipelines/pull_request/security_solution/osquery.yml'));
}

pipeline.push(getPipeline('.buildkite/pipelines/pull_request/post_build.yml'));

// remove duplicated steps
Expand Down