[Ops] Run kibana quality gate suites #165346
Conversation
…e catalog-info is pointing
delanni
changed the title
delanni
force-pushed
the
run-kibana-quality-gate-suites
branch
from
231228e to
695ae9d
Compare
## Summary Kibana's build jobs work on a different subset of executors than the buildkite pipelines defined in `catalog-info.yaml`. The former jobs require a set of `pre_command` / `post_command` steps to prepare the jobs for building/testing kibana. The latter don't have access rights to certain vault secrets (and possibly other missing config from the Kibana world), but for now, they're also not building Kibana, they just trigger other jobs, so we can just skip the problematic hooks. ~~A probably good indicator I found for deciding whether we need the kibana-related `pre_command` is the `BUILDKITE_AGENT_META_DATA_AGENT_MANAGER` flag, that's set to `"kibana"` in the case of the kibana executors.~~ We can try to match on the agent names for the CI-systems agents. They seem to be starting with `bk-agent`. This should allow for the [kibana-tests](https://buildkite.com/elastic/kibana-tests) job to run. Split from: #165346 --------- Co-authored-by: Tiago Costa <[email protected]>
delanni
added
test
chore
Team:Operations
|
Pinging @elastic/kibana-operations (Team:Operations) |
| key: "security" | ||
| steps: | ||
| - label: ":pipeline::female-detective::seedling: Trigger Security Solution quality gate script" | ||
| command: .buildkite/scripts/pipelines/security_solution_quality_gate/pipeline.sh |
There was a problem hiding this comment.
This looks like a placeholder file for now, but we may have to follow the trigger pattern in the future based on the pre-command/post-command skips.
There was a problem hiding this comment.
Yeah, I agree, having external pipelines are probably a lot better for us, especially because that can run in a Kibana-managed agent. Once the team finds time to work on these tests, we could move it to a separate pipeline.
| agents: | ||
| image: "docker.elastic.co/ci-agent-images/basic-buildkite-agent:1688566364" | ||
| - label: ":pipeline::kibana::seedling: Trigger Kibana Serverless Tests for ${ENVIRONMENT}" | ||
| trigger: appex-qa-kibana-serverless-ftr-tests # https://buildkite.com/elastic/appex-qa-kibana-serverless-ftr-tests |
There was a problem hiding this comment.
I really appreciate the URL to the Buildkite pipeline - very user friendly 💯
There was a problem hiding this comment.
By the way, I just noticed that all files and folders in the .buildkite/pipelines directory use underscore as word separators, except for quality-gates and its files 🙂
There was a problem hiding this comment.
it was a guest appearance from a non-kibana team, I guess they brought practise that they had everywhere else.
There was a problem hiding this comment.
Yeah, not a big deal. Dashes vs. underscores is something that has always caught my eye for some unknown reason...
There was a problem hiding this comment.
high pattern recognition, I'm also bothered by it :D I'll change this once serverless releases are smooth
…n't have the downstream pipelines working well, but we might want to push releases through the whole env chain
💔 Build FailedFailed CI StepsTest Failures
Metrics [docs]
History
To update your PR or re-run it, just comment with: |
Summary
Sets up quality gate triggering in Kibana. The tests are mostly triggering external pipelines and relying on their results. Here's an example run: https://buildkite.com/elastic/kibana-tests/builds/28#job-018a69a6-c860-405e-ab2b-bce2aed07df3
According to this doc many of the quality gates are really required for the QA->Staging promotion step.
Most of the tests are in the QA stage:
QAF RAC load testsRemoved, see [Ops] Run kibana quality gate suites #165346 (comment)Staging has:
Production has:
Quirks
SKIP_KIBANA_HOOKS=1needs to be set from the triggering jobelastic/kibanarepo's branches (because buildkite doesn't see forks' branches)Reference: https://docs.google.com/document/d/15rx2Z-soL20An0nBUcXX0o_HHf1OU_IgrHXgz20NndI/edit
Depends on: #165009
Closes: https://github.com/elastic/kibana-operations/issues/10