Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ops] Use vault kv store when on new Buildkite infra #174915

Merged
merged 5 commits into from
Jan 22, 2024

Conversation

delanni
Copy link
Contributor

@delanni delanni commented Jan 16, 2024

Summary

On the new Buildkite infra, our agents won't have write access to the paths we used to write the deployment information upon deployment.

We're allowed to use KV writes if we enable it (https://docs.elastic.dev/ci/using-secrets#using-shared-secrets, enabling PRs below).

I've built this in a way that we can enable the feature before the final rollout, and we can clear up the branches once done with the rollout.

Cloud deployment works on the old infra as well as the new, tested on this PR and #171317

Enabled by: https://github.com/elastic/ci/pull/2594 & https://github.com/elastic/ci/pull/2553
Part of: https://github.com/elastic/kibana-operations/issues/15
Related: https://elasticco.atlassian.net/browse/ENGPRD-414

@delanni delanni added Team:Operations Team label for Operations Team release_note:skip Skip the PR/issue when compiling release notes backport:skip This commit does not require backporting ci:cloud-deploy Create or update a Cloud deployment labels Jan 16, 2024
@delanni
Copy link
Contributor Author

delanni commented Jan 16, 2024

/ci

@delanni delanni marked this pull request as ready for review January 17, 2024 00:15
@delanni delanni requested a review from a team as a code owner January 17, 2024 00:15
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-operations (Team:Operations)

Copy link
Contributor

@Ikuni17 Ikuni17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one small comment, lgtm

@@ -171,15 +171,22 @@ download_artifact() {
retry 3 1 timeout 3m buildkite-agent artifact download "$@"
}

if [[ "$VAULT_ADDR" == *"secrets.elastic.co"* ]]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need the TODO: remove... comment as well so that we do not forget about it?

@kibana-ci
Copy link
Collaborator

kibana-ci commented Jan 22, 2024

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@delanni delanni merged commit dfb72ef into elastic:main Jan 22, 2024
17 checks passed
@delanni delanni deleted the use-kvstore-write-for-kme branch January 22, 2024 12:22
mistic pushed a commit that referenced this pull request Jan 22, 2024
…175223)

## Summary
In some scenarios, the VAULT_ADDR is not defined, or at least not at the
time when `util.sh` is loaded.

Follow up to #174915 - fixes:
https://buildkite.com/elastic/kibana-agent-packer-cache/builds/539#018d31d6-86bf-4f30-98fc-a82f42d16a4d
CoenWarmer pushed a commit to CoenWarmer/kibana that referenced this pull request Feb 15, 2024
## Summary
On the new Buildkite infra, our agents won't have write access to the
paths we used to write the deployment information upon deployment.

We're allowed to use KV writes if we enable it
(https://docs.elastic.dev/ci/using-secrets#using-shared-secrets,
enabling PRs below).

I've built this in a way that we can enable the feature before the final
rollout, and we can clear up the branches once done with the rollout.

Cloud deployment works on the old infra as well as the new, tested on
this PR and elastic#171317

Enabled by: elastic/ci#2594 &
elastic/ci#2553
Part of: elastic/kibana-operations#15
Related: https://elasticco.atlassian.net/browse/ENGPRD-414
CoenWarmer pushed a commit to CoenWarmer/kibana that referenced this pull request Feb 15, 2024
…lastic#175223)

## Summary
In some scenarios, the VAULT_ADDR is not defined, or at least not at the
time when `util.sh` is loaded.

Follow up to elastic#174915 - fixes:
https://buildkite.com/elastic/kibana-agent-packer-cache/builds/539#018d31d6-86bf-4f30-98fc-a82f42d16a4d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting ci:cloud-deploy Create or update a Cloud deployment release_note:skip Skip the PR/issue when compiling release notes Team:Operations Team label for Operations Team v8.13.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants