Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for overriding storage API endpoint for GCS #737

Merged
merged 2 commits into from
Dec 26, 2023

Conversation

shreyas-s-rao
Copy link
Contributor

@shreyas-s-rao shreyas-s-rao commented Dec 18, 2023

How to categorize this PR?

/area backup
/kind enhancement
/platform gcp

What this PR does / why we need it:
Add support for overriding storage API endpoint for Google GCS provider. This allows users to specify their own storage API endpoint override, such as a regional endpoint, or even a local GCS emulator endpoint. Users need to add a new field storageAPIEndpoint in their GCS backup secret, with the value in the format http[s]://host[:port]/storage/v1/, to utilise this feature.

Additionally, clean up some of the code and reduce duplicate code:

  • combine controllers/compaction.getCompactionJobEnvVar() and pkg/component/etcd/statefulset.getBackupRestoreEnvVars() into pkg/utils.GetBackupRestoreContainerEnvVars() under pkg/utils/envvar.go, along with associated helper functions
  • rename env var constants from pkg/common/constants.go to the format Env* with correct camel casing, and use in pkg/utils.GetBackupRestoreContainerEnvVars()

Which issue(s) this PR fixes:
Fixes #736

Special notes for your reviewer:
/cc @dguendisch @anveshreddy18
/invite @ishan16696 @seshachalam-yv

Release note:

Add support for overriding storage API endpoint for provider GCS, by adding new field `storageAPIEndpoint` in the GCP/GCS backup secret, with the value in the format `http[s]://host[:port]/storage/v1/`. ⚠️ Note: GCS storage API endpoint will not be overridden for `EtcdCopyBackupsTask`s, since backup buckets may reside in different regions.

@shreyas-s-rao shreyas-s-rao added this to the v0.22.0 milestone Dec 18, 2023
@shreyas-s-rao shreyas-s-rao requested a review from a team as a code owner December 18, 2023 06:50
@gardener-robot gardener-robot added the needs/review Needs review label Dec 18, 2023
@gardener-robot gardener-robot added area/backup Backup related kind/enhancement Enhancement, improvement, extension platform/gcp Google cloud platform/infrastructure size/l Size of pull request is large (see gardener-robot robot/bots/size.py) needs/second-opinion Needs second review by someone else labels Dec 18, 2023
@gardener-robot-ci-3 gardener-robot-ci-3 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Dec 18, 2023
@gardener-robot-ci-1 gardener-robot-ci-1 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Dec 18, 2023
@shreyas-s-rao shreyas-s-rao added needs/cherry-pick Needs to be cherry-picked to older version and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Dec 18, 2023
@gardener-robot gardener-robot added size/xl Size of pull request is huge (see gardener-robot robot/bots/size.py) and removed size/l Size of pull request is large (see gardener-robot robot/bots/size.py) labels Dec 18, 2023
@gardener-robot-ci-2 gardener-robot-ci-2 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Dec 18, 2023
@shreyas-s-rao
Copy link
Contributor Author

/retest

@seshachalam-yv
Copy link
Contributor

seshachalam-yv commented Dec 18, 2023

/assign @seshachalam-yv

@seshachalam-yv seshachalam-yv self-assigned this Dec 18, 2023
@ishan16696
Copy link
Member

/assign

@gardener-robot
Copy link

@seshachalam-yv, @ishan16696 You have pull request review open invite, please check

Copy link
Contributor

@seshachalam-yv seshachalam-yv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @shreyas-s-rao for introducing new constants for env vars.

Overall looks good to me.
Nice PR ❤️
/lgtm

@gardener-robot gardener-robot added the reviewed/lgtm Has approval for merging label Dec 22, 2023
@gardener-robot gardener-robot removed needs/review Needs review needs/second-opinion Needs second review by someone else labels Dec 22, 2023
@gardener-robot-ci-2 gardener-robot-ci-2 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Dec 22, 2023
@gardener-robot gardener-robot added needs/second-opinion Needs second review by someone else and removed reviewed/lgtm Has approval for merging labels Dec 22, 2023
@gardener-robot-ci-3 gardener-robot-ci-3 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Dec 22, 2023
@seshachalam-yv
Copy link
Contributor

/test pull-etcd-druid-e2e-kind

@ishan16696
Copy link
Member

/test pull-etcd-druid-e2e-kind

@seshachalam-yv seshachalam-yv merged commit da48402 into gardener:master Dec 26, 2023
1 check passed
@gardener-robot gardener-robot added the status/closed Issue is closed (either delivered or triaged) label Dec 26, 2023
@shreyas-s-rao shreyas-s-rao deleted the enh/gcs-rep branch January 1, 2024 03:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/backup Backup related kind/enhancement Enhancement, improvement, extension needs/cherry-pick Needs to be cherry-picked to older version needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) needs/second-opinion Needs second review by someone else platform/gcp Google cloud platform/infrastructure size/xl Size of pull request is huge (see gardener-robot robot/bots/size.py) status/closed Issue is closed (either delivered or triaged)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature] Consume GCS regional endpoints feature from etcd-backup-restore
7 participants