Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for storage backend #83

Closed
omerlh opened this issue Aug 24, 2020 · 1 comment · Fixed by #106
Closed

Add support for storage backend #83

omerlh opened this issue Aug 24, 2020 · 1 comment · Fixed by #106
Assignees
Labels
kind/enhancement Improvements or new features

Comments

@omerlh
Copy link

omerlh commented Aug 24, 2020

Pulumi support storing the state in a backend similar to Terraform - AWS S3, GCS etc
The suggestion is to add support for this - allow to set in the secret, instead of access token, the relevant backend:

stringData:
  account: "gs://pulumi-state"

Looking at the code, this is relatively simple to add - unless there is a reason it is not there from the beginning :)
This could also support on-prem deployments of pulumi - by setting the account key to the relevant server URL.

lukehoban pushed a commit to pulumi/pulumi that referenced this issue Nov 20, 2020
The PULUMI_BACKEND_URL env var allows specifying the backend to use instead of deferring to the project or the ~/.pulumi/credentials.json file to decide on the "current" backend.  This allows for using Pulumi without a dependence on this piece of global filesystem state, so that each `pulumi` invocation can control the exact backend it want's to operate on, without having to do stateful `pulumi login`/`pulumi logout` operations.

This is especially useful for automation scenarios like Automation API generally (and effectively solves #5591), or pulumi/pulumi-kubernetes-operator#83 specifically.

This also makes things like https://github.com/pulumi/pulumi/blob/efe7a599e65637378b9ceeac125386b5effd0d68/dist/actions/entrypoint.sh#L10 less necessary, and possible to accomplish for any containerized `pulumi` execution without the need for this logic to be embedded in bash scripts wrapping the CLI.
lukehoban pushed a commit that referenced this issue Nov 20, 2020
Adds a top level `backend` field to the Stack CR that allows pointing all operations for the target Stack at the given backend.  In cases where `backend` is set to a non-service backend, the AccessTokenSecret field is also no longer required.

This is dependent on pulumi/pulumi#5789, and cannot be merged until the operator can take a dependency on that change in the core `pulumi` CLI.

Fixes #83.
@lukehoban lukehoban self-assigned this Nov 20, 2020
lukehoban pushed a commit to pulumi/pulumi that referenced this issue Nov 20, 2020
The PULUMI_BACKEND_URL env var allows specifying the backend to use instead of deferring to the project or the ~/.pulumi/credentials.json file to decide on the "current" backend.  This allows for using Pulumi without a dependence on this piece of global filesystem state, so that each `pulumi` invocation can control the exact backend it want's to operate on, without having to do stateful `pulumi login`/`pulumi logout` operations.

This is especially useful for automation scenarios like Automation API generally (and effectively solves #5591), or pulumi/pulumi-kubernetes-operator#83 specifically.

This also makes things like https://github.com/pulumi/pulumi/blob/efe7a599e65637378b9ceeac125386b5effd0d68/dist/actions/entrypoint.sh#L10 less necessary, and possible to accomplish for any containerized `pulumi` execution without the need for this logic to be embedded in bash scripts wrapping the CLI.
@lukehoban
Copy link
Contributor

The changes to the Pulumi CLI proposed in pulumi/pulumi#5789 would enable this to be accomplished with this:

apiVersion: v1
kind: ConfigMap
metadata:
  name: pulumi-s3-backend
data:
  PULUMI_BACKEND_URL: s3://luke-s3-backend
---
apiVersion: pulumi.com/v1alpha1
kind: Stack
metadata:
  name: s3-bucket-stack
spec:
  stack: lukehoban/operator-dev
  projectRepo: https://github.com/lukehoban/pytest1
  initOnCreate: true
  commit: 078124dae5f3c01e59619ee6947ee3cda260586b
  config:
    aws:region: us-west-2
  envs:
    - pulumi-s3-backend

In #106 there is a proposed extension to the operator to make this more first-class.

apiVersion: pulumi.com/v1alpha1
kind: Stack
metadata:
  name: s3-bucket-stack
spec:
  backend: s3://luke-s3-backend
  stack: lukehoban/operator-dev
  projectRepo: https://github.com/lukehoban/pytest1
  initOnCreate: true
  commit: 078124dae5f3c01e59619ee6947ee3cda260586b
  config:
    aws:region: us-west-2

lukehoban pushed a commit to pulumi/pulumi that referenced this issue Nov 22, 2020
The PULUMI_BACKEND_URL env var allows specifying the backend to use instead of deferring to the project or the ~/.pulumi/credentials.json file to decide on the "current" backend.  This allows for using Pulumi without a dependence on this piece of global filesystem state, so that each `pulumi` invocation can control the exact backend it want's to operate on, without having to do stateful `pulumi login`/`pulumi logout` operations.

This is especially useful for automation scenarios like Automation API generally (and effectively solves #5591), or pulumi/pulumi-kubernetes-operator#83 specifically.

This also makes things like https://github.com/pulumi/pulumi/blob/efe7a599e65637378b9ceeac125386b5effd0d68/dist/actions/entrypoint.sh#L10 less necessary, and possible to accomplish for any containerized `pulumi` execution without the need for this logic to be embedded in bash scripts wrapping the CLI.
lukehoban pushed a commit that referenced this issue Dec 3, 2020
Adds a top level `backend` field to the Stack CR that allows pointing all operations for the target Stack at the given backend.  In cases where `backend` is set to a non-service backend, the AccessTokenSecret field is also no longer required.

This is dependent on pulumi/pulumi#5789, and cannot be merged until the operator can take a dependency on that change in the core `pulumi` CLI.

Fixes #83.
lukehoban pushed a commit that referenced this issue Dec 3, 2020
Adds a top level `backend` field to the Stack CR that allows pointing all operations for the target Stack at the given backend.  In cases where `backend` is set to a non-service backend, the AccessTokenSecret field is also no longer required.

This is dependent on pulumi/pulumi#5789, and cannot be merged until the operator can take a dependency on that change in the core `pulumi` CLI.

Fixes #83.
@infin8x infin8x added the kind/enhancement Improvements or new features label Jul 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement Improvements or new features
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants