From 72723e194caf550740ec2f883b7808b0d6e1804d Mon Sep 17 00:00:00 2001 From: Elad Ben-Israel Date: Wed, 18 Sep 2019 15:49:29 +0300 Subject: [PATCH 1/3] rfc(core): default stack environments --- design/default-env.md | 60 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) create mode 100644 design/default-env.md diff --git a/design/default-env.md b/design/default-env.md new file mode 100644 index 0000000000000..f52a8f84eb04d --- /dev/null +++ b/design/default-env.md @@ -0,0 +1,60 @@ +# RFC: Default Stack Environments + +## Background + +At the moment, if a `Stack` is defined without `env`, it is said to be an environment-agnostic stack. This technically means that `stack.account` and `stack.region` will resolve to `{ "Ref": "AWS::AccountId" }` and `{ "Ref": "AWS::Region" }`. The implication are that code that relies on parsing one of these values will fail (e.g. the EKS library has an [AMI map](https://github.com/aws/aws-cdk/blob/master/packages/%40aws-cdk/aws-eks/lib/ami.ts) keyed by `stack.region`). + +Users are able to explicitly "inherit" the CLI's default environment by specifying using a couple of environment variables passed through from the CLI: + +```ts +new Stack(this, 'MyStack', { + env: { + account: process.env.CDK_DEFAULT_ACCOUNT, + region: process.env.CDK_DEFAULT_REGION + } +} +``` + +Prior to introducing explicit support for [environment agnostic stacks](https://github.com/aws/aws-cdk/pull/2922), using the current account and region was the default behavior. + +The problem with this approach was that if users will `cdk synth` such an app and would want to use the resulting artifact (template/cloud-assembly), they can't rely on the fact that results will be deterministic since they will be dependent on the specific runtime environment. For example, in one environment `AWS_DEFAULT_REGION` can be set to `us-east-1` and in another it will be set to `us-west-2`, which can technically result in a totally different output. + +## Problem + +The current situation creates undesired friction when getting started with certain modules of the CDK (and potentially 3rd party modules as well that depend on explicit env). For example, if a user wishes to deploy an EKS cluster with `cdk deploy`, they will have to define their stacks with explicit `env` (either specify account/region or use `CDK_DEFAULT_XXX`). + +## Opportunity + +When using `cdk deploy` directly with an app that has stack(s) without explicit `env`, the synthesized output is an **intermediate artifact** that is never visible. In this case, defaulting to the current account/region is safe and is probably what most users expect. + +## Proposal + +The proposal is: + +1. Add an `--default-env=inherit|explicit|agnostic` switch to the CLI which will control the default env mode. +2. Extend the semantics of `CDK_DEFAULT_ACCOUNT` and `CDK_DEFAULT_REGION` to be able to express these three modes (currently it cannot express environment-agnosticness). +3. Change the default behavior of `env` in `Stack` to fall back to `CDK_DEFAULT_ACCOUNT` and `CDK_DEFAULT_REGION` if an explicit value is not specified. +4. If `env` is not explicitly set and `CDK_DEFAULT_XXX` are not defined, stack initialization will fail with an error. +5. The default mode for `cdk deploy` will be `inherit`. This is because when a stack without env is deployed with the CLI, we *know* where it's going. There's no point playing this game here. +6. The default mode for `cdk synth` will be `explicit`. +7. Allow users to *explicitly* indicate that a stack is environment-agnostic when they define the stack. They can achieve this today by specifying `{ env: { account: Aws.ACCOUNT, region: Aws.REGION } }`. + +## Implications + +This means, that now, if users wish to explicitly synthesize environment-agnostic template(s), they will have to opt-in for those stacks to be environment-agnostic (we can also provide an a constant such as `{ env: Stack.ANY_ENVIRONMENT }`) as sugar). + +It also means that `cdk synth` will fail unless all stacks have a specific environment specified (either env-specific or env-agnostic), which may be undesirable for certain use cases. Let's look at what people use `cdk synth` for: + +1. **Inspection/debugging/demo**: usually this involves emitting a template output to STDOUT and inspecting it or showing it to others. +2. **CI/CD**: using `cdk.out` (the "cloud assembly") or the templates inside it as a self-contained artifact which can later be deployed using `cdk-deploy --app cdk.out`. + +Practically this means we have two `synth` commands: one which prints to STDOUT and one which stores the output in `cdk.out` (or the directory specified in `--out`). The latter is designed for the CI/CD use case and the former is designed for the inspection use case. + +The 2nd use case (CI/CD), where `cdk synth` is used during build to produce a deployment artifact, we feel that requiring explicitness is actually a benefit. Deterministic infrastructure definitions are an important tenent of the CDK, and requiring that users be explicit when they deploy apps through CI/CD is actually an added value. To allow users to opt-in for the previous behavior, we can add a switch `cdk synth --default-env=inherit` (as oppose to the default `--default-env=explicit`). + +The 1st use case, however, where `cdk synth` is used to print a template to STDOUT for inspection is something we still wish to support, without requiring users to explicitly specify `env`s. The current behavior of `cdk synth` is a bit convoluted as is: + +- If the app includes more than a single stack, `cdk synth` requires that you specify the stack name in order to print to STDOUT, otherwise it just creates `cdk.out`. +- When `cdk synth` prints to STDOUT is prints in YAML (unless `--json` is specified), while `cdk.out` always uses JSON ([#2965](https://github.com/aws/aws-cdk/issues/2965)). +- `cdk.out` is created no matter what, for both use cases of `synth` and even if it the app was synthesized implicitly as part of `cdk deploy`. This also somewhat causes confusion. + From 03961da2ec557b7de2fc1ceab9619bbce9cfb36b Mon Sep 17 00:00:00 2001 From: Elad Ben-Israel Date: Wed, 18 Sep 2019 17:53:39 +0300 Subject: [PATCH 2/3] Update default-env.md --- design/default-env.md | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/design/default-env.md b/design/default-env.md index f52a8f84eb04d..46bc630f6c725 100644 --- a/design/default-env.md +++ b/design/default-env.md @@ -4,7 +4,7 @@ At the moment, if a `Stack` is defined without `env`, it is said to be an environment-agnostic stack. This technically means that `stack.account` and `stack.region` will resolve to `{ "Ref": "AWS::AccountId" }` and `{ "Ref": "AWS::Region" }`. The implication are that code that relies on parsing one of these values will fail (e.g. the EKS library has an [AMI map](https://github.com/aws/aws-cdk/blob/master/packages/%40aws-cdk/aws-eks/lib/ami.ts) keyed by `stack.region`). -Users are able to explicitly "inherit" the CLI's default environment by specifying using a couple of environment variables passed through from the CLI: +Users are able to explicitly "inherit" the CLI's current environment by using a couple of environment variables which are always populated by the CLI: ```ts new Stack(this, 'MyStack', { @@ -15,9 +15,9 @@ new Stack(this, 'MyStack', { } ``` -Prior to introducing explicit support for [environment agnostic stacks](https://github.com/aws/aws-cdk/pull/2922), using the current account and region was the default behavior. +Prior to introducing explicit support for environment agnostic stacks ([PR](https://github.com/aws/aws-cdk/pull/2922)), falling back to the CLI's current environment was the default behavior if you didn't specify `env`. -The problem with this approach was that if users will `cdk synth` such an app and would want to use the resulting artifact (template/cloud-assembly), they can't rely on the fact that results will be deterministic since they will be dependent on the specific runtime environment. For example, in one environment `AWS_DEFAULT_REGION` can be set to `us-east-1` and in another it will be set to `us-west-2`, which can technically result in a totally different output. +The problem with this approach was that if users wanted to use `cdk synth` and use the resulting artifact (template/cloud-assembly) in CI/CD systems, they couldn't rely on the fact that results will be deterministic since they will be dependent on the specific runtime environment. For example, in one environment `AWS_DEFAULT_REGION` can be set to `us-east-1` and in another it will be set to `us-west-2`, which can technically result in a totally different output. ## Problem @@ -34,27 +34,28 @@ The proposal is: 1. Add an `--default-env=inherit|explicit|agnostic` switch to the CLI which will control the default env mode. 2. Extend the semantics of `CDK_DEFAULT_ACCOUNT` and `CDK_DEFAULT_REGION` to be able to express these three modes (currently it cannot express environment-agnosticness). 3. Change the default behavior of `env` in `Stack` to fall back to `CDK_DEFAULT_ACCOUNT` and `CDK_DEFAULT_REGION` if an explicit value is not specified. -4. If `env` is not explicitly set and `CDK_DEFAULT_XXX` are not defined, stack initialization will fail with an error. +4. If `env` is not explicitly set and `CDK_DEFAULT_XXX` is undefined, stack initialization **will fail with an error**. 5. The default mode for `cdk deploy` will be `inherit`. This is because when a stack without env is deployed with the CLI, we *know* where it's going. There's no point playing this game here. -6. The default mode for `cdk synth` will be `explicit`. -7. Allow users to *explicitly* indicate that a stack is environment-agnostic when they define the stack. They can achieve this today by specifying `{ env: { account: Aws.ACCOUNT, region: Aws.REGION } }`. +6. The default mode for `cdk synth` will be `explicit`. The implication of this (detalied below) is that if users wish to synthesize an artifact from their CDK app and deploy it at a later stage, they will have to either explicitly specify `env` for all stacks or use `cdk synth --default-env=inherit|agnostic`. +7. Make it easier to explicitly indicate that a stack is environment-agnostic. This can already be acheieved today using `{ env: { region: Aws.REGION, account: Aws.ACCOUNT_ID } }` but can be made easier with `{ env: Stack.ANY_ENV }`. ## Implications -This means, that now, if users wish to explicitly synthesize environment-agnostic template(s), they will have to opt-in for those stacks to be environment-agnostic (we can also provide an a constant such as `{ env: Stack.ANY_ENVIRONMENT }`) as sugar). +It also means that `cdk synth` will fail unless all stacks have a specific environment specified (either env-specific or env-agnostic), which may be undesirable for certain use cases. -It also means that `cdk synth` will fail unless all stacks have a specific environment specified (either env-specific or env-agnostic), which may be undesirable for certain use cases. Let's look at what people use `cdk synth` for: +Let's look at what people use `cdk synth` for: 1. **Inspection/debugging/demo**: usually this involves emitting a template output to STDOUT and inspecting it or showing it to others. 2. **CI/CD**: using `cdk.out` (the "cloud assembly") or the templates inside it as a self-contained artifact which can later be deployed using `cdk-deploy --app cdk.out`. -Practically this means we have two `synth` commands: one which prints to STDOUT and one which stores the output in `cdk.out` (or the directory specified in `--out`). The latter is designed for the CI/CD use case and the former is designed for the inspection use case. +For the 2nd use case (CI/CD), where `cdk synth` is used during build to produce a deployment artifact, we feel that requiring explicitness is actually a benefit. Deterministic infrastructure definitions are an important tenent of the AWS CDK, and requiring that users be explicit when they deploy apps through CI/CD is actually an added value. If users wish to synthesize environment agnostic templates, they can simply specify `env: Stack.ANY_ENV` when they define their stacks. -The 2nd use case (CI/CD), where `cdk synth` is used during build to produce a deployment artifact, we feel that requiring explicitness is actually a benefit. Deterministic infrastructure definitions are an important tenent of the CDK, and requiring that users be explicit when they deploy apps through CI/CD is actually an added value. To allow users to opt-in for the previous behavior, we can add a switch `cdk synth --default-env=inherit` (as oppose to the default `--default-env=explicit`). +The 1st use case, however, where `cdk synth` is used to print a template to STDOUT for inspection, is something we still wish to support without requiring users to explicitly specify `env`s. -The 1st use case, however, where `cdk synth` is used to print a template to STDOUT for inspection is something we still wish to support, without requiring users to explicitly specify `env`s. The current behavior of `cdk synth` is a bit convoluted as is: +Before we decide what to do about that, we should acknowledge that the current behavior of `cdk synth` is a bit confusing today: - If the app includes more than a single stack, `cdk synth` requires that you specify the stack name in order to print to STDOUT, otherwise it just creates `cdk.out`. - When `cdk synth` prints to STDOUT is prints in YAML (unless `--json` is specified), while `cdk.out` always uses JSON ([#2965](https://github.com/aws/aws-cdk/issues/2965)). - `cdk.out` is created no matter what, for both use cases of `synth` and even if it the app was synthesized implicitly as part of `cdk deploy`. This also somewhat causes confusion. +We propose to separate the use cases into two different CLI commands: `cdk synth` and `cdk build`. The first will serve the inspection use case and the second for the CI/CD use case. This way, we can decide that `--default-env` will have a different default behavior for these two commands. For `cdk synth` (STDOUT) we will default to `agnostic` and for `cdk build` we will default to `explicit`. From 065da665bada196307bff4b286b6d9bcd27c152e Mon Sep 17 00:00:00 2001 From: Elad Ben-Israel Date: Thu, 19 Sep 2019 09:21:48 +0300 Subject: [PATCH 3/3] "print" and "synth" (instead of "build") --- design/default-env.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/design/default-env.md b/design/default-env.md index 46bc630f6c725..a998a6ed35e4f 100644 --- a/design/default-env.md +++ b/design/default-env.md @@ -58,4 +58,4 @@ Before we decide what to do about that, we should acknowledge that the current b - When `cdk synth` prints to STDOUT is prints in YAML (unless `--json` is specified), while `cdk.out` always uses JSON ([#2965](https://github.com/aws/aws-cdk/issues/2965)). - `cdk.out` is created no matter what, for both use cases of `synth` and even if it the app was synthesized implicitly as part of `cdk deploy`. This also somewhat causes confusion. -We propose to separate the use cases into two different CLI commands: `cdk synth` and `cdk build`. The first will serve the inspection use case and the second for the CI/CD use case. This way, we can decide that `--default-env` will have a different default behavior for these two commands. For `cdk synth` (STDOUT) we will default to `agnostic` and for `cdk build` we will default to `explicit`. +We propose to separate the "synth" use cases into two different CLI commands: `cdk print` and `cdk synth`. The first will serve the inspection use case and the second the CI/CD use case. This way, we can decide that `--default-env` will have a different default behavior for these two commands. For `cdk print` we will default to `agnostic` and for `cdk synth` we will default to `explicit`.