Skip to content

Commit

Permalink
feat(ecr-assets): simplify docker asset publishing (#5733)
Browse files Browse the repository at this point in the history
* feat(ecr-assets): simplify docker asset publishing

As part of our work on [continuous delivery for CDK apps], we decided to store all docker assets in a single ECR repository per environment. This is consistent with file assets, which are stored in a single S3 bucket which is part of the bootstrap stack. This environment-specific ECR repository uses a well-known physical name `aws-cdk/assets` and will be automatically created if needed. In the future, this repository will be created as part of the environment bootstrapping process.

The primary change is that the location of docker image assets will be fully determined by the framework. The ECR repository name will be hard-coded to `aws-cdk/assets` and the image tag will be based on the source hash of the docker asset. This basically means that we could get rid of the CloudFormation parameter that the CLI used to assign with the image name which helps to reduce the number of asset parameters (#3463).

Since from now on the asset ECR repository will contain different types of images (and not versions of the same image), there is no concept of "latest" image anymore and the optimization that was triggered by the `--ci` flag in the CLI is no longer relevant (pull the "latest"). Luckily CodeBuild now supports docker image layer caching, so this should be the preferred way to optimize docker build times. The `--ci` feature of the CLI is no longer doing anything.

Furthermore, before this change, in order to clean up ECR repositories, a custom resource called `AdoptedRepository` was automatically added to the stack for each asset. The purpose of this resource was to remove the asset's ECR repository it if the asset was no longer referenced by this stack. To address this need with the centralized repository, we plan to introduce a garbage collection capability that users will be able to invoke in order to clean up unused assets both from ECR and S3.

We will introduce a way to customize asset repository names as part of the CI/CD project. In the meantime, if you need to override the default "aws-cdk/assets" name, you can specify a repo name through the context key `assets-ecr-repository-name` (`--context` in the CLI, `cdk.json`, `new App({ context })` or `stack.setContext`).

BACKWARDS COMPATIBILITY

As per our guidelines for backwards compatibility, the CLI must be backwards compatible with apps from before this change. However, apps that use CDK >= 1.21.0 will require an upgrade of the CLI. Therefore, to introduce this change, we have made the following non-breaking changes in cx-api:

1. Make `imageNameParameter` optional. If it is specified, the CLI will continue ti
2. Add an optional `imageTag` which instructs the CLI what tag to use for the image. If omitted (by previous versions), `latest` will be used as before.

To make it easy to reason about the behavior for old apps, the CLI now has a new implementations for `prepareContainerAsset` called `prepareContainerImageAssetNew`. This new code path is triggered when the asset metadata *does not include* `imageNameParameter`. The new implementation requires that both `repositoryName` and `imageTag` will be defined. The old code path was only modified to support the new optional `imageTag` property (although it is unlikely to be exercised).

Additional backwards compatibility concerns:

- New ECR repositories the CLI creates will not have the lifecycle policy that retains only the last 5 docker images. This should not have a functional impact on users, but goes back to the imminent garbage collection project.
- The removal of the `AdoptedRepository` resource from all stacks will result in the deletion of all ECR previously created ECR repositories (this is what the AdoptedRepository resource is designed to do). This can be harmful since these repositories are being referenced by the stack. To address this, we invalidate the image ID by salting the source hash. This means that after this change, all container images will have a new ID, which is not maintained by the removed adopted repository resource.

TESTING

- Unit tests for `prepareContainerImage` were duplicated and extended to exercise the new code path while preserving tests for old path.
- All CLI integration tests were executed successfully against the new version.
- Manually tested that the new CLI works with old apps.

This change also fixes #5807 so that custom docker file names are relative and not absolute paths.

[continuous delivery for CDK apps]: aws/aws-cdk#3437

BREAKING CHANGE: all docker image assets are now pushed to a single ECR repository named `aws-cdk/assets` with an image tag based on the hash of the docker build source directory (the directory where your `Dockerfile` resides). See PR #5733 for details and discussion.

* update test expectations
  • Loading branch information
Elad Ben-Israel committed Jan 18, 2022
1 parent 399aa53 commit bcd85fc
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions packages/decdk/test/fixture/tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
"inlineSourceMap": true,
"inlineSources": true,
"lib": [
"es2016"
"es2018"
],
"module": "CommonJS",
"noEmitOnError": true,
Expand All @@ -22,7 +22,7 @@
"strictNullChecks": true,
"strictPropertyInitialization": true,
"stripInternal": true,
"target": "ES2017"
"target": "ES2018"
},
"include": [
"**/*.ts"
Expand Down

0 comments on commit bcd85fc

Please sign in to comment.