[Design] Environment manifest

[Lou1415926]

This issue is a design proposal to introduce manifest files for environments.

By introducing a declarative configuration similar to service or job manifests, environments will gain all the benefits of infrastructure-as-code: code reviews, easy to replicate and mutate. The manifest file will also allow Copilot to surface more environment configuration over time.

Problem statement

Today, environments can only be configured during creation with copilot env init <flags>. It’s not possible to toggle features without re-creating the environment (such as --container-insights). This becomes especially problematic as Copilot surfaces new environment configuration fields such as --import-cert-arns (#3503) that users cannot enable in their existing environments. Furthermore, commands cannot be code reviewed to ensure the organization standards are met.

Proposal

New deploy workflow

Running copilot env init -n <name> will now generate a manifest file for the environment under:

copilot/
└── environments/
    └── test/
        └── manifest.yml

After peeking and optionally making modifications to the manifest file, users will be able to deploy their environments using a new command: copilot env deploy.
Note: this means that once this feature is released, env init will no longer automatically deploy environments. Instead, clients will have to run env deploy to create or update environment stacks.

In order to view the CloudFormation template that will be generated, users can run copilot env package.

In order to generate a manifest file for existing environments, a new flag will be introduced to copilot env show --manifest that will print the manifest file representing their existing environment. Therefore, users with existing environments can create a manifest using env show —manifest instead of having deleting and re-creating the environment for the sake of manifest.

Manifest specification

The environment manifest file will initially ship with just a parity with the existing env init flags:

name: prod
type: Environment
network:
    vpc:
        id: 'vpc-12345'           # Import.
        cidr: '10.0.0.0/16'       # Configure.
        subnets:
            public:
                - id: 'subnet-11111'    # Import.
                  cidr: '10.0.0.0/24'   # Configure.
                  az: 'us-east-2a'      # Configure.
                - id: 'subnet-22222'    # Import.
                  cidr: '10.0.1.0/24'   # Configure.
                  az: 'us-east-2b'      # Configure.
            private:
                - id: 'subnet-33333'    # Import.
                  cidr: '10.0.3.0/24'   # Configure.
                  az: 'us-east-2a'      # Configure.
                - id: 'subnet-44444'    # Import.
                  cidr: '10.0.4.0/24'   # Configure.
                  az: 'us-east-2b'      # Configure.
http:  
    internet:
        cert:
            - certificate-arn-1
            - certificate-arn-2

observability:
    container_insights: true

Over time, we’ll add new properties to the manifest file to surface more power for environment such as security groups for load balancers, number of AZs in a VPC, private domain names.

Continuous delivery

Running copilot pipeline init will now prompt whether you’d like to create a pipeline for deploying your services or environments:

$ copilot pipeline init
Pipeline name: release-envs
What would you like to deploy with this pipeline?
> Services or jobs
> Environments
Repository url: https://github.com/acmecorp/repo

And the appropriate pipeline manifest and buildspec will be generated for you. Existing pipelines won’t be affected.

---

We’re extremely excited to surface environment manifests and bring more configurability for environments over time. Please let us know if you’ve any feedback!

[Lou1415926]

Related: #3030

[efekarakus]

This feature is now out in v1.20 🚀
Release notes: https://github.com/aws/copilot-cli/releases/tag/v1.20.0 !
Blog post: https://aws.github.io/copilot-cli/blogs/release-v120/