docs: add task def override #2763
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
iamhopaul123
force-pushed
the
taskdef-doc
branch
from
ba702f7 to
8ee445a
Compare
huanjani
reviewed
Aug 25, 2021
|
|
||
| - Use `-` as index to append a new member to a `list` field. The field will be initiated with this new member if it doesn't exist | ||
|
|
||
| - **value** must be scalar value like `bool`, `string`, or `int`, and the new value replaces the old value if exists |
There was a problem hiding this comment.
nit:
Suggested change
| - **value** must be scalar value like `bool`, `string`, or `int`, and the new value replaces the old value if exists | |
| - **value** must be a scalar value like `bool`, `string`, or `int`, and the new value replaces an existing value |
huanjani
reviewed
Aug 25, 2021
| value: 2048 | ||
| ``` | ||
|
|
||
| Below is exposing an extra UDP port for the main container. |
There was a problem hiding this comment.
Suggested change
| Below is exposing an extra UDP port for the main container. | |
| Below is an example of exposing an extra UDP port for the main container. |
Lou1415926
reviewed
Aug 25, 2021
| @@ -0,0 +1,50 @@ | |||
| # Overrides | |||
| If there are fields that are not configurable in the [manifest](../manifest/overview.en.md), users can bypass some of them by applying override rules to the CloudFormation template Copilot generates out of the manifest. | |||
There was a problem hiding this comment.
We can probably mention that right now only task definition override is supported. Otherwise the user would need to infer this knowledge from the fact that there is only taskdef_overrides field in manifest.
There was a problem hiding this comment.
Agreed I think we should be specific to only task def overrides in this doc. Eventually we can create more pages if we have other types of overrides.
There was a problem hiding this comment.
Thought about that too. Problem is that once we add more sections I don't want to add them accordingly. Maybe it's ok for them to figure out what's available? I'm not sure doc writing is so hard
There was a problem hiding this comment.
Agreed I think we should be specific to only task def overrides in this doc. Eventually we can create more pages if we have other types of overrides.
But then they'll share the same overriding pattern. should we concentrate them into one override page?
There was a problem hiding this comment.
ok discussed offline we'll start from very specific page then in future we might move them together.
| For each override rule, users need to construct the **path** and **value** of the CloudFormation resource field they want to override. | ||
|
|
||
| ``` yaml | ||
| <override section>: |
There was a problem hiding this comment.
If we only have taskdef_overrides and don't have specific plan in expanding this to other resources right now, I think we can just say taskdef_overrides instead of <override section>
There was a problem hiding this comment.
But then wouldn't they assume override = taskdef_override now and in future?
|
|
||
| ``` yaml | ||
| <override section>: | ||
| - path: <CFN field path> |
There was a problem hiding this comment.
We can probably add some description of what exactly is a CFN field path, accompanied by some examples. Otherwise the user would still need to infer this from the "Task Definition Override" example below.
For example, how deep a path need to be (e.g. can I just write ContainerDefinitions[0]: blahblah? -- which I know we can't), how to write a path that contains list parts? etc.
There was a problem hiding this comment.
Otherwise the user would still need to infer this from the "Task Definition Override" example below.
Hmm this is exactly why we have example section right 🤔 I was referring to CDK https://docs.aws.amazon.com/cdk/latest/guide/cfn_layer.html#cfn_layer_raw and for those who want to use this feature it might be safe for us to assume they have some CFN background.
|
|
||
| ## Override Behaviors | ||
|
|
||
| - Copilot recursively creates `map` fields if they don't exist in the **path**. For example: if `B` and `C` don't exist, `A.B.C` will create `B` and `C` |
There was a problem hiding this comment.
Can it be "Copilot recursively creates fields....", removing the word map? I'm not sure if this is a universal terminology used across CloudFormation templates.
There was a problem hiding this comment.
that's a good idea!
| ## Override Behaviors | ||
|
|
||
| - Copilot recursively creates `map` fields if they don't exist in the **path**. For example: if `B` and `C` don't exist, `A.B.C` will create `B` and `C` | ||
|
|
There was a problem hiding this comment.
Also, maybe we can include a general description first, something like "When interpreting the override rule, Copilot insert or update (upsert) the fields along the path."
And then discuss the "insert" case: Copilot recursively creates fields if they don't exist...
|
|
||
| - Copilot recursively creates `map` fields if they don't exist in the **path**. For example: if `B` and `C` don't exist, `A.B.C` will create `B` and `C` | ||
|
|
||
| - Use `-` as index to append a new member to a `list` field. The field will be initiated with this new member if it doesn't exist |
There was a problem hiding this comment.
Suggested change
| - Use `-` as index to append a new member to a `list` field. The field will be initiated with this new member if it doesn't exist | |
| - Use `-` as index to append a new member to the end of a `list` field. |
I'm also not sure what does "The field will be initiated with this new member if it doesn't exist" mean 🤔
|
|
||
| - Use `-` as index to append a new member to a `list` field. The field will be initiated with this new member if it doesn't exist | ||
|
|
||
| - **value** must be scalar value like `bool`, `string`, or `int`, and the new value replaces the old value if exists |
There was a problem hiding this comment.
If bool, string and int are the only acceptable scalar value, we should say so: "value must be scalar value: bool, string, or int"
There was a problem hiding this comment.
Is it strictly scalars, will this blow up?
- path: ContainerDefinitions[0].Ulimits[-]
value:
Name: "cpu"
SoftLimit: 1024
HardLimit: 2048
There was a problem hiding this comment.
hmm actually this will not blow up
| !!! Attention | ||
| Users are not allowed to modify the following fields in the task definition. | ||
|
|
||
| * Family |
There was a problem hiding this comment.
Suggested change
| * Family | |
| * [Family](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-taskdefinition.html#cfn-ecs-taskdefinition-family) |
| Users are not allowed to modify the following fields in the task definition. | ||
|
|
||
| * Family | ||
| * ContainerDefinitions[<index>].Name (name of any existing container) |
There was a problem hiding this comment.
Suggested change
| * ContainerDefinitions[<index>].Name (name of any existing container) | |
| * [ContainerDefinitions[<index>].Name](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ecs-taskdefinition-containerdefinitions.html#cfn-ecs-taskdefinition-containerdefinition-name) |
I wonder if instead of "(name of any existing container)", we should just say they can't change the name of any container definitions created by Copilot.
| taskdef_overrides: | ||
| - path: "ContainerDefinitions[0].PortMappings[-].ContainerPort" | ||
| value: 2056 | ||
| - path: "ContainerDefinitions[0].PortMappings[1].Protocol" |
There was a problem hiding this comment.
We should explain how PortMappins[1] is the port mappings added by PortMappins[-] and how the order of the rules matters.
efekarakus
reviewed
Aug 25, 2021
| @@ -0,0 +1,50 @@ | |||
| # Overrides | |||
There was a problem hiding this comment.
Can we rename the file and this header to Task Definition Overrides? 🙏
| @@ -0,0 +1,50 @@ | |||
| # Overrides | |||
| If there are fields that are not configurable in the [manifest](../manifest/overview.en.md), users can bypass some of them by applying override rules to the CloudFormation template Copilot generates out of the manifest. | |||
There was a problem hiding this comment.
Agreed I think we should be specific to only task def overrides in this doc. Eventually we can create more pages if we have other types of overrides.
|
|
||
| - Use `-` as index to append a new member to a `list` field. The field will be initiated with this new member if it doesn't exist | ||
|
|
||
| - **value** must be scalar value like `bool`, `string`, or `int`, and the new value replaces the old value if exists |
There was a problem hiding this comment.
Is it strictly scalars, will this blow up?
- path: ContainerDefinitions[0].Ulimits[-]
value:
Name: "cpu"
SoftLimit: 1024
HardLimit: 2048
Lou1415926
reviewed
Aug 25, 2021
| SoftLimit: 1024 | ||
| HardLimit: 2048 | ||
| ``` | ||
|
|
There was a problem hiding this comment.
We might want to explain a bit how this works, for example
What Copilot does:
1. In the CloudFormation template that Copilot generates, the first `ContainerDefinition` is the main container for your workload. Using this override rule, Copilot looks for the first `ContainerDefinition` (i.e. the main container definition).
2. Since the `Ulimits` field doesn't exist in the original template, Copilot inserts a `Ulimits` field.
3. To add a first element to the `Ulimits` field, use `[-]` to signal an "append" action. You can think of this as appending to an empty list.
and samesies for the other examples.
efekarakus
reviewed
Aug 26, 2021
mkdocs.yml
Outdated
| @@ -47,6 +47,7 @@ nav: | |||
| - Custom Environment Resources: docs/developing/custom-environment-resources.en.md | |||
| - Secrets: docs/developing/secrets.en.md | |||
| - Service Discovery: docs/developing/service-discovery.en.md | |||
| - Task Definition Overrides: docs/developing/taskdef-overrides.en.md | |||
There was a problem hiding this comment.
let's move this to last in the list as it's an advanced feature 🙏
| @@ -0,0 +1,57 @@ | |||
| # Task Definition Overrides | |||
| Copilot generates CloudFormation templates using configuration specified in the [manifest](../manifest/overview.en.md). However, there are fields that are not configurable in the manifest. For example, You might want to configure the [`Ulimits`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ecs-taskdefinition-containerdefinitions.html#cfn-ecs-taskdefinition-containerdefinition-ulimits) for your workload container, but it is not exposed in our manifest. | |||
| # Task Definition Overrides | ||
| Copilot generates CloudFormation templates using configuration specified in the [manifest](../manifest/overview.en.md). However, there are fields that are not configurable in the manifest. For example, You might want to configure the [`Ulimits`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ecs-taskdefinition-containerdefinitions.html#cfn-ecs-taskdefinition-containerdefinition-ulimits) for your workload container, but it is not exposed in our manifest. | ||
|
|
||
| You can configure [ECS Task Definition settings](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-taskdefinition.html) by specifying override rules, which will be applied to the CloudFormation template that Copilot generates out of the manifest. |
There was a problem hiding this comment.
Suggested change
| You can configure [ECS Task Definition settings](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-taskdefinition.html) by specifying override rules, which will be applied to the CloudFormation template that Copilot generates out of the manifest. | |
| You can configure additional [ECS Task Definition settings](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-taskdefinition.html) by specifying `taskdef_overrides` rules, which will be applied to the CloudFormation template that Copilot generates out of the manifest. |
|
|
||
| ## Override Behaviors | ||
|
|
||
| - Use `-` as index to append a new member to a `list` field |
There was a problem hiding this comment.
Can we make this the last item in the list?
Comment on lines
8
to
13
|
|
||
| ``` yaml | ||
| taskdef_overrides: | ||
| - path: <ECS Task Definition field path> | ||
| value: <value> | ||
| ``` |
There was a problem hiding this comment.
What do you think of this? (inspired by https://datatracker.ietf.org/doc/html/rfc6902)
The following is an example valid `taskdef_overrides` field that can be applied to a manifest file:
``` yaml
taskdef_overrides:
- path: ContainerDefinitions[0].Cpu
value: 512
- path: ContainerDefinitions[0].Memory
value: 1024
Each rule is applied sequentially to the CloudFormation template. The resulting CloudFormation template becomes the target of the next rule. Evaluation continues until all rules are successfully applied or an error is encountered.
| value: <value> | ||
| ``` | ||
|
|
||
| ## Override Behaviors |
There was a problem hiding this comment.
Suggested change
| ## Override Behaviors | |
| ### Path evaluation | |
| The `path` field is a `'.'` character separated path to a target [Task Definition field under `Properties` in CloudFormation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-taskdefinition.html). | |
| - Copilot recursively inserts fields if they don't .... | |
| - To append a new member to a `list` field such as `Ulimits` you can use the special character `-`: `Ulimits[-]`. |
|
|
||
| - Use `-` as index to append a new member to a `list` field | ||
|
|
||
| - When applying an override rule, Copilot inserts or updates the fields along the path. More specifically, Copilot recursively inserts fields if they don't exist in the CloudFormation template. For example: if a rule has the path `A.B[-].C` (`B` and `C` don't exist), Copilot will insert the field `B` and `C`. A concrete example can be found [below](#add-ulimits-to-the-main-container). |
There was a problem hiding this comment.
What happens if the field exists? Can we say that it's going to replace it.
- If the target path specifies an member that already exists, that member's value is replaced.| * [Family](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-taskdefinition.html#cfn-ecs-taskdefinition-family) | ||
| * [ContainerDefinitions[<index>].Name](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ecs-taskdefinition-containerdefinitions.html#cfn-ecs-taskdefinition-containerdefinition-name) | ||
|
|
||
| ## Examples |
iamhopaul123
force-pushed
the
taskdef-doc
branch
from
b7ddf99 to
145aa77
Compare
efekarakus
approved these changes
Aug 26, 2021
|
|
||
| - Copilot recursively inserts fields if they don't exist in the CloudFormation template. For example: if a rule has the path `A.B[-].C` (`B` and `C` don't exist), Copilot will insert the field `B` and `C`. A concrete example can be found [below](#add-ulimits-to-the-main-container). | ||
|
|
||
| - If the target path specifies an member that already exists, that member's value is replaced. |
There was a problem hiding this comment.
nit:
Suggested change
| - If the target path specifies an member that already exists, that member's value is replaced. | |
| - If the target path specifies a member that already exists, that member's value is replaced. |
| @@ -240,5 +240,16 @@ Optional. The full config file path in your custom Fluent Bit image. | |||
|
|
|||
| <div class="separator"></div> | |||
|
|
|||
| <a id="taskdef_overrides" href="#taskdef_overrides" class="field">`taskdef_overrides`</a> <span class="type">Array</span> | |||
There was a problem hiding this comment.
nit: maybe
Suggested change
| <a id="taskdef_overrides" href="#taskdef_overrides" class="field">`taskdef_overrides`</a> <span class="type">Array</span> | |
| <a id="taskdef_overrides" href="#taskdef_overrides" class="field">`taskdef_overrides`</a> <span class="type">Array of Rules</span> |
Lou1415926
approved these changes
Aug 26, 2021
huanjani
added a commit
that referenced
this pull request
Sep 15, 2021
* chore: fix npm dependency alert (#2740) <!-- Provide summary of changes --> https://github.com/aws/copilot-cli/security/dependabot/cf-custom-resources/package-lock.json/path-parse/open <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: add SNS topic selector to `select` (#2663) <!-- Provide summary of changes --> Adds a "deployed SNS topic" selector to the `select` package and some data structures to accompany it. Uses the `deploy store` under the hood and relies on resource group tagging API, so it can only be used on one environment at a time. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: upgrade mergo and implement transformers to maintain existing behaviors (#2738) This PR is part of the second step (writing units test) to address #2492. It: 1. Upgrades `mergo` 2. Fixes code by implementing custom transformers so that all `applyEnv` tests that have been passing are still passing Next: 1. Change pointer struct to struct and remove `pStruct` transformers 2. Fix bugs (tests that have been failing and commented out in previous PRs (#2700 ~ #2734) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: Bump golang.org/x/mod from 0.4.2 to 0.5.0 (#2741) Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.4.2 to 0.5.0. - [Release notes](https://github.com/golang/mod/releases) - [Commits](golang/mod@v0.4.2...v0.5.0) --- updated-dependencies: - dependency-name: golang.org/x/mod dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: Bump github.com/aws/aws-sdk-go from 1.40.17 to 1.40.22 (#2742) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.17 to 1.40.22. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md) - [Commits](aws/aws-sdk-go@v1.40.17...v1.40.22) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * chore: add worker services to svc show (#2744) <!-- Provide summary of changes --> Continued from #2703. Refactor described [here](https://github.com/aws/copilot-cli/pull/2703/files#r683800143) to be implemented in a future PR. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(manifest): add taskdef_override and parsing func (#2743) <!-- Provide summary of changes --> Part of #2588. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * feat(cli): add --force for svc deploy (#2736) <!-- Provide summary of changes --> Add `--force` flag for `svc deploy` and `deploy`, allowing users to force deploy their ECS service. Fixes #2597. E2E test and doc will be added in an upcoming PR. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * refactor: embed common ecsServiceDescriber struct (#2748) <!-- Provide summary of changes --> As mentioned in #2744, refactors describer structs to remove redundancy. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(stack): apply manifest taskdef override rules to CFN template (#2750) <!-- Provide summary of changes --> Part of #2588 <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * feat: add worker services to `svc init` (#2752) <!-- Provide summary of changes --> Subsumes #2704. Also refactors SNS selector to a single prompt, where we only ask the customer to select topics that are deployed across all existing environments. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * fix(manifest): fix previously found `ApplyEnv` bugs (#2746) This PR This PR is part of the second step (writing units test) to address #2492. It: 1. Fix previously found `ApplyEnv` bugs by implementing transformers needed 2. Refactor `transform.go` Previous: #2738 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * feat: add --force flag, e2e test, and doc (#2749) <!-- Provide summary of changes --> <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: Bump github.com/AlecAivazis/survey/v2 from 2.2.16 to 2.3.0 (#2757) Bumps [github.com/AlecAivazis/survey/v2](https://github.com/AlecAivazis/survey) from 2.2.16 to 2.3.0. - [Release notes](https://github.com/AlecAivazis/survey/releases) - [Commits](AlecAivazis/survey@v2.2.16...v2.3.0) --- updated-dependencies: - dependency-name: github.com/AlecAivazis/survey/v2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: Bump github.com/onsi/gomega from 1.15.0 to 1.16.0 (#2759) Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.15.0 to 1.16.0. - [Release notes](https://github.com/onsi/gomega/releases) - [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md) - [Commits](onsi/gomega@v1.15.0...v1.16.0) --- updated-dependencies: - dependency-name: github.com/onsi/gomega dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * chore: Bump github.com/aws/aws-sdk-go from 1.40.22 to 1.40.27 (#2758) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.22 to 1.40.27. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md) - [Commits](aws/aws-sdk-go@v1.40.22...v1.40.27) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * docs: Add --yes flag to svc exec docs. (#2761) <!-- Provide summary of changes --> Adds the --yes flag to `copilot svc exec` docs. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> Fixes #2760 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(override): unmarshal/marshal CFN template and parse node from rule (#2756) This PR: 1. unmarshal/marshal CFN template to/from YAML nodes. 2. parse node from rule. 3. add a unit test for the exported function `CloudformationTemplate`. Part of #2588 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: group segments into a `pathSegment` struct (#2764) Addressing: #2756 (comment) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * build: upgrade e2e/Dockerfile go version to 1.16.6 from 1.13.15 (#2767) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(stack): add rule prefix and override integ test (#2762) <!-- Provide summary of changes --> Part of #2588. Work on the basis of #2756. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: implement the transformer for `FIFOOrBool` (#2768) `FIFOOrBool` is a composite field and need custom merge logic. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * fix(cli): don't log platform msg unless both redirected and non-RDWS (#2770) Previously, any non-App Runner service threw out the log statement about the platform field in the manifest. Now, this happens only when the platform has been redirected _and_ a non-App Runner service has been created. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(manifest): use COPILOT_QUEUE_URIS in worker svc manifest (#2772) <!-- Provide summary of changes --> <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: update help text to include Woker Service on init cmds (#2771) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: add metadata to queue resources (#2773) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * docs: add task def override (#2763) <!-- Provide summary of changes --> Part of #2588. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: remove fifo field from queue settings in the Worker Service manifest (#2775) ⚰️ By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: add progress tracker metadata to sns resources (#2774) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: update manifest env var to COPILOT_QUEUE_URI (#2779) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * fix: correctly inject environment variables for subscriptions. (#2778) <!-- Provide summary of changes --> <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: add worker service to svc deploy (#2765) <!-- Provide summary of changes --> Continued from #2705. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: always create the events queue for Worker Services (#2782) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(cli): display recommended actions on "copilot deploy" (#2783) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: Bump github.com/aws/aws-sdk-go from 1.40.27 to 1.40.32 (#2789) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.27 to 1.40.32. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md) - [Commits](aws/aws-sdk-go@v1.40.27...v1.40.32) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: Bump github.com/AlecAivazis/survey/v2 from 2.3.0 to 2.3.1 (#2788) Bumps [github.com/AlecAivazis/survey/v2](https://github.com/AlecAivazis/survey) from 2.3.0 to 2.3.1. - [Release notes](https://github.com/AlecAivazis/survey/releases) - [Commits](AlecAivazis/survey@v2.3.0...v2.3.1) --- updated-dependencies: - dependency-name: github.com/AlecAivazis/survey/v2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * chore(stack/rdws): fix add permissions and env var for sns (#2790) _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._ * fix(cli): command with env/svc flag should work without app flag (#2781) <!-- Provide summary of changes --> fixes #2776 <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: leverage interfaces to recommend deploy actions instead of switch statements (#2787) There are also a _bajillion_ tiny refactors along the way. The changes look big but it shouldn't be too bad mostly simplifying code or moving them. The main new logic can be found in this commit: efekarakus@6897454, the rest are refactors. _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._ * chore: format recommended actions slightly more inward (#2792) 🥺 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: add pointer to map to volume to take advantage of mergo's merge logic (#2793) In `mergo`, if the map is to pointer-to-struct, then it performs deep merge on the pointer. By changing `map[string]Volume` to `map[string]*Volume`, we can take advantage of this logic instead of implementing a custom transformer. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * docs: add Worker Services to docs site. (#2785) <!-- Provide summary of changes --> <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(manifest): test each individual transformers (#2795) This PR tests each individual transformer. In addition, it also includes a small refactoring on basic transformer. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * feat: support darwin-arm64 CLI binary (#2802) <!-- Provide summary of changes --> - Compile and publish darwin/arm64 copilot CLI binary. - This will allow copilot to run natively on newer macs rather than being emulated. I've tested the binary compilation but not sure if I can test the actual release build specs easily. Resolves #2745 <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: use forward slashes in file paths for go:embed (#2807) Fixes #2803. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: PR template to specify the correct license (#2801) <!-- Provide summary of changes --> This fixes the PR template to specify the Apache 2.0 License rather than leaving it the default "under the terms of your choice". <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: add Lambda fn to write BacklogPerTask CW metric (#2804) We need to write this metric for Worker Service autoscaling. Related #2796 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * fix: error out if the credential is invalid (#2816) Previously, we don't error out if the credential is invalid or we fail to get credentials because of time out. A result of this is that we will keep using the session with the **invalid/empty credential**, which eventually causes the CLI to hang during a call to `GetCallerIdentity()` https://github.com/aws/copilot-cli/blob/77983421537d9f34575655ab8cba309f30610f33/internal/pkg/aws/identity/identity.go#L40 The hanging issue is reported in #2686. An **invalid/empty credential** happens when: 1. The profile is not found, AND 2. The environment variables `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are NOT set. The profile is not found when: 1. The file `~/.aws/credentials` is not found 2. The file is found, but the profile is not defined After this fix, the output from `copilot app init` or `copilot init` in the case of an **invalid/empty credential** would be: ``` Welcome to the Copilot CLI! We're going to walk you through some questions to help you get set up with a containerized application on AWS. An application is a collection of containerized services that operate together. ✘ ask app init: get credentials of session: RequestCanceled: request context canceled caused by: context deadline exceeded ``` instead of hanging. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.. * chore(template): add cfn resources for worker svc autoscaling (#2813) ✅ Tested the template by deploying a worker service with autoscaling Related #2796 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.. * chore(manifest): refactor pointer struct to struct in manifest (#2798) This PR is a refactor that removes unnecessary pointers ™️ from manifest. "Necessary pointers" ™️ to the following structs are not removed: ##### <s>`ContainerHealthCheck`</s> and `PlatfromArgsOrString` for all workloads (e.g. https://github.com/aws/copilot-cli/blob/ff0a32e3d0d6821408ba6f05fcad6a0450f75f24/internal/pkg/manifest/lb_web_svc.go#L64) This is because of the way we marshal manifest object into YAML during `svc init` or `job init`. <s>##### `EFSConfigOrBool` A manifest input that has: ``` efs: <the rest of the manifest> ``` is deemed invalid, and we error out during the conversion from a manifest object to a template object. We cannot remove the pointer right now because otherwise we cannot differentiate a said invalid input, and a valid input that didn't specify `efs` at all. This validation should instead happens at an earlier stage, potentially from within a dedicated `manifest.validation` package. We can remove the pointer after we can move the validation to the said earlier stage.</s> ##### `SQSQueue ` For this particular manifest field, a manifest input that has: ``` topics: - name: topic1 queue: <the rest of the manifest> ``` is deemed valid. If the `queue` field is specified without any value, we create the queue for that topic with default configurations. Hence, there is indeed a need to differentiate an empty queue from an absent queue. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(cli): enable combination of flags and prompting for custom env resources (#2784) Previously, it was assumed that users would 1. import neither VPC nor subnets and use the guided experience, or 2. import both VPC and subnets. This enables them to import one without the other. Fixes #2777. Use cases with these changes: - import all custom resources: proceed as usual, including validation of two private subnets (okay to leave public subnets blank--log warning about load balancers needing two, but must have two private) - import no custom resources: prompt to select VPC and subnets - import just VPC: prompt to select subnets - import two private: prompt to select VPC and public subnets - import two public: prompt to select VPC and private subnets - import just one private: error - import just one public: warning, prompt to select VPC and private subnets By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: remove `applyEnv` unit tests and keep only those that are representative (#2806) Previously I've added a plethora of `ApplyEnv` unit tests to baseline our `ApplyEnv` bahaviours on each of the manifest, in attempt to upgrade mergo and fix manifest behaviors. Now that the upgrade is completed, these extensive tests can be removed, replaced with a few `ApplyEnv` tests for each of the representative field. This PR is dependent on #2798. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * fix(manifest): remove unnecessary `allowed_workers` field (#2821) There is no way today to limit an SNS topic subscription to be based on the tag of SQS queues (https://aws.amazon.com/premiumsupport/knowledge-center/sqs-sns-subscribe-cloudformation/). Instead, by default SNS will limit subscriptions to be only within the same AWS account. Therefore, the `allowed_workers` field is not needed. This is not a breaking change, it makes a required field now obsolete. _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License._ * fix: extend ARM platform changes through to deployment (#2814) These changes enable workload images on ARM machines to build as amd64 images. If the underlying docker engine detected is ARM architecture-based, Copilot builds with `--platform linux/amd64` so workloads deploy rather than going through the whole workflow only to fail to spin up. Related: #2636. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.. * refactor(manifest): remove pointer for Range (#2820) <!-- Provide summary of changes --> <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.. * chore(manifest): add manifest validation boilerplate (#2815) <!-- Provide summary of changes --> Part of #2818. This PR adds boilerplate for new manifest validation for LB svc. Also added validation for mutually exclusive fields. Will add for the other workload types, advanced type validation, and consolidate existing validations in following PRs <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.. * refactor: move exec/dockerfile* to new docker/dockerfile pkg (#2824) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License. * chore(manifest): add validate boilerplate for rest workload types (#2826) <!-- Provide summary of changes --> Part of #2818. After this PR, we should have boilerplate completed for every workload type. Then, we'll add more advanced basic types validation as well as start to consolidate validation elsewhere. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.. * docs(ja): periodic update (#2805) This PR updates the Japanese localized docs based on the changes in the English docs after the v1.9.0 tag(b5c78a6) to the latest mainline(b152aed). By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: update PR template to remove extra period (#2828) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License. * chore: remove unused code Co-authored-by: Penghao He <[email protected]> Co-authored-by: Austin Ely <[email protected]> Co-authored-by: Wanxian Yang <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: John Lucas <[email protected]> Co-authored-by: Efe Karakus <[email protected]> Co-authored-by: Brandon Wagner <[email protected]> Co-authored-by: Kaji <[email protected]>
huanjani
added a commit
that referenced
this pull request
Oct 12, 2021
…ogging logic (#2904) * chore: fix npm dependency alert (#2740) <!-- Provide summary of changes --> https://github.com/aws/copilot-cli/security/dependabot/cf-custom-resources/package-lock.json/path-parse/open <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: add SNS topic selector to `select` (#2663) <!-- Provide summary of changes --> Adds a "deployed SNS topic" selector to the `select` package and some data structures to accompany it. Uses the `deploy store` under the hood and relies on resource group tagging API, so it can only be used on one environment at a time. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: upgrade mergo and implement transformers to maintain existing behaviors (#2738) This PR is part of the second step (writing units test) to address #2492. It: 1. Upgrades `mergo` 2. Fixes code by implementing custom transformers so that all `applyEnv` tests that have been passing are still passing Next: 1. Change pointer struct to struct and remove `pStruct` transformers 2. Fix bugs (tests that have been failing and commented out in previous PRs (#2700 ~ #2734) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: Bump golang.org/x/mod from 0.4.2 to 0.5.0 (#2741) Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.4.2 to 0.5.0. - [Release notes](https://github.com/golang/mod/releases) - [Commits](golang/mod@v0.4.2...v0.5.0) --- updated-dependencies: - dependency-name: golang.org/x/mod dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: Bump github.com/aws/aws-sdk-go from 1.40.17 to 1.40.22 (#2742) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.17 to 1.40.22. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md) - [Commits](aws/aws-sdk-go@v1.40.17...v1.40.22) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * chore: add worker services to svc show (#2744) <!-- Provide summary of changes --> Continued from #2703. Refactor described [here](https://github.com/aws/copilot-cli/pull/2703/files#r683800143) to be implemented in a future PR. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(manifest): add taskdef_override and parsing func (#2743) <!-- Provide summary of changes --> Part of #2588. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * feat(cli): add --force for svc deploy (#2736) <!-- Provide summary of changes --> Add `--force` flag for `svc deploy` and `deploy`, allowing users to force deploy their ECS service. Fixes #2597. E2E test and doc will be added in an upcoming PR. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * refactor: embed common ecsServiceDescriber struct (#2748) <!-- Provide summary of changes --> As mentioned in #2744, refactors describer structs to remove redundancy. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(stack): apply manifest taskdef override rules to CFN template (#2750) <!-- Provide summary of changes --> Part of #2588 <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * feat: add worker services to `svc init` (#2752) <!-- Provide summary of changes --> Subsumes #2704. Also refactors SNS selector to a single prompt, where we only ask the customer to select topics that are deployed across all existing environments. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * fix(manifest): fix previously found `ApplyEnv` bugs (#2746) This PR This PR is part of the second step (writing units test) to address #2492. It: 1. Fix previously found `ApplyEnv` bugs by implementing transformers needed 2. Refactor `transform.go` Previous: #2738 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * feat: add --force flag, e2e test, and doc (#2749) <!-- Provide summary of changes --> <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: Bump github.com/AlecAivazis/survey/v2 from 2.2.16 to 2.3.0 (#2757) Bumps [github.com/AlecAivazis/survey/v2](https://github.com/AlecAivazis/survey) from 2.2.16 to 2.3.0. - [Release notes](https://github.com/AlecAivazis/survey/releases) - [Commits](AlecAivazis/survey@v2.2.16...v2.3.0) --- updated-dependencies: - dependency-name: github.com/AlecAivazis/survey/v2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: Bump github.com/onsi/gomega from 1.15.0 to 1.16.0 (#2759) Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.15.0 to 1.16.0. - [Release notes](https://github.com/onsi/gomega/releases) - [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md) - [Commits](onsi/gomega@v1.15.0...v1.16.0) --- updated-dependencies: - dependency-name: github.com/onsi/gomega dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * chore: Bump github.com/aws/aws-sdk-go from 1.40.22 to 1.40.27 (#2758) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.22 to 1.40.27. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md) - [Commits](aws/aws-sdk-go@v1.40.22...v1.40.27) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * docs: Add --yes flag to svc exec docs. (#2761) <!-- Provide summary of changes --> Adds the --yes flag to `copilot svc exec` docs. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> Fixes #2760 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(override): unmarshal/marshal CFN template and parse node from rule (#2756) This PR: 1. unmarshal/marshal CFN template to/from YAML nodes. 2. parse node from rule. 3. add a unit test for the exported function `CloudformationTemplate`. Part of #2588 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: group segments into a `pathSegment` struct (#2764) Addressing: #2756 (comment) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * build: upgrade e2e/Dockerfile go version to 1.16.6 from 1.13.15 (#2767) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(stack): add rule prefix and override integ test (#2762) <!-- Provide summary of changes --> Part of #2588. Work on the basis of #2756. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: implement the transformer for `FIFOOrBool` (#2768) `FIFOOrBool` is a composite field and need custom merge logic. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * fix(cli): don't log platform msg unless both redirected and non-RDWS (#2770) Previously, any non-App Runner service threw out the log statement about the platform field in the manifest. Now, this happens only when the platform has been redirected _and_ a non-App Runner service has been created. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(manifest): use COPILOT_QUEUE_URIS in worker svc manifest (#2772) <!-- Provide summary of changes --> <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: update help text to include Woker Service on init cmds (#2771) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: add metadata to queue resources (#2773) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * docs: add task def override (#2763) <!-- Provide summary of changes --> Part of #2588. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: remove fifo field from queue settings in the Worker Service manifest (#2775) ⚰️ By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: add progress tracker metadata to sns resources (#2774) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: update manifest env var to COPILOT_QUEUE_URI (#2779) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * fix: correctly inject environment variables for subscriptions. (#2778) <!-- Provide summary of changes --> <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: add worker service to svc deploy (#2765) <!-- Provide summary of changes --> Continued from #2705. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: always create the events queue for Worker Services (#2782) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(cli): display recommended actions on "copilot deploy" (#2783) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: Bump github.com/aws/aws-sdk-go from 1.40.27 to 1.40.32 (#2789) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.27 to 1.40.32. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md) - [Commits](aws/aws-sdk-go@v1.40.27...v1.40.32) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: Bump github.com/AlecAivazis/survey/v2 from 2.3.0 to 2.3.1 (#2788) Bumps [github.com/AlecAivazis/survey/v2](https://github.com/AlecAivazis/survey) from 2.3.0 to 2.3.1. - [Release notes](https://github.com/AlecAivazis/survey/releases) - [Commits](AlecAivazis/survey@v2.3.0...v2.3.1) --- updated-dependencies: - dependency-name: github.com/AlecAivazis/survey/v2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * chore(stack/rdws): fix add permissions and env var for sns (#2790) _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._ * fix(cli): command with env/svc flag should work without app flag (#2781) <!-- Provide summary of changes --> fixes #2776 <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: leverage interfaces to recommend deploy actions instead of switch statements (#2787) There are also a _bajillion_ tiny refactors along the way. The changes look big but it shouldn't be too bad mostly simplifying code or moving them. The main new logic can be found in this commit: efekarakus@6897454, the rest are refactors. _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._ * chore: format recommended actions slightly more inward (#2792) 🥺 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: add pointer to map to volume to take advantage of mergo's merge logic (#2793) In `mergo`, if the map is to pointer-to-struct, then it performs deep merge on the pointer. By changing `map[string]Volume` to `map[string]*Volume`, we can take advantage of this logic instead of implementing a custom transformer. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * docs: add Worker Services to docs site. (#2785) <!-- Provide summary of changes --> <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(manifest): test each individual transformers (#2795) This PR tests each individual transformer. In addition, it also includes a small refactoring on basic transformer. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * feat: support darwin-arm64 CLI binary (#2802) <!-- Provide summary of changes --> - Compile and publish darwin/arm64 copilot CLI binary. - This will allow copilot to run natively on newer macs rather than being emulated. I've tested the binary compilation but not sure if I can test the actual release build specs easily. Resolves #2745 <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: use forward slashes in file paths for go:embed (#2807) Fixes #2803. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: PR template to specify the correct license (#2801) <!-- Provide summary of changes --> This fixes the PR template to specify the Apache 2.0 License rather than leaving it the default "under the terms of your choice". <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: add Lambda fn to write BacklogPerTask CW metric (#2804) We need to write this metric for Worker Service autoscaling. Related #2796 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * fix: error out if the credential is invalid (#2816) Previously, we don't error out if the credential is invalid or we fail to get credentials because of time out. A result of this is that we will keep using the session with the **invalid/empty credential**, which eventually causes the CLI to hang during a call to `GetCallerIdentity()` https://github.com/aws/copilot-cli/blob/77983421537d9f34575655ab8cba309f30610f33/internal/pkg/aws/identity/identity.go#L40 The hanging issue is reported in #2686. An **invalid/empty credential** happens when: 1. The profile is not found, AND 2. The environment variables `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are NOT set. The profile is not found when: 1. The file `~/.aws/credentials` is not found 2. The file is found, but the profile is not defined After this fix, the output from `copilot app init` or `copilot init` in the case of an **invalid/empty credential** would be: ``` Welcome to the Copilot CLI! We're going to walk you through some questions to help you get set up with a containerized application on AWS. An application is a collection of containerized services that operate together. ✘ ask app init: get credentials of session: RequestCanceled: request context canceled caused by: context deadline exceeded ``` instead of hanging. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.. * chore(template): add cfn resources for worker svc autoscaling (#2813) ✅ Tested the template by deploying a worker service with autoscaling Related #2796 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.. * chore(manifest): refactor pointer struct to struct in manifest (#2798) This PR is a refactor that removes unnecessary pointers ™️ from manifest. "Necessary pointers" ™️ to the following structs are not removed: ##### <s>`ContainerHealthCheck`</s> and `PlatfromArgsOrString` for all workloads (e.g. https://github.com/aws/copilot-cli/blob/ff0a32e3d0d6821408ba6f05fcad6a0450f75f24/internal/pkg/manifest/lb_web_svc.go#L64) This is because of the way we marshal manifest object into YAML during `svc init` or `job init`. <s>##### `EFSConfigOrBool` A manifest input that has: ``` efs: <the rest of the manifest> ``` is deemed invalid, and we error out during the conversion from a manifest object to a template object. We cannot remove the pointer right now because otherwise we cannot differentiate a said invalid input, and a valid input that didn't specify `efs` at all. This validation should instead happens at an earlier stage, potentially from within a dedicated `manifest.validation` package. We can remove the pointer after we can move the validation to the said earlier stage.</s> ##### `SQSQueue ` For this particular manifest field, a manifest input that has: ``` topics: - name: topic1 queue: <the rest of the manifest> ``` is deemed valid. If the `queue` field is specified without any value, we create the queue for that topic with default configurations. Hence, there is indeed a need to differentiate an empty queue from an absent queue. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(cli): enable combination of flags and prompting for custom env resources (#2784) Previously, it was assumed that users would 1. import neither VPC nor subnets and use the guided experience, or 2. import both VPC and subnets. This enables them to import one without the other. Fixes #2777. Use cases with these changes: - import all custom resources: proceed as usual, including validation of two private subnets (okay to leave public subnets blank--log warning about load balancers needing two, but must have two private) - import no custom resources: prompt to select VPC and subnets - import just VPC: prompt to select subnets - import two private: prompt to select VPC and public subnets - import two public: prompt to select VPC and private subnets - import just one private: error - import just one public: warning, prompt to select VPC and private subnets By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: remove `applyEnv` unit tests and keep only those that are representative (#2806) Previously I've added a plethora of `ApplyEnv` unit tests to baseline our `ApplyEnv` bahaviours on each of the manifest, in attempt to upgrade mergo and fix manifest behaviors. Now that the upgrade is completed, these extensive tests can be removed, replaced with a few `ApplyEnv` tests for each of the representative field. This PR is dependent on #2798. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * fix(manifest): remove unnecessary `allowed_workers` field (#2821) There is no way today to limit an SNS topic subscription to be based on the tag of SQS queues (https://aws.amazon.com/premiumsupport/knowledge-center/sqs-sns-subscribe-cloudformation/). Instead, by default SNS will limit subscriptions to be only within the same AWS account. Therefore, the `allowed_workers` field is not needed. This is not a breaking change, it makes a required field now obsolete. _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License._ * fix: extend ARM platform changes through to deployment (#2814) These changes enable workload images on ARM machines to build as amd64 images. If the underlying docker engine detected is ARM architecture-based, Copilot builds with `--platform linux/amd64` so workloads deploy rather than going through the whole workflow only to fail to spin up. Related: #2636. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.. * refactor(manifest): remove pointer for Range (#2820) <!-- Provide summary of changes --> <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.. * chore(manifest): add manifest validation boilerplate (#2815) <!-- Provide summary of changes --> Part of #2818. This PR adds boilerplate for new manifest validation for LB svc. Also added validation for mutually exclusive fields. Will add for the other workload types, advanced type validation, and consolidate existing validations in following PRs <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.. * refactor: move exec/dockerfile* to new docker/dockerfile pkg (#2824) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License. * chore(manifest): add validate boilerplate for rest workload types (#2826) <!-- Provide summary of changes --> Part of #2818. After this PR, we should have boilerplate completed for every workload type. Then, we'll add more advanced basic types validation as well as start to consolidate validation elsewhere. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.. * docs(ja): periodic update (#2805) This PR updates the Japanese localized docs based on the changes in the English docs after the v1.9.0 tag(b5c78a6) to the latest mainline(b152aed). By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: update PR template to remove extra period (#2828) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License. * chore: remove unused code * fix: narrow acceptible OS and Arch options because task run flags go straight to CFN * fix: change from array of RuntimePlatform objects to one object * chore: log platform manifest msg for all non-default, not just redirected * chore: change job and svc logging * fix: add linux as valid OS * fix: tweak unit test * fix: for real this time * fix: change platform for CFN Windows OS formats * chore: disable exec for windows tasks * chore: add enableExec to unit test * chore: add example of task run * chore: remove repetitive code * chore: rename var * chore: change log statements, validate full platform not os/arch separately Co-authored-by: Penghao He <[email protected]> Co-authored-by: Austin Ely <[email protected]> Co-authored-by: Wanxian Yang <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: John Lucas <[email protected]> Co-authored-by: Efe Karakus <[email protected]> Co-authored-by: Brandon Wagner <[email protected]> Co-authored-by: Kaji <[email protected]>
huanjani
added a commit
that referenced
this pull request
Oct 15, 2021
* chore: fix npm dependency alert (#2740) <!-- Provide summary of changes --> https://github.com/aws/copilot-cli/security/dependabot/cf-custom-resources/package-lock.json/path-parse/open <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: add SNS topic selector to `select` (#2663) <!-- Provide summary of changes --> Adds a "deployed SNS topic" selector to the `select` package and some data structures to accompany it. Uses the `deploy store` under the hood and relies on resource group tagging API, so it can only be used on one environment at a time. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: upgrade mergo and implement transformers to maintain existing behaviors (#2738) This PR is part of the second step (writing units test) to address #2492. It: 1. Upgrades `mergo` 2. Fixes code by implementing custom transformers so that all `applyEnv` tests that have been passing are still passing Next: 1. Change pointer struct to struct and remove `pStruct` transformers 2. Fix bugs (tests that have been failing and commented out in previous PRs (#2700 ~ #2734) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: Bump golang.org/x/mod from 0.4.2 to 0.5.0 (#2741) Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.4.2 to 0.5.0. - [Release notes](https://github.com/golang/mod/releases) - [Commits](golang/mod@v0.4.2...v0.5.0) --- updated-dependencies: - dependency-name: golang.org/x/mod dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: Bump github.com/aws/aws-sdk-go from 1.40.17 to 1.40.22 (#2742) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.17 to 1.40.22. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md) - [Commits](aws/aws-sdk-go@v1.40.17...v1.40.22) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * chore: add worker services to svc show (#2744) <!-- Provide summary of changes --> Continued from #2703. Refactor described [here](https://github.com/aws/copilot-cli/pull/2703/files#r683800143) to be implemented in a future PR. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(manifest): add taskdef_override and parsing func (#2743) <!-- Provide summary of changes --> Part of #2588. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * feat(cli): add --force for svc deploy (#2736) <!-- Provide summary of changes --> Add `--force` flag for `svc deploy` and `deploy`, allowing users to force deploy their ECS service. Fixes #2597. E2E test and doc will be added in an upcoming PR. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * refactor: embed common ecsServiceDescriber struct (#2748) <!-- Provide summary of changes --> As mentioned in #2744, refactors describer structs to remove redundancy. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(stack): apply manifest taskdef override rules to CFN template (#2750) <!-- Provide summary of changes --> Part of #2588 <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * feat: add worker services to `svc init` (#2752) <!-- Provide summary of changes --> Subsumes #2704. Also refactors SNS selector to a single prompt, where we only ask the customer to select topics that are deployed across all existing environments. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * fix(manifest): fix previously found `ApplyEnv` bugs (#2746) This PR This PR is part of the second step (writing units test) to address #2492. It: 1. Fix previously found `ApplyEnv` bugs by implementing transformers needed 2. Refactor `transform.go` Previous: #2738 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * feat: add --force flag, e2e test, and doc (#2749) <!-- Provide summary of changes --> <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: Bump github.com/AlecAivazis/survey/v2 from 2.2.16 to 2.3.0 (#2757) Bumps [github.com/AlecAivazis/survey/v2](https://github.com/AlecAivazis/survey) from 2.2.16 to 2.3.0. - [Release notes](https://github.com/AlecAivazis/survey/releases) - [Commits](AlecAivazis/survey@v2.2.16...v2.3.0) --- updated-dependencies: - dependency-name: github.com/AlecAivazis/survey/v2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: Bump github.com/onsi/gomega from 1.15.0 to 1.16.0 (#2759) Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.15.0 to 1.16.0. - [Release notes](https://github.com/onsi/gomega/releases) - [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md) - [Commits](onsi/gomega@v1.15.0...v1.16.0) --- updated-dependencies: - dependency-name: github.com/onsi/gomega dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * chore: Bump github.com/aws/aws-sdk-go from 1.40.22 to 1.40.27 (#2758) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.22 to 1.40.27. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md) - [Commits](aws/aws-sdk-go@v1.40.22...v1.40.27) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * docs: Add --yes flag to svc exec docs. (#2761) <!-- Provide summary of changes --> Adds the --yes flag to `copilot svc exec` docs. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> Fixes #2760 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(override): unmarshal/marshal CFN template and parse node from rule (#2756) This PR: 1. unmarshal/marshal CFN template to/from YAML nodes. 2. parse node from rule. 3. add a unit test for the exported function `CloudformationTemplate`. Part of #2588 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: group segments into a `pathSegment` struct (#2764) Addressing: #2756 (comment) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * build: upgrade e2e/Dockerfile go version to 1.16.6 from 1.13.15 (#2767) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(stack): add rule prefix and override integ test (#2762) <!-- Provide summary of changes --> Part of #2588. Work on the basis of #2756. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: implement the transformer for `FIFOOrBool` (#2768) `FIFOOrBool` is a composite field and need custom merge logic. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * fix(cli): don't log platform msg unless both redirected and non-RDWS (#2770) Previously, any non-App Runner service threw out the log statement about the platform field in the manifest. Now, this happens only when the platform has been redirected _and_ a non-App Runner service has been created. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(manifest): use COPILOT_QUEUE_URIS in worker svc manifest (#2772) <!-- Provide summary of changes --> <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: update help text to include Woker Service on init cmds (#2771) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: add metadata to queue resources (#2773) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * docs: add task def override (#2763) <!-- Provide summary of changes --> Part of #2588. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: remove fifo field from queue settings in the Worker Service manifest (#2775) ⚰️ By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: add progress tracker metadata to sns resources (#2774) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: update manifest env var to COPILOT_QUEUE_URI (#2779) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * fix: correctly inject environment variables for subscriptions. (#2778) <!-- Provide summary of changes --> <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: add worker service to svc deploy (#2765) <!-- Provide summary of changes --> Continued from #2705. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: always create the events queue for Worker Services (#2782) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(cli): display recommended actions on "copilot deploy" (#2783) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: Bump github.com/aws/aws-sdk-go from 1.40.27 to 1.40.32 (#2789) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.27 to 1.40.32. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md) - [Commits](aws/aws-sdk-go@v1.40.27...v1.40.32) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: Bump github.com/AlecAivazis/survey/v2 from 2.3.0 to 2.3.1 (#2788) Bumps [github.com/AlecAivazis/survey/v2](https://github.com/AlecAivazis/survey) from 2.3.0 to 2.3.1. - [Release notes](https://github.com/AlecAivazis/survey/releases) - [Commits](AlecAivazis/survey@v2.3.0...v2.3.1) --- updated-dependencies: - dependency-name: github.com/AlecAivazis/survey/v2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * chore(stack/rdws): fix add permissions and env var for sns (#2790) _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._ * fix(cli): command with env/svc flag should work without app flag (#2781) <!-- Provide summary of changes --> fixes #2776 <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: leverage interfaces to recommend deploy actions instead of switch statements (#2787) There are also a _bajillion_ tiny refactors along the way. The changes look big but it shouldn't be too bad mostly simplifying code or moving them. The main new logic can be found in this commit: efekarakus@6897454, the rest are refactors. _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._ * chore: format recommended actions slightly more inward (#2792) 🥺 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: add pointer to map to volume to take advantage of mergo's merge logic (#2793) In `mergo`, if the map is to pointer-to-struct, then it performs deep merge on the pointer. By changing `map[string]Volume` to `map[string]*Volume`, we can take advantage of this logic instead of implementing a custom transformer. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * docs: add Worker Services to docs site. (#2785) <!-- Provide summary of changes --> <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(manifest): test each individual transformers (#2795) This PR tests each individual transformer. In addition, it also includes a small refactoring on basic transformer. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * feat: support darwin-arm64 CLI binary (#2802) <!-- Provide summary of changes --> - Compile and publish darwin/arm64 copilot CLI binary. - This will allow copilot to run natively on newer macs rather than being emulated. I've tested the binary compilation but not sure if I can test the actual release build specs easily. Resolves #2745 <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: use forward slashes in file paths for go:embed (#2807) Fixes #2803. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: PR template to specify the correct license (#2801) <!-- Provide summary of changes --> This fixes the PR template to specify the Apache 2.0 License rather than leaving it the default "under the terms of your choice". <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: add Lambda fn to write BacklogPerTask CW metric (#2804) We need to write this metric for Worker Service autoscaling. Related #2796 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * fix: error out if the credential is invalid (#2816) Previously, we don't error out if the credential is invalid or we fail to get credentials because of time out. A result of this is that we will keep using the session with the **invalid/empty credential**, which eventually causes the CLI to hang during a call to `GetCallerIdentity()` https://github.com/aws/copilot-cli/blob/77983421537d9f34575655ab8cba309f30610f33/internal/pkg/aws/identity/identity.go#L40 The hanging issue is reported in #2686. An **invalid/empty credential** happens when: 1. The profile is not found, AND 2. The environment variables `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are NOT set. The profile is not found when: 1. The file `~/.aws/credentials` is not found 2. The file is found, but the profile is not defined After this fix, the output from `copilot app init` or `copilot init` in the case of an **invalid/empty credential** would be: ``` Welcome to the Copilot CLI! We're going to walk you through some questions to help you get set up with a containerized application on AWS. An application is a collection of containerized services that operate together. ✘ ask app init: get credentials of session: RequestCanceled: request context canceled caused by: context deadline exceeded ``` instead of hanging. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.. * chore(template): add cfn resources for worker svc autoscaling (#2813) ✅ Tested the template by deploying a worker service with autoscaling Related #2796 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.. * chore(manifest): refactor pointer struct to struct in manifest (#2798) This PR is a refactor that removes unnecessary pointers ™️ from manifest. "Necessary pointers" ™️ to the following structs are not removed: ##### <s>`ContainerHealthCheck`</s> and `PlatfromArgsOrString` for all workloads (e.g. https://github.com/aws/copilot-cli/blob/ff0a32e3d0d6821408ba6f05fcad6a0450f75f24/internal/pkg/manifest/lb_web_svc.go#L64) This is because of the way we marshal manifest object into YAML during `svc init` or `job init`. <s>##### `EFSConfigOrBool` A manifest input that has: ``` efs: <the rest of the manifest> ``` is deemed invalid, and we error out during the conversion from a manifest object to a template object. We cannot remove the pointer right now because otherwise we cannot differentiate a said invalid input, and a valid input that didn't specify `efs` at all. This validation should instead happens at an earlier stage, potentially from within a dedicated `manifest.validation` package. We can remove the pointer after we can move the validation to the said earlier stage.</s> ##### `SQSQueue ` For this particular manifest field, a manifest input that has: ``` topics: - name: topic1 queue: <the rest of the manifest> ``` is deemed valid. If the `queue` field is specified without any value, we create the queue for that topic with default configurations. Hence, there is indeed a need to differentiate an empty queue from an absent queue. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore(cli): enable combination of flags and prompting for custom env resources (#2784) Previously, it was assumed that users would 1. import neither VPC nor subnets and use the guided experience, or 2. import both VPC and subnets. This enables them to import one without the other. Fixes #2777. Use cases with these changes: - import all custom resources: proceed as usual, including validation of two private subnets (okay to leave public subnets blank--log warning about load balancers needing two, but must have two private) - import no custom resources: prompt to select VPC and subnets - import just VPC: prompt to select subnets - import two private: prompt to select VPC and public subnets - import two public: prompt to select VPC and private subnets - import just one private: error - import just one public: warning, prompt to select VPC and private subnets By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: remove `applyEnv` unit tests and keep only those that are representative (#2806) Previously I've added a plethora of `ApplyEnv` unit tests to baseline our `ApplyEnv` bahaviours on each of the manifest, in attempt to upgrade mergo and fix manifest behaviors. Now that the upgrade is completed, these extensive tests can be removed, replaced with a few `ApplyEnv` tests for each of the representative field. This PR is dependent on #2798. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * fix(manifest): remove unnecessary `allowed_workers` field (#2821) There is no way today to limit an SNS topic subscription to be based on the tag of SQS queues (https://aws.amazon.com/premiumsupport/knowledge-center/sqs-sns-subscribe-cloudformation/). Instead, by default SNS will limit subscriptions to be only within the same AWS account. Therefore, the `allowed_workers` field is not needed. This is not a breaking change, it makes a required field now obsolete. _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License._ * fix: extend ARM platform changes through to deployment (#2814) These changes enable workload images on ARM machines to build as amd64 images. If the underlying docker engine detected is ARM architecture-based, Copilot builds with `--platform linux/amd64` so workloads deploy rather than going through the whole workflow only to fail to spin up. Related: #2636. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.. * refactor(manifest): remove pointer for Range (#2820) <!-- Provide summary of changes --> <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.. * chore(manifest): add manifest validation boilerplate (#2815) <!-- Provide summary of changes --> Part of #2818. This PR adds boilerplate for new manifest validation for LB svc. Also added validation for mutually exclusive fields. Will add for the other workload types, advanced type validation, and consolidate existing validations in following PRs <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.. * refactor: move exec/dockerfile* to new docker/dockerfile pkg (#2824) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License. * chore(manifest): add validate boilerplate for rest workload types (#2826) <!-- Provide summary of changes --> Part of #2818. After this PR, we should have boilerplate completed for every workload type. Then, we'll add more advanced basic types validation as well as start to consolidate validation elsewhere. <!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" --> By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.. * docs(ja): periodic update (#2805) This PR updates the Japanese localized docs based on the changes in the English docs after the v1.9.0 tag(b5c78a6) to the latest mainline(b152aed). By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. * chore: update PR template to remove extra period (#2828) By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License. * chore: remove unused code * chore: add platformArgs documentation and fix casing * chore: tweak doc wording * chore: lowercase short platform in case user changes to anything w/ uc Co-authored-by: Penghao He <[email protected]> Co-authored-by: Austin Ely <[email protected]> Co-authored-by: Wanxian Yang <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: John Lucas <[email protected]> Co-authored-by: Efe Karakus <[email protected]> Co-authored-by: Brandon Wagner <[email protected]> Co-authored-by: Kaji <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Part of #2588.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.