Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: add Task Definition Overrides #2588

Closed
7 of 8 tasks
iamhopaul123 opened this issue Jul 9, 2021 · 3 comments
Closed
7 of 8 tasks

proposal: add Task Definition Overrides #2588

iamhopaul123 opened this issue Jul 9, 2021 · 3 comments
Labels
type/design Issues that are design proposals.

Comments

@iamhopaul123
Copy link
Contributor

iamhopaul123 commented Jul 9, 2021

Background

As more customers are using Copilot, our current manifest might not meet all their needs. For example in #2101, customers want http.credential to be configurable so as to support private repositories. Also as stated in one of our top issues with the most thumb-ups #948, it would be very helpful if they could overwrite arbitrary fields in their task definitions.

Example Related Issues

UX Design

For specific fields to override, please refer to https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_TaskDefinition.html

taskdef_override:
  - path: ContainerDefinitions[0].Ulimits.HardLimit
    value: 1024
  - ContainerDefinitions[0].Ulimits.Name: some limit
  - ContainerDefinitions[0].Ulimits.SoftLimit: 512
  - RequiresCompatibilities: [FARGATE]

Restriction

Users will not be allowed to modify the following fields in the task definition

  • Family
  • ContainerDefinition.Name (our main container name)

Override Behaviors

  • For single-value options like bool, string, or int, the new value replaces the old value.
  • For multi-value options like list or map, Copilot concatenates both sets of values.

Milestones

  • new override package (subject to future changes), which
    • validate the override rule
    • translates the override rule
    • applies override rules to the unmarshalled CFN template
    • unmarshals and marshals the CFN template correctly
  • read the override section in the manifest package
  • apply the template override in stack pkg
  • add docs and integ tests
@iamhopaul123 iamhopaul123 added the type/design Issues that are design proposals. label Jul 9, 2021
@iamhopaul123 iamhopaul123 pinned this issue Jul 9, 2021
@efekarakus efekarakus changed the title Proposal: Task Definition Override proposal: add Task Definition Overrides Jul 9, 2021
@buddhike
Copy link
Contributor

Hey @iamhopaul123, Thanks for the awesome summary of this feature 🙏🏾. Do you know if overrides are going to be available for sidecars as well 😊?

@iamhopaul123
Copy link
Contributor Author

Hello @BuddySpike sure it will be!

mergify bot pushed a commit that referenced this issue Jul 15, 2021
<!-- Provide summary of changes -->
Create `override` pkg and add some stub functions. Part of #2588.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
mergify bot pushed a commit that referenced this issue Aug 5, 2021
<!-- Provide summary of changes -->
Part of #2588
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
mergify bot pushed a commit that referenced this issue Aug 9, 2021
<!-- Provide summary of changes -->
Part of #2588.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
mergify bot pushed a commit that referenced this issue Aug 17, 2021
<!-- Provide summary of changes -->
Part of #2588.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
mergify bot pushed a commit that referenced this issue Aug 19, 2021
…2750)

<!-- Provide summary of changes -->
Part of #2588
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
mergify bot pushed a commit that referenced this issue Aug 24, 2021
…ule (#2756)

This PR:
1. unmarshal/marshal CFN template to/from YAML nodes.
2. parse node from rule.
3. add a unit test for the exported function `CloudformationTemplate`.

Part of #2588

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
mergify bot pushed a commit that referenced this issue Aug 25, 2021
<!-- Provide summary of changes -->
Part of #2588. Work on the basis of #2756.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
mergify bot pushed a commit that referenced this issue Aug 26, 2021
<!-- Provide summary of changes -->
Part of #2588.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
@efekarakus
Copy link
Contributor

The taskdef_overrides feature is now released! https://github.com/aws/copilot-cli/releases/tag/v1.10.0 🥳

@efekarakus efekarakus unpinned this issue Aug 31, 2021
huanjani added a commit that referenced this issue Sep 15, 2021
* chore: fix npm dependency alert (#2740)

<!-- Provide summary of changes -->
https://github.com/aws/copilot-cli/security/dependabot/cf-custom-resources/package-lock.json/path-parse/open
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: add SNS topic selector to `select` (#2663)

<!-- Provide summary of changes -->
Adds a "deployed SNS topic" selector to the `select` package and some data structures to accompany it. Uses the `deploy store` under the hood and relies on resource group tagging API, so it can only be used on one environment at a time. 

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: upgrade mergo and implement transformers to maintain existing behaviors (#2738)

This PR is part of the second step (writing units test) to address #2492. It:
1. Upgrades `mergo`
2. Fixes code by implementing custom transformers so that all `applyEnv` tests that have been passing are still passing

Next: 
1. Change pointer struct to struct and remove `pStruct` transformers
2. Fix bugs (tests that have been failing and commented out in previous PRs (#2700 ~ #2734)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: Bump golang.org/x/mod from 0.4.2 to 0.5.0 (#2741)

Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.4.2 to 0.5.0.
- [Release notes](https://github.com/golang/mod/releases)
- [Commits](golang/mod@v0.4.2...v0.5.0)

---
updated-dependencies:
- dependency-name: golang.org/x/mod
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump github.com/aws/aws-sdk-go from 1.40.17 to 1.40.22 (#2742)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.17 to 1.40.22.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](aws/aws-sdk-go@v1.40.17...v1.40.22)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* chore: add worker services to svc show (#2744)

<!-- Provide summary of changes -->
Continued from #2703. 

Refactor described [here](https://github.com/aws/copilot-cli/pull/2703/files#r683800143) to be implemented in a future PR.

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(manifest): add taskdef_override and parsing func (#2743)

<!-- Provide summary of changes -->
Part of #2588.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* feat(cli): add --force for svc deploy (#2736)

<!-- Provide summary of changes -->
Add `--force` flag for `svc deploy` and `deploy`, allowing users to force deploy their ECS service. Fixes #2597. E2E test and doc will be added in an upcoming PR.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* refactor: embed common ecsServiceDescriber struct (#2748)

<!-- Provide summary of changes -->
As mentioned in #2744, refactors describer structs to remove redundancy.

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(stack): apply manifest taskdef override rules to CFN template (#2750)

<!-- Provide summary of changes -->
Part of #2588
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* feat: add worker services to `svc init` (#2752)

<!-- Provide summary of changes -->
Subsumes #2704. 

Also refactors SNS selector to a single prompt, where we only ask the customer to select topics that are deployed across all existing environments.

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* fix(manifest): fix previously found `ApplyEnv` bugs (#2746)

This PR This PR is part of the second step (writing units test) to address #2492. It:
1. Fix previously found `ApplyEnv` bugs by implementing transformers needed
2. Refactor `transform.go`

Previous: #2738 
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* feat: add --force flag, e2e test, and doc (#2749)

<!-- Provide summary of changes -->
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: Bump github.com/AlecAivazis/survey/v2 from 2.2.16 to 2.3.0 (#2757)

Bumps [github.com/AlecAivazis/survey/v2](https://github.com/AlecAivazis/survey) from 2.2.16 to 2.3.0.
- [Release notes](https://github.com/AlecAivazis/survey/releases)
- [Commits](AlecAivazis/survey@v2.2.16...v2.3.0)

---
updated-dependencies:
- dependency-name: github.com/AlecAivazis/survey/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump github.com/onsi/gomega from 1.15.0 to 1.16.0 (#2759)

Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.15.0 to 1.16.0.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](onsi/gomega@v1.15.0...v1.16.0)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* chore: Bump github.com/aws/aws-sdk-go from 1.40.22 to 1.40.27 (#2758)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.22 to 1.40.27.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](aws/aws-sdk-go@v1.40.22...v1.40.27)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* docs: Add --yes flag to svc exec docs. (#2761)

<!-- Provide summary of changes -->
Adds the --yes flag to `copilot svc exec` docs.

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->
Fixes #2760

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(override): unmarshal/marshal CFN template and parse node from rule (#2756)

This PR:
1. unmarshal/marshal CFN template to/from YAML nodes.
2. parse node from rule.
3. add a unit test for the exported function `CloudformationTemplate`.

Part of #2588

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: group segments into a `pathSegment` struct (#2764)

Addressing: #2756 (comment)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* build: upgrade e2e/Dockerfile go version to 1.16.6 from 1.13.15 (#2767)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(stack): add rule prefix and override integ test (#2762)

<!-- Provide summary of changes -->
Part of #2588. Work on the basis of #2756.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: implement the transformer for `FIFOOrBool` (#2768)

`FIFOOrBool` is a composite field and need custom merge logic.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* fix(cli): don't log platform msg unless both redirected and non-RDWS (#2770)

Previously, any non-App Runner service threw out the log statement about the platform field in the manifest. 
Now, this happens only when the platform has been redirected _and_ a non-App Runner service has been created.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(manifest): use COPILOT_QUEUE_URIS in worker svc manifest (#2772)

<!-- Provide summary of changes -->

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: update help text to include Woker Service on init cmds (#2771)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: add metadata to queue resources  (#2773)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* docs: add task def override (#2763)

<!-- Provide summary of changes -->
Part of #2588.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: remove fifo field from queue settings in the Worker Service manifest (#2775)

⚰️

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: add progress tracker metadata to sns resources (#2774)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: update manifest env var to COPILOT_QUEUE_URI (#2779)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* fix: correctly inject environment variables for subscriptions. (#2778)

<!-- Provide summary of changes -->

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: add worker service to svc deploy (#2765)

<!-- Provide summary of changes -->
Continued from #2705. 

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: always create the events queue for Worker Services (#2782)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(cli): display recommended actions on "copilot deploy" (#2783)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: Bump github.com/aws/aws-sdk-go from 1.40.27 to 1.40.32 (#2789)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.27 to 1.40.32.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](aws/aws-sdk-go@v1.40.27...v1.40.32)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump github.com/AlecAivazis/survey/v2 from 2.3.0 to 2.3.1 (#2788)

Bumps [github.com/AlecAivazis/survey/v2](https://github.com/AlecAivazis/survey) from 2.3.0 to 2.3.1.
- [Release notes](https://github.com/AlecAivazis/survey/releases)
- [Commits](AlecAivazis/survey@v2.3.0...v2.3.1)

---
updated-dependencies:
- dependency-name: github.com/AlecAivazis/survey/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* chore(stack/rdws): fix add permissions and env var for sns (#2790)

_By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._

* fix(cli): command with env/svc flag should work without app flag (#2781)

<!-- Provide summary of changes -->
fixes #2776 
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: leverage interfaces to recommend deploy actions instead of switch statements (#2787)

There are also a _bajillion_ tiny refactors along the way.
The changes look big but it shouldn't be too bad mostly simplifying code or moving them. 

The main new logic can be found in this commit: efekarakus@6897454, the rest are refactors.

_By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._

* chore: format recommended actions slightly more inward (#2792)

🥺

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: add pointer to map to volume to take advantage of mergo's merge logic (#2793)

In `mergo`, if the map is to pointer-to-struct, then it performs deep merge on the pointer. By changing `map[string]Volume` to `map[string]*Volume`, we can take advantage of this logic instead of implementing a custom transformer.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* docs: add Worker Services to docs site. (#2785)

<!-- Provide summary of changes -->

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(manifest): test each individual transformers (#2795)

This PR tests each individual transformer. In addition, it also includes a small refactoring on basic transformer.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* feat: support darwin-arm64 CLI binary (#2802)

<!-- Provide summary of changes -->

- Compile and publish darwin/arm64 copilot CLI binary. 
  - This will allow copilot to run natively on newer macs rather than being emulated. 

I've tested the binary compilation but not sure if I can test the actual release build specs easily. 

Resolves #2745 

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: use forward slashes in file paths for go:embed (#2807)

Fixes #2803.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: PR template to specify the correct license (#2801)

<!-- Provide summary of changes -->

This fixes the PR template to specify the Apache 2.0 License rather than leaving it the default "under the terms of your choice". 

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: add Lambda fn to write BacklogPerTask CW metric (#2804)

We need to write this metric for Worker Service autoscaling.

Related #2796

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* fix: error out if the credential is invalid (#2816)

Previously, we don't error out if the credential is invalid or we fail to get credentials because of time out. 

A result of this is that we will keep using the session with the **invalid/empty credential**, which eventually causes the CLI to hang during a call to `GetCallerIdentity()` https://github.com/aws/copilot-cli/blob/77983421537d9f34575655ab8cba309f30610f33/internal/pkg/aws/identity/identity.go#L40

The hanging issue is reported in #2686.

An  **invalid/empty credential** happens when:
1. The profile is not found, AND
2. The environment variables `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are NOT set.

The profile is not found when:
1. The file `~/.aws/credentials` is not found
2. The file is found, but the profile is not defined


After this fix, the output from `copilot app init` or `copilot init` in the case of  an **invalid/empty credential** would be:
```
Welcome to the Copilot CLI! We're going to walk you through some questions
to help you get set up with a containerized application on AWS. An application is a collection of
containerized services that operate together.

✘ ask app init: get credentials of session: RequestCanceled: request context canceled
caused by: context deadline exceeded
```

instead of hanging.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License..

* chore(template): add cfn resources for worker svc autoscaling (#2813)

✅ Tested the template by deploying a worker service with autoscaling

Related #2796

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License..

* chore(manifest): refactor pointer struct to struct in manifest (#2798)

This PR is a refactor that removes unnecessary pointers ™️ from manifest.

"Necessary pointers" ™️  to the following structs are not removed:
##### <s>`ContainerHealthCheck`</s> and `PlatfromArgsOrString` for all workloads (e.g. https://github.com/aws/copilot-cli/blob/ff0a32e3d0d6821408ba6f05fcad6a0450f75f24/internal/pkg/manifest/lb_web_svc.go#L64)
This is because of the way we marshal manifest object into YAML during `svc init` or `job init`.

<s>##### `EFSConfigOrBool`
A manifest input that has:
```
efs:
<the rest of the manifest>
```
is deemed invalid, and we error out during the conversion from a manifest object to a template object. We cannot remove the pointer right now because otherwise we cannot differentiate a said invalid input, and a valid input that didn't specify `efs` at all. 

This validation should instead happens at an earlier stage, potentially from within a dedicated `manifest.validation` package. We can remove the pointer after we can move the validation to the said earlier stage.</s>

##### `SQSQueue `
For this particular manifest field, a manifest input that has:
```
topics:
     - name: topic1
        queue:
<the rest of the manifest>
```
is deemed valid. If the `queue` field is specified without any value, we create the queue for that topic with default configurations. Hence, there is indeed a need to differentiate an empty queue from an absent queue. 

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(cli): enable combination of flags and prompting for custom env resources (#2784)

Previously, it was assumed that users would 1. import neither VPC nor subnets and use the guided experience, or 2. import both VPC and subnets. This enables them to import one without the other.

Fixes #2777.

Use cases with these changes:

- import all custom resources: proceed as usual, including validation of two private subnets (okay to leave public subnets blank--log warning about load balancers needing two, but must have two private)
- import no custom resources: prompt to select VPC and subnets
- import just VPC: prompt to select subnets
- import two private: prompt to select VPC and public subnets
- import two public: prompt to select VPC and private subnets
- import just one private: error
- import just one public: warning, prompt to select VPC and private subnets

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: remove `applyEnv` unit tests and keep only those that are representative (#2806)

Previously I've added a plethora of `ApplyEnv` unit tests to baseline our `ApplyEnv` bahaviours on each of the manifest, in attempt to upgrade mergo and fix manifest behaviors.

Now that the upgrade is completed, these extensive tests can be removed, replaced with a few `ApplyEnv` tests for each of the representative field. 

This PR is dependent on #2798.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* fix(manifest): remove unnecessary `allowed_workers` field (#2821)

There is no way today to limit an SNS topic subscription to be based on the tag of SQS queues (https://aws.amazon.com/premiumsupport/knowledge-center/sqs-sns-subscribe-cloudformation/). Instead, by default SNS will limit subscriptions to be only within the same AWS account.  

Therefore, the `allowed_workers` field is not needed. This is not a breaking change, it makes a required field now obsolete. 


_By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License._

* fix: extend ARM platform changes through to deployment (#2814)

These changes enable workload images on ARM machines to build as amd64 images.
If the underlying docker engine detected is ARM architecture-based, Copilot builds with `--platform linux/amd64` so workloads deploy rather than going through the whole workflow only to fail to spin up. 

Related: #2636.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License..

* refactor(manifest): remove pointer for Range (#2820)

<!-- Provide summary of changes -->

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License..

* chore(manifest): add manifest validation boilerplate (#2815)

<!-- Provide summary of changes -->
Part of #2818. This PR adds boilerplate for new manifest validation for LB svc. Also added validation for mutually exclusive fields. Will add for the other workload types, advanced type validation, and consolidate existing validations in following PRs
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License..

* refactor: move exec/dockerfile* to new docker/dockerfile pkg (#2824)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.

* chore(manifest): add validate boilerplate for rest workload types (#2826)

<!-- Provide summary of changes -->
Part of #2818.

After this PR, we should have boilerplate completed for every workload type. Then, we'll add more advanced basic types validation as well as start to consolidate validation elsewhere.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License..

* docs(ja): periodic update (#2805)

This PR updates the Japanese localized docs based on the changes in the English docs after the v1.9.0 tag(b5c78a6) to the latest mainline(b152aed).

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: update PR template to remove extra period (#2828)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.

* chore: remove unused code

Co-authored-by: Penghao He <[email protected]>
Co-authored-by: Austin Ely <[email protected]>
Co-authored-by: Wanxian Yang <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: John Lucas <[email protected]>
Co-authored-by: Efe Karakus <[email protected]>
Co-authored-by: Brandon Wagner <[email protected]>
Co-authored-by: Kaji <[email protected]>
huanjani added a commit that referenced this issue Oct 12, 2021
…ogging logic (#2904)

* chore: fix npm dependency alert (#2740)

<!-- Provide summary of changes -->
https://github.com/aws/copilot-cli/security/dependabot/cf-custom-resources/package-lock.json/path-parse/open
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: add SNS topic selector to `select` (#2663)

<!-- Provide summary of changes -->
Adds a "deployed SNS topic" selector to the `select` package and some data structures to accompany it. Uses the `deploy store` under the hood and relies on resource group tagging API, so it can only be used on one environment at a time. 

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: upgrade mergo and implement transformers to maintain existing behaviors (#2738)

This PR is part of the second step (writing units test) to address #2492. It:
1. Upgrades `mergo`
2. Fixes code by implementing custom transformers so that all `applyEnv` tests that have been passing are still passing

Next: 
1. Change pointer struct to struct and remove `pStruct` transformers
2. Fix bugs (tests that have been failing and commented out in previous PRs (#2700 ~ #2734)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: Bump golang.org/x/mod from 0.4.2 to 0.5.0 (#2741)

Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.4.2 to 0.5.0.
- [Release notes](https://github.com/golang/mod/releases)
- [Commits](golang/mod@v0.4.2...v0.5.0)

---
updated-dependencies:
- dependency-name: golang.org/x/mod
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump github.com/aws/aws-sdk-go from 1.40.17 to 1.40.22 (#2742)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.17 to 1.40.22.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](aws/aws-sdk-go@v1.40.17...v1.40.22)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* chore: add worker services to svc show (#2744)

<!-- Provide summary of changes -->
Continued from #2703. 

Refactor described [here](https://github.com/aws/copilot-cli/pull/2703/files#r683800143) to be implemented in a future PR.

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(manifest): add taskdef_override and parsing func (#2743)

<!-- Provide summary of changes -->
Part of #2588.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* feat(cli): add --force for svc deploy (#2736)

<!-- Provide summary of changes -->
Add `--force` flag for `svc deploy` and `deploy`, allowing users to force deploy their ECS service. Fixes #2597. E2E test and doc will be added in an upcoming PR.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* refactor: embed common ecsServiceDescriber struct (#2748)

<!-- Provide summary of changes -->
As mentioned in #2744, refactors describer structs to remove redundancy.

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(stack): apply manifest taskdef override rules to CFN template (#2750)

<!-- Provide summary of changes -->
Part of #2588
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* feat: add worker services to `svc init` (#2752)

<!-- Provide summary of changes -->
Subsumes #2704. 

Also refactors SNS selector to a single prompt, where we only ask the customer to select topics that are deployed across all existing environments.

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* fix(manifest): fix previously found `ApplyEnv` bugs (#2746)

This PR This PR is part of the second step (writing units test) to address #2492. It:
1. Fix previously found `ApplyEnv` bugs by implementing transformers needed
2. Refactor `transform.go`

Previous: #2738 
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* feat: add --force flag, e2e test, and doc (#2749)

<!-- Provide summary of changes -->
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: Bump github.com/AlecAivazis/survey/v2 from 2.2.16 to 2.3.0 (#2757)

Bumps [github.com/AlecAivazis/survey/v2](https://github.com/AlecAivazis/survey) from 2.2.16 to 2.3.0.
- [Release notes](https://github.com/AlecAivazis/survey/releases)
- [Commits](AlecAivazis/survey@v2.2.16...v2.3.0)

---
updated-dependencies:
- dependency-name: github.com/AlecAivazis/survey/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump github.com/onsi/gomega from 1.15.0 to 1.16.0 (#2759)

Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.15.0 to 1.16.0.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](onsi/gomega@v1.15.0...v1.16.0)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* chore: Bump github.com/aws/aws-sdk-go from 1.40.22 to 1.40.27 (#2758)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.22 to 1.40.27.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](aws/aws-sdk-go@v1.40.22...v1.40.27)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* docs: Add --yes flag to svc exec docs. (#2761)

<!-- Provide summary of changes -->
Adds the --yes flag to `copilot svc exec` docs.

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->
Fixes #2760

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(override): unmarshal/marshal CFN template and parse node from rule (#2756)

This PR:
1. unmarshal/marshal CFN template to/from YAML nodes.
2. parse node from rule.
3. add a unit test for the exported function `CloudformationTemplate`.

Part of #2588

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: group segments into a `pathSegment` struct (#2764)

Addressing: #2756 (comment)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* build: upgrade e2e/Dockerfile go version to 1.16.6 from 1.13.15 (#2767)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(stack): add rule prefix and override integ test (#2762)

<!-- Provide summary of changes -->
Part of #2588. Work on the basis of #2756.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: implement the transformer for `FIFOOrBool` (#2768)

`FIFOOrBool` is a composite field and need custom merge logic.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* fix(cli): don't log platform msg unless both redirected and non-RDWS (#2770)

Previously, any non-App Runner service threw out the log statement about the platform field in the manifest. 
Now, this happens only when the platform has been redirected _and_ a non-App Runner service has been created.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(manifest): use COPILOT_QUEUE_URIS in worker svc manifest (#2772)

<!-- Provide summary of changes -->

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: update help text to include Woker Service on init cmds (#2771)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: add metadata to queue resources  (#2773)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* docs: add task def override (#2763)

<!-- Provide summary of changes -->
Part of #2588.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: remove fifo field from queue settings in the Worker Service manifest (#2775)

⚰️

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: add progress tracker metadata to sns resources (#2774)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: update manifest env var to COPILOT_QUEUE_URI (#2779)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* fix: correctly inject environment variables for subscriptions. (#2778)

<!-- Provide summary of changes -->

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: add worker service to svc deploy (#2765)

<!-- Provide summary of changes -->
Continued from #2705. 

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: always create the events queue for Worker Services (#2782)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(cli): display recommended actions on "copilot deploy" (#2783)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: Bump github.com/aws/aws-sdk-go from 1.40.27 to 1.40.32 (#2789)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.27 to 1.40.32.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](aws/aws-sdk-go@v1.40.27...v1.40.32)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump github.com/AlecAivazis/survey/v2 from 2.3.0 to 2.3.1 (#2788)

Bumps [github.com/AlecAivazis/survey/v2](https://github.com/AlecAivazis/survey) from 2.3.0 to 2.3.1.
- [Release notes](https://github.com/AlecAivazis/survey/releases)
- [Commits](AlecAivazis/survey@v2.3.0...v2.3.1)

---
updated-dependencies:
- dependency-name: github.com/AlecAivazis/survey/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* chore(stack/rdws): fix add permissions and env var for sns (#2790)

_By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._

* fix(cli): command with env/svc flag should work without app flag (#2781)

<!-- Provide summary of changes -->
fixes #2776 
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: leverage interfaces to recommend deploy actions instead of switch statements (#2787)

There are also a _bajillion_ tiny refactors along the way.
The changes look big but it shouldn't be too bad mostly simplifying code or moving them. 

The main new logic can be found in this commit: efekarakus@6897454, the rest are refactors.

_By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._

* chore: format recommended actions slightly more inward (#2792)

🥺

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: add pointer to map to volume to take advantage of mergo's merge logic (#2793)

In `mergo`, if the map is to pointer-to-struct, then it performs deep merge on the pointer. By changing `map[string]Volume` to `map[string]*Volume`, we can take advantage of this logic instead of implementing a custom transformer.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* docs: add Worker Services to docs site. (#2785)

<!-- Provide summary of changes -->

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(manifest): test each individual transformers (#2795)

This PR tests each individual transformer. In addition, it also includes a small refactoring on basic transformer.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* feat: support darwin-arm64 CLI binary (#2802)

<!-- Provide summary of changes -->

- Compile and publish darwin/arm64 copilot CLI binary. 
  - This will allow copilot to run natively on newer macs rather than being emulated. 

I've tested the binary compilation but not sure if I can test the actual release build specs easily. 

Resolves #2745 

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: use forward slashes in file paths for go:embed (#2807)

Fixes #2803.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: PR template to specify the correct license (#2801)

<!-- Provide summary of changes -->

This fixes the PR template to specify the Apache 2.0 License rather than leaving it the default "under the terms of your choice". 

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: add Lambda fn to write BacklogPerTask CW metric (#2804)

We need to write this metric for Worker Service autoscaling.

Related #2796

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* fix: error out if the credential is invalid (#2816)

Previously, we don't error out if the credential is invalid or we fail to get credentials because of time out. 

A result of this is that we will keep using the session with the **invalid/empty credential**, which eventually causes the CLI to hang during a call to `GetCallerIdentity()` https://github.com/aws/copilot-cli/blob/77983421537d9f34575655ab8cba309f30610f33/internal/pkg/aws/identity/identity.go#L40

The hanging issue is reported in #2686.

An  **invalid/empty credential** happens when:
1. The profile is not found, AND
2. The environment variables `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are NOT set.

The profile is not found when:
1. The file `~/.aws/credentials` is not found
2. The file is found, but the profile is not defined


After this fix, the output from `copilot app init` or `copilot init` in the case of  an **invalid/empty credential** would be:
```
Welcome to the Copilot CLI! We're going to walk you through some questions
to help you get set up with a containerized application on AWS. An application is a collection of
containerized services that operate together.

✘ ask app init: get credentials of session: RequestCanceled: request context canceled
caused by: context deadline exceeded
```

instead of hanging.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License..

* chore(template): add cfn resources for worker svc autoscaling (#2813)

✅ Tested the template by deploying a worker service with autoscaling

Related #2796

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License..

* chore(manifest): refactor pointer struct to struct in manifest (#2798)

This PR is a refactor that removes unnecessary pointers ™️ from manifest.

"Necessary pointers" ™️  to the following structs are not removed:
##### <s>`ContainerHealthCheck`</s> and `PlatfromArgsOrString` for all workloads (e.g. https://github.com/aws/copilot-cli/blob/ff0a32e3d0d6821408ba6f05fcad6a0450f75f24/internal/pkg/manifest/lb_web_svc.go#L64)
This is because of the way we marshal manifest object into YAML during `svc init` or `job init`.

<s>##### `EFSConfigOrBool`
A manifest input that has:
```
efs:
<the rest of the manifest>
```
is deemed invalid, and we error out during the conversion from a manifest object to a template object. We cannot remove the pointer right now because otherwise we cannot differentiate a said invalid input, and a valid input that didn't specify `efs` at all. 

This validation should instead happens at an earlier stage, potentially from within a dedicated `manifest.validation` package. We can remove the pointer after we can move the validation to the said earlier stage.</s>

##### `SQSQueue `
For this particular manifest field, a manifest input that has:
```
topics:
     - name: topic1
        queue:
<the rest of the manifest>
```
is deemed valid. If the `queue` field is specified without any value, we create the queue for that topic with default configurations. Hence, there is indeed a need to differentiate an empty queue from an absent queue. 

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(cli): enable combination of flags and prompting for custom env resources (#2784)

Previously, it was assumed that users would 1. import neither VPC nor subnets and use the guided experience, or 2. import both VPC and subnets. This enables them to import one without the other.

Fixes #2777.

Use cases with these changes:

- import all custom resources: proceed as usual, including validation of two private subnets (okay to leave public subnets blank--log warning about load balancers needing two, but must have two private)
- import no custom resources: prompt to select VPC and subnets
- import just VPC: prompt to select subnets
- import two private: prompt to select VPC and public subnets
- import two public: prompt to select VPC and private subnets
- import just one private: error
- import just one public: warning, prompt to select VPC and private subnets

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: remove `applyEnv` unit tests and keep only those that are representative (#2806)

Previously I've added a plethora of `ApplyEnv` unit tests to baseline our `ApplyEnv` bahaviours on each of the manifest, in attempt to upgrade mergo and fix manifest behaviors.

Now that the upgrade is completed, these extensive tests can be removed, replaced with a few `ApplyEnv` tests for each of the representative field. 

This PR is dependent on #2798.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* fix(manifest): remove unnecessary `allowed_workers` field (#2821)

There is no way today to limit an SNS topic subscription to be based on the tag of SQS queues (https://aws.amazon.com/premiumsupport/knowledge-center/sqs-sns-subscribe-cloudformation/). Instead, by default SNS will limit subscriptions to be only within the same AWS account.  

Therefore, the `allowed_workers` field is not needed. This is not a breaking change, it makes a required field now obsolete. 


_By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License._

* fix: extend ARM platform changes through to deployment (#2814)

These changes enable workload images on ARM machines to build as amd64 images.
If the underlying docker engine detected is ARM architecture-based, Copilot builds with `--platform linux/amd64` so workloads deploy rather than going through the whole workflow only to fail to spin up. 

Related: #2636.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License..

* refactor(manifest): remove pointer for Range (#2820)

<!-- Provide summary of changes -->

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License..

* chore(manifest): add manifest validation boilerplate (#2815)

<!-- Provide summary of changes -->
Part of #2818. This PR adds boilerplate for new manifest validation for LB svc. Also added validation for mutually exclusive fields. Will add for the other workload types, advanced type validation, and consolidate existing validations in following PRs
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License..

* refactor: move exec/dockerfile* to new docker/dockerfile pkg (#2824)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.

* chore(manifest): add validate boilerplate for rest workload types (#2826)

<!-- Provide summary of changes -->
Part of #2818.

After this PR, we should have boilerplate completed for every workload type. Then, we'll add more advanced basic types validation as well as start to consolidate validation elsewhere.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License..

* docs(ja): periodic update (#2805)

This PR updates the Japanese localized docs based on the changes in the English docs after the v1.9.0 tag(b5c78a6) to the latest mainline(b152aed).

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: update PR template to remove extra period (#2828)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.

* chore: remove unused code

* fix: narrow acceptible OS and Arch options because task run flags go straight to CFN

* fix: change from array of RuntimePlatform objects to one object

* chore: log platform manifest msg for all non-default, not just redirected

* chore: change job and svc logging

* fix: add linux as valid OS

* fix: tweak unit test

* fix: for real this time

* fix: change platform for CFN Windows OS formats

* chore: disable exec for windows tasks

* chore: add enableExec to unit test

* chore: add example of task run

* chore: remove repetitive code

* chore: rename var

* chore: change log statements, validate full platform not os/arch separately

Co-authored-by: Penghao He <[email protected]>
Co-authored-by: Austin Ely <[email protected]>
Co-authored-by: Wanxian Yang <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: John Lucas <[email protected]>
Co-authored-by: Efe Karakus <[email protected]>
Co-authored-by: Brandon Wagner <[email protected]>
Co-authored-by: Kaji <[email protected]>
huanjani added a commit that referenced this issue Oct 15, 2021
* chore: fix npm dependency alert (#2740)

<!-- Provide summary of changes -->
https://github.com/aws/copilot-cli/security/dependabot/cf-custom-resources/package-lock.json/path-parse/open
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: add SNS topic selector to `select` (#2663)

<!-- Provide summary of changes -->
Adds a "deployed SNS topic" selector to the `select` package and some data structures to accompany it. Uses the `deploy store` under the hood and relies on resource group tagging API, so it can only be used on one environment at a time. 

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: upgrade mergo and implement transformers to maintain existing behaviors (#2738)

This PR is part of the second step (writing units test) to address #2492. It:
1. Upgrades `mergo`
2. Fixes code by implementing custom transformers so that all `applyEnv` tests that have been passing are still passing

Next: 
1. Change pointer struct to struct and remove `pStruct` transformers
2. Fix bugs (tests that have been failing and commented out in previous PRs (#2700 ~ #2734)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: Bump golang.org/x/mod from 0.4.2 to 0.5.0 (#2741)

Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.4.2 to 0.5.0.
- [Release notes](https://github.com/golang/mod/releases)
- [Commits](golang/mod@v0.4.2...v0.5.0)

---
updated-dependencies:
- dependency-name: golang.org/x/mod
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump github.com/aws/aws-sdk-go from 1.40.17 to 1.40.22 (#2742)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.17 to 1.40.22.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](aws/aws-sdk-go@v1.40.17...v1.40.22)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* chore: add worker services to svc show (#2744)

<!-- Provide summary of changes -->
Continued from #2703. 

Refactor described [here](https://github.com/aws/copilot-cli/pull/2703/files#r683800143) to be implemented in a future PR.

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(manifest): add taskdef_override and parsing func (#2743)

<!-- Provide summary of changes -->
Part of #2588.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* feat(cli): add --force for svc deploy (#2736)

<!-- Provide summary of changes -->
Add `--force` flag for `svc deploy` and `deploy`, allowing users to force deploy their ECS service. Fixes #2597. E2E test and doc will be added in an upcoming PR.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* refactor: embed common ecsServiceDescriber struct (#2748)

<!-- Provide summary of changes -->
As mentioned in #2744, refactors describer structs to remove redundancy.

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(stack): apply manifest taskdef override rules to CFN template (#2750)

<!-- Provide summary of changes -->
Part of #2588
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* feat: add worker services to `svc init` (#2752)

<!-- Provide summary of changes -->
Subsumes #2704. 

Also refactors SNS selector to a single prompt, where we only ask the customer to select topics that are deployed across all existing environments.

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* fix(manifest): fix previously found `ApplyEnv` bugs (#2746)

This PR This PR is part of the second step (writing units test) to address #2492. It:
1. Fix previously found `ApplyEnv` bugs by implementing transformers needed
2. Refactor `transform.go`

Previous: #2738 
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* feat: add --force flag, e2e test, and doc (#2749)

<!-- Provide summary of changes -->
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: Bump github.com/AlecAivazis/survey/v2 from 2.2.16 to 2.3.0 (#2757)

Bumps [github.com/AlecAivazis/survey/v2](https://github.com/AlecAivazis/survey) from 2.2.16 to 2.3.0.
- [Release notes](https://github.com/AlecAivazis/survey/releases)
- [Commits](AlecAivazis/survey@v2.2.16...v2.3.0)

---
updated-dependencies:
- dependency-name: github.com/AlecAivazis/survey/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump github.com/onsi/gomega from 1.15.0 to 1.16.0 (#2759)

Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.15.0 to 1.16.0.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](onsi/gomega@v1.15.0...v1.16.0)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* chore: Bump github.com/aws/aws-sdk-go from 1.40.22 to 1.40.27 (#2758)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.22 to 1.40.27.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](aws/aws-sdk-go@v1.40.22...v1.40.27)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* docs: Add --yes flag to svc exec docs. (#2761)

<!-- Provide summary of changes -->
Adds the --yes flag to `copilot svc exec` docs.

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->
Fixes #2760

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(override): unmarshal/marshal CFN template and parse node from rule (#2756)

This PR:
1. unmarshal/marshal CFN template to/from YAML nodes.
2. parse node from rule.
3. add a unit test for the exported function `CloudformationTemplate`.

Part of #2588

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: group segments into a `pathSegment` struct (#2764)

Addressing: #2756 (comment)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* build: upgrade e2e/Dockerfile go version to 1.16.6 from 1.13.15 (#2767)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(stack): add rule prefix and override integ test (#2762)

<!-- Provide summary of changes -->
Part of #2588. Work on the basis of #2756.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: implement the transformer for `FIFOOrBool` (#2768)

`FIFOOrBool` is a composite field and need custom merge logic.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* fix(cli): don't log platform msg unless both redirected and non-RDWS (#2770)

Previously, any non-App Runner service threw out the log statement about the platform field in the manifest. 
Now, this happens only when the platform has been redirected _and_ a non-App Runner service has been created.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(manifest): use COPILOT_QUEUE_URIS in worker svc manifest (#2772)

<!-- Provide summary of changes -->

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: update help text to include Woker Service on init cmds (#2771)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: add metadata to queue resources  (#2773)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* docs: add task def override (#2763)

<!-- Provide summary of changes -->
Part of #2588.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: remove fifo field from queue settings in the Worker Service manifest (#2775)

⚰️

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: add progress tracker metadata to sns resources (#2774)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: update manifest env var to COPILOT_QUEUE_URI (#2779)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* fix: correctly inject environment variables for subscriptions. (#2778)

<!-- Provide summary of changes -->

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: add worker service to svc deploy (#2765)

<!-- Provide summary of changes -->
Continued from #2705. 

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: always create the events queue for Worker Services (#2782)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(cli): display recommended actions on "copilot deploy" (#2783)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: Bump github.com/aws/aws-sdk-go from 1.40.27 to 1.40.32 (#2789)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.40.27 to 1.40.32.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](aws/aws-sdk-go@v1.40.27...v1.40.32)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: Bump github.com/AlecAivazis/survey/v2 from 2.3.0 to 2.3.1 (#2788)

Bumps [github.com/AlecAivazis/survey/v2](https://github.com/AlecAivazis/survey) from 2.3.0 to 2.3.1.
- [Release notes](https://github.com/AlecAivazis/survey/releases)
- [Commits](AlecAivazis/survey@v2.3.0...v2.3.1)

---
updated-dependencies:
- dependency-name: github.com/AlecAivazis/survey/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* chore(stack/rdws): fix add permissions and env var for sns (#2790)

_By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._

* fix(cli): command with env/svc flag should work without app flag (#2781)

<!-- Provide summary of changes -->
fixes #2776 
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: leverage interfaces to recommend deploy actions instead of switch statements (#2787)

There are also a _bajillion_ tiny refactors along the way.
The changes look big but it shouldn't be too bad mostly simplifying code or moving them. 

The main new logic can be found in this commit: efekarakus@6897454, the rest are refactors.

_By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._

* chore: format recommended actions slightly more inward (#2792)

🥺

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: add pointer to map to volume to take advantage of mergo's merge logic (#2793)

In `mergo`, if the map is to pointer-to-struct, then it performs deep merge on the pointer. By changing `map[string]Volume` to `map[string]*Volume`, we can take advantage of this logic instead of implementing a custom transformer.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* docs: add Worker Services to docs site. (#2785)

<!-- Provide summary of changes -->

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(manifest): test each individual transformers (#2795)

This PR tests each individual transformer. In addition, it also includes a small refactoring on basic transformer.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* feat: support darwin-arm64 CLI binary (#2802)

<!-- Provide summary of changes -->

- Compile and publish darwin/arm64 copilot CLI binary. 
  - This will allow copilot to run natively on newer macs rather than being emulated. 

I've tested the binary compilation but not sure if I can test the actual release build specs easily. 

Resolves #2745 

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: use forward slashes in file paths for go:embed (#2807)

Fixes #2803.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: PR template to specify the correct license (#2801)

<!-- Provide summary of changes -->

This fixes the PR template to specify the Apache 2.0 License rather than leaving it the default "under the terms of your choice". 

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: add Lambda fn to write BacklogPerTask CW metric (#2804)

We need to write this metric for Worker Service autoscaling.

Related #2796

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* fix: error out if the credential is invalid (#2816)

Previously, we don't error out if the credential is invalid or we fail to get credentials because of time out. 

A result of this is that we will keep using the session with the **invalid/empty credential**, which eventually causes the CLI to hang during a call to `GetCallerIdentity()` https://github.com/aws/copilot-cli/blob/77983421537d9f34575655ab8cba309f30610f33/internal/pkg/aws/identity/identity.go#L40

The hanging issue is reported in #2686.

An  **invalid/empty credential** happens when:
1. The profile is not found, AND
2. The environment variables `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are NOT set.

The profile is not found when:
1. The file `~/.aws/credentials` is not found
2. The file is found, but the profile is not defined


After this fix, the output from `copilot app init` or `copilot init` in the case of  an **invalid/empty credential** would be:
```
Welcome to the Copilot CLI! We're going to walk you through some questions
to help you get set up with a containerized application on AWS. An application is a collection of
containerized services that operate together.

✘ ask app init: get credentials of session: RequestCanceled: request context canceled
caused by: context deadline exceeded
```

instead of hanging.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License..

* chore(template): add cfn resources for worker svc autoscaling (#2813)

✅ Tested the template by deploying a worker service with autoscaling

Related #2796

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License..

* chore(manifest): refactor pointer struct to struct in manifest (#2798)

This PR is a refactor that removes unnecessary pointers ™️ from manifest.

"Necessary pointers" ™️  to the following structs are not removed:
##### <s>`ContainerHealthCheck`</s> and `PlatfromArgsOrString` for all workloads (e.g. https://github.com/aws/copilot-cli/blob/ff0a32e3d0d6821408ba6f05fcad6a0450f75f24/internal/pkg/manifest/lb_web_svc.go#L64)
This is because of the way we marshal manifest object into YAML during `svc init` or `job init`.

<s>##### `EFSConfigOrBool`
A manifest input that has:
```
efs:
<the rest of the manifest>
```
is deemed invalid, and we error out during the conversion from a manifest object to a template object. We cannot remove the pointer right now because otherwise we cannot differentiate a said invalid input, and a valid input that didn't specify `efs` at all. 

This validation should instead happens at an earlier stage, potentially from within a dedicated `manifest.validation` package. We can remove the pointer after we can move the validation to the said earlier stage.</s>

##### `SQSQueue `
For this particular manifest field, a manifest input that has:
```
topics:
     - name: topic1
        queue:
<the rest of the manifest>
```
is deemed valid. If the `queue` field is specified without any value, we create the queue for that topic with default configurations. Hence, there is indeed a need to differentiate an empty queue from an absent queue. 

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore(cli): enable combination of flags and prompting for custom env resources (#2784)

Previously, it was assumed that users would 1. import neither VPC nor subnets and use the guided experience, or 2. import both VPC and subnets. This enables them to import one without the other.

Fixes #2777.

Use cases with these changes:

- import all custom resources: proceed as usual, including validation of two private subnets (okay to leave public subnets blank--log warning about load balancers needing two, but must have two private)
- import no custom resources: prompt to select VPC and subnets
- import just VPC: prompt to select subnets
- import two private: prompt to select VPC and public subnets
- import two public: prompt to select VPC and private subnets
- import just one private: error
- import just one public: warning, prompt to select VPC and private subnets

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: remove `applyEnv` unit tests and keep only those that are representative (#2806)

Previously I've added a plethora of `ApplyEnv` unit tests to baseline our `ApplyEnv` bahaviours on each of the manifest, in attempt to upgrade mergo and fix manifest behaviors.

Now that the upgrade is completed, these extensive tests can be removed, replaced with a few `ApplyEnv` tests for each of the representative field. 

This PR is dependent on #2798.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* fix(manifest): remove unnecessary `allowed_workers` field (#2821)

There is no way today to limit an SNS topic subscription to be based on the tag of SQS queues (https://aws.amazon.com/premiumsupport/knowledge-center/sqs-sns-subscribe-cloudformation/). Instead, by default SNS will limit subscriptions to be only within the same AWS account.  

Therefore, the `allowed_workers` field is not needed. This is not a breaking change, it makes a required field now obsolete. 


_By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License._

* fix: extend ARM platform changes through to deployment (#2814)

These changes enable workload images on ARM machines to build as amd64 images.
If the underlying docker engine detected is ARM architecture-based, Copilot builds with `--platform linux/amd64` so workloads deploy rather than going through the whole workflow only to fail to spin up. 

Related: #2636.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License..

* refactor(manifest): remove pointer for Range (#2820)

<!-- Provide summary of changes -->

<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License..

* chore(manifest): add manifest validation boilerplate (#2815)

<!-- Provide summary of changes -->
Part of #2818. This PR adds boilerplate for new manifest validation for LB svc. Also added validation for mutually exclusive fields. Will add for the other workload types, advanced type validation, and consolidate existing validations in following PRs
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License..

* refactor: move exec/dockerfile* to new docker/dockerfile pkg (#2824)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.

* chore(manifest): add validate boilerplate for rest workload types (#2826)

<!-- Provide summary of changes -->
Part of #2818.

After this PR, we should have boilerplate completed for every workload type. Then, we'll add more advanced basic types validation as well as start to consolidate validation elsewhere.
<!-- Issue number, if available. E.g. "Fixes #31", "Addresses #42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License..

* docs(ja): periodic update (#2805)

This PR updates the Japanese localized docs based on the changes in the English docs after the v1.9.0 tag(b5c78a6) to the latest mainline(b152aed).

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

* chore: update PR template to remove extra period (#2828)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.

* chore: remove unused code

* chore: add platformArgs documentation and fix casing

* chore: tweak doc wording

* chore: lowercase short platform in case user changes to anything w/ uc

Co-authored-by: Penghao He <[email protected]>
Co-authored-by: Austin Ely <[email protected]>
Co-authored-by: Wanxian Yang <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: John Lucas <[email protected]>
Co-authored-by: Efe Karakus <[email protected]>
Co-authored-by: Brandon Wagner <[email protected]>
Co-authored-by: Kaji <[email protected]>
@efekarakus efekarakus moved this to Complete in Copilot Roadmap 🗺️ Nov 13, 2022
thrau pushed a commit to localstack/copilot-cli-local that referenced this issue Dec 9, 2022
…#2612)

<!-- Provide summary of changes -->
Create `override` pkg and add some stub functions. Part of aws#2588.
<!-- Issue number, if available. E.g. "Fixes aws#31", "Addresses aws#42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
thrau pushed a commit to localstack/copilot-cli-local that referenced this issue Dec 9, 2022
<!-- Provide summary of changes -->
Part of aws#2588
<!-- Issue number, if available. E.g. "Fixes aws#31", "Addresses aws#42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
thrau pushed a commit to localstack/copilot-cli-local that referenced this issue Dec 9, 2022
<!-- Provide summary of changes -->
Part of aws#2588.
<!-- Issue number, if available. E.g. "Fixes aws#31", "Addresses aws#42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
thrau pushed a commit to localstack/copilot-cli-local that referenced this issue Dec 9, 2022
<!-- Provide summary of changes -->
Part of aws#2588.
<!-- Issue number, if available. E.g. "Fixes aws#31", "Addresses aws#42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
thrau pushed a commit to localstack/copilot-cli-local that referenced this issue Dec 9, 2022
…ws#2750)

<!-- Provide summary of changes -->
Part of aws#2588
<!-- Issue number, if available. E.g. "Fixes aws#31", "Addresses aws#42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
thrau pushed a commit to localstack/copilot-cli-local that referenced this issue Dec 9, 2022
…ule (aws#2756)

This PR:
1. unmarshal/marshal CFN template to/from YAML nodes.
2. parse node from rule.
3. add a unit test for the exported function `CloudformationTemplate`.

Part of aws#2588

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
thrau pushed a commit to localstack/copilot-cli-local that referenced this issue Dec 9, 2022
<!-- Provide summary of changes -->
Part of aws#2588. Work on the basis of aws#2756.
<!-- Issue number, if available. E.g. "Fixes aws#31", "Addresses aws#42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
thrau pushed a commit to localstack/copilot-cli-local that referenced this issue Dec 9, 2022
<!-- Provide summary of changes -->
Part of aws#2588.
<!-- Issue number, if available. E.g. "Fixes aws#31", "Addresses aws#42, 77" -->

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/design Issues that are design proposals.
Projects
Status: Complete
Development

No branches or pull requests

3 participants