Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

awsproviderlint: New Check: Prefer aws_partition data source instead of hardcoded Partition identifier/domain #12997

Closed
breathingdust opened this issue Apr 24, 2020 · 2 comments
Labels
linter Pertains to changes to or issues with the various linters. provider Pertains to the provider itself, rather than any interaction with AWS. stale Old or inactive issues managed by automation, if no further action taken these will get closed. technical-debt Addresses areas of the codebase that need refactoring or redesign.

Comments

@breathingdust
Copy link
Member

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

To ensure test configurations are partition agnostic, any hardcoded Partition identifiers ([a-z]{2}(-[a-z]+)+) and domains should be replaced with the aws_partition data source.

Flagged Code

In test configuration:

"arn:aws:..."

// or

ec2.amazonaws.com

Passing Code

In test configuration:

data "aws_partition" "current" {}

"arn:${data.aws_partition.current.partition}:..."

ec2.${data.aws_partition.current.dns_suffix}
@breathingdust breathingdust added technical-debt Addresses areas of the codebase that need refactoring or redesign. provider Pertains to the provider itself, rather than any interaction with AWS. labels Apr 24, 2020
bflad added a commit that referenced this issue Oct 3, 2020
…new secrets support in container properties

Reference: #12997
Reference: #15469
Reference: https://aws.amazon.com/about-aws/whats-new/2020/10/aws-batch-now-supports-custom-logging-configurations-swap-space-and-shared-memory/

Changes:

```
* data-source/aws_batch_compute_environment: Add `tags` attribute
* data-source/aws_batch_job_queue: Add `tags` attribute
* resource/aws_batch_compute_environment: Add `tags` argument
* resource/aws_batch_job_definition: Add `tags` argument
* resource/aws_batch_job_queue: Add `tags` argument

BUG FIXES

* resource/aws_batch_job_definition: Prevent unexpected plan difference for `container_properties` argument value with new secrets support
```

With the recent Batch service updates, the new secrets support in container properties would show as a confusing plan difference:

```
              ~ container_properties = jsonencode(
                  ~ {
                        command              = [
                            "echo",
                            "test",
                        ]
                      - environment          = [] -> null
                        image                = "busybox"
                        memory               = 128
                      - mountPoints          = [] -> null
                      - resourceRequirements = [] -> null
                      - secrets              = [] -> null
                      - ulimits              = [] -> null
                        vcpus                = 1
                      - volumes              = [] -> null
                    }
                )
```

This augments the Batch equivalency to ensure the canonicalized API response of an empty secrets array matches no secrets configuration.

While verifying the updates, this fixes the following two AWS GovCloud (US) test failures due to hardcoded partition handling:

```
=== CONT  TestAccDataSourceAwsBatchJobQueue_basic
    data_source_aws_batch_job_queue_test.go:16: Step 1/1 error: terraform failed: exit status 1

        stderr:

        Error: Provider produced inconsistent result after apply

        When applying changes to aws_iam_role_policy_attachment.ecs_instance_role,
        provider "registry.terraform.io/-/aws" produced an unexpected new value for
        was present, but now absent.

        This is a bug in the provider, which should be reported in the provider's own
        issue tracker.

        Error: Provider produced inconsistent result after apply

        When applying changes to
        aws_iam_role_policy_attachment.aws_batch_service_role, provider
        "registry.terraform.io/-/aws" produced an unexpected new value for was
        present, but now absent.

        This is a bug in the provider, which should be reported in the provider's own
        issue tracker.

=== CONT  TestAccDataSourceAwsBatchComputeEnvironment_basic
    data_source_aws_batch_compute_environment_test.go:16: Step 1/1 error: terraform failed: exit status 1

        stderr:

        Error: Provider produced inconsistent result after apply

        When applying changes to aws_iam_role_policy_attachment.ecs_instance_role,
        provider "registry.terraform.io/-/aws" produced an unexpected new value for
        was present, but now absent.

        This is a bug in the provider, which should be reported in the provider's own
        issue tracker.

        Error: Provider produced inconsistent result after apply

        When applying changes to
        aws_iam_role_policy_attachment.aws_batch_service_role, provider
        "registry.terraform.io/-/aws" produced an unexpected new value for was
        present, but now absent.

        This is a bug in the provider, which should be reported in the provider's own
        issue tracker.
```

Output from acceptance testing in AWS Commercial:

```
--- PASS: TestAccAWSBatchComputeEnvironment_ComputeResources_DesiredVcpus_Computed (224.88s)
--- PASS: TestAccAWSBatchComputeEnvironment_ComputeResources_MaxVcpus (105.82s)
--- PASS: TestAccAWSBatchComputeEnvironment_ComputeResources_MinVcpus (213.34s)
--- PASS: TestAccAWSBatchComputeEnvironment_createEc2 (51.12s)
--- PASS: TestAccAWSBatchComputeEnvironment_createEc2WithoutComputeResources (32.16s)
--- PASS: TestAccAWSBatchComputeEnvironment_createEc2WithTags (46.93s)
--- PASS: TestAccAWSBatchComputeEnvironment_createSpot (54.38s)
--- PASS: TestAccAWSBatchComputeEnvironment_createSpotWithAllocationStrategy (54.66s)
--- PASS: TestAccAWSBatchComputeEnvironment_createSpotWithoutBidPercentage (22.83s)
--- PASS: TestAccAWSBatchComputeEnvironment_createUnmanaged (46.86s)
--- PASS: TestAccAWSBatchComputeEnvironment_createUnmanagedWithComputeResources (54.73s)
--- PASS: TestAccAWSBatchComputeEnvironment_createWithNamePrefix (54.19s)
--- PASS: TestAccAWSBatchComputeEnvironment_disappears (49.79s)
--- PASS: TestAccAWSBatchComputeEnvironment_launchTemplate (62.07s)
--- PASS: TestAccAWSBatchComputeEnvironment_Tags (107.51s)
--- PASS: TestAccAWSBatchComputeEnvironment_updateComputeEnvironmentName (97.15s)
--- PASS: TestAccAWSBatchComputeEnvironment_updateInstanceType (96.17s)
--- PASS: TestAccAWSBatchComputeEnvironment_UpdateLaunchTemplate (110.14s)
--- PASS: TestAccAWSBatchComputeEnvironment_updateState (92.78s)

--- PASS: TestAccAWSBatchJobDefinition_basic (31.33s)
--- PASS: TestAccAWSBatchJobDefinition_ContainerProperties_Advanced (30.82s)
--- PASS: TestAccAWSBatchJobDefinition_Tags (51.78s)
--- PASS: TestAccAWSBatchJobDefinition_updateForcesNewResource (36.03s)

--- PASS: TestAccAWSBatchJobQueue_basic (142.93s)
--- PASS: TestAccAWSBatchJobQueue_ComputeEnvironments_ExternalOrderUpdate (146.15s)
--- PASS: TestAccAWSBatchJobQueue_disappears (159.82s)
--- PASS: TestAccAWSBatchJobQueue_Priority (182.59s)
--- PASS: TestAccAWSBatchJobQueue_State (146.39s)
--- PASS: TestAccAWSBatchJobQueue_Tags (210.84s)

--- PASS: TestAccDataSourceAwsBatchComputeEnvironment_basic (58.52s)

--- PASS: TestAccDataSourceAwsBatchJobQueue_basic (145.37s)
```

Output from acceptance testing in AWS GovCloud (US):

```
--- PASS: TestAccAWSBatchComputeEnvironment_ComputeResources_DesiredVcpus_Computed (246.11s)
--- PASS: TestAccAWSBatchComputeEnvironment_ComputeResources_MaxVcpus (143.57s)
--- PASS: TestAccAWSBatchComputeEnvironment_ComputeResources_MinVcpus (271.07s)
--- PASS: TestAccAWSBatchComputeEnvironment_createEc2 (52.91s)
--- PASS: TestAccAWSBatchComputeEnvironment_createEc2WithoutComputeResources (24.92s)
--- PASS: TestAccAWSBatchComputeEnvironment_createEc2WithTags (58.91s)
--- PASS: TestAccAWSBatchComputeEnvironment_createSpot (62.24s)
--- PASS: TestAccAWSBatchComputeEnvironment_createSpotWithAllocationStrategy (57.05s)
--- PASS: TestAccAWSBatchComputeEnvironment_createSpotWithoutBidPercentage (22.58s)
--- PASS: TestAccAWSBatchComputeEnvironment_createUnmanaged (51.34s)
--- PASS: TestAccAWSBatchComputeEnvironment_createUnmanagedWithComputeResources (64.74s)
--- PASS: TestAccAWSBatchComputeEnvironment_createWithNamePrefix (51.71s)
--- PASS: TestAccAWSBatchComputeEnvironment_disappears (60.41s)
--- PASS: TestAccAWSBatchComputeEnvironment_launchTemplate (77.36s)
--- PASS: TestAccAWSBatchComputeEnvironment_Tags (121.87s)
--- PASS: TestAccAWSBatchComputeEnvironment_updateComputeEnvironmentName (108.52s)
--- PASS: TestAccAWSBatchComputeEnvironment_updateInstanceType (125.32s)
--- PASS: TestAccAWSBatchComputeEnvironment_UpdateLaunchTemplate (110.03s)
--- PASS: TestAccAWSBatchComputeEnvironment_updateState (91.09s)

--- PASS: TestAccAWSBatchJobDefinition_basic (24.22s)
--- PASS: TestAccAWSBatchJobDefinition_ContainerProperties_Advanced (24.59s)
--- PASS: TestAccAWSBatchJobDefinition_Tags (67.28s)
--- PASS: TestAccAWSBatchJobDefinition_updateForcesNewResource (36.09s)

--- PASS: TestAccAWSBatchJobQueue_basic (150.57s)
--- PASS: TestAccAWSBatchJobQueue_ComputeEnvironments_ExternalOrderUpdate (160.87s)
--- PASS: TestAccAWSBatchJobQueue_disappears (139.57s)
--- PASS: TestAccAWSBatchJobQueue_Priority (155.30s)
--- PASS: TestAccAWSBatchJobQueue_State (182.97s)
--- PASS: TestAccAWSBatchJobQueue_Tags (159.11s)

--- PASS: TestAccDataSourceAwsBatchComputeEnvironment_basic (54.23s)

--- PASS: TestAccDataSourceAwsBatchJobQueue_basic (139.32s)
```
bflad added a commit that referenced this issue Oct 7, 2020
…new secrets support in container properties (#15470)

* service/batch: Support resource tagging and prevent differences with new secrets support in container properties

Reference: #12997
Reference: #15469
Reference: https://aws.amazon.com/about-aws/whats-new/2020/10/aws-batch-now-supports-custom-logging-configurations-swap-space-and-shared-memory/

Changes:

```
* data-source/aws_batch_compute_environment: Add `tags` attribute
* data-source/aws_batch_job_queue: Add `tags` attribute
* resource/aws_batch_compute_environment: Add `tags` argument
* resource/aws_batch_job_definition: Add `tags` argument
* resource/aws_batch_job_queue: Add `tags` argument

BUG FIXES

* resource/aws_batch_job_definition: Prevent unexpected plan difference for `container_properties` argument value with new secrets support
```

With the recent Batch service updates, the new secrets support in container properties would show as a confusing plan difference:

```
              ~ container_properties = jsonencode(
                  ~ {
                        command              = [
                            "echo",
                            "test",
                        ]
                      - environment          = [] -> null
                        image                = "busybox"
                        memory               = 128
                      - mountPoints          = [] -> null
                      - resourceRequirements = [] -> null
                      - secrets              = [] -> null
                      - ulimits              = [] -> null
                        vcpus                = 1
                      - volumes              = [] -> null
                    }
                )
```

This augments the Batch equivalency to ensure the canonicalized API response of an empty secrets array matches no secrets configuration.

While verifying the updates, this fixes the following two AWS GovCloud (US) test failures due to hardcoded partition handling:

```
=== CONT  TestAccDataSourceAwsBatchJobQueue_basic
    data_source_aws_batch_job_queue_test.go:16: Step 1/1 error: terraform failed: exit status 1

        stderr:

        Error: Provider produced inconsistent result after apply

        When applying changes to aws_iam_role_policy_attachment.ecs_instance_role,
        provider "registry.terraform.io/-/aws" produced an unexpected new value for
        was present, but now absent.

        This is a bug in the provider, which should be reported in the provider's own
        issue tracker.

        Error: Provider produced inconsistent result after apply

        When applying changes to
        aws_iam_role_policy_attachment.aws_batch_service_role, provider
        "registry.terraform.io/-/aws" produced an unexpected new value for was
        present, but now absent.

        This is a bug in the provider, which should be reported in the provider's own
        issue tracker.

=== CONT  TestAccDataSourceAwsBatchComputeEnvironment_basic
    data_source_aws_batch_compute_environment_test.go:16: Step 1/1 error: terraform failed: exit status 1

        stderr:

        Error: Provider produced inconsistent result after apply

        When applying changes to aws_iam_role_policy_attachment.ecs_instance_role,
        provider "registry.terraform.io/-/aws" produced an unexpected new value for
        was present, but now absent.

        This is a bug in the provider, which should be reported in the provider's own
        issue tracker.

        Error: Provider produced inconsistent result after apply

        When applying changes to
        aws_iam_role_policy_attachment.aws_batch_service_role, provider
        "registry.terraform.io/-/aws" produced an unexpected new value for was
        present, but now absent.

        This is a bug in the provider, which should be reported in the provider's own
        issue tracker.
```

Output from acceptance testing in AWS Commercial:

```
--- PASS: TestAccAWSBatchComputeEnvironment_ComputeResources_DesiredVcpus_Computed (224.88s)
--- PASS: TestAccAWSBatchComputeEnvironment_ComputeResources_MaxVcpus (105.82s)
--- PASS: TestAccAWSBatchComputeEnvironment_ComputeResources_MinVcpus (213.34s)
--- PASS: TestAccAWSBatchComputeEnvironment_createEc2 (51.12s)
--- PASS: TestAccAWSBatchComputeEnvironment_createEc2WithoutComputeResources (32.16s)
--- PASS: TestAccAWSBatchComputeEnvironment_createEc2WithTags (46.93s)
--- PASS: TestAccAWSBatchComputeEnvironment_createSpot (54.38s)
--- PASS: TestAccAWSBatchComputeEnvironment_createSpotWithAllocationStrategy (54.66s)
--- PASS: TestAccAWSBatchComputeEnvironment_createSpotWithoutBidPercentage (22.83s)
--- PASS: TestAccAWSBatchComputeEnvironment_createUnmanaged (46.86s)
--- PASS: TestAccAWSBatchComputeEnvironment_createUnmanagedWithComputeResources (54.73s)
--- PASS: TestAccAWSBatchComputeEnvironment_createWithNamePrefix (54.19s)
--- PASS: TestAccAWSBatchComputeEnvironment_disappears (49.79s)
--- PASS: TestAccAWSBatchComputeEnvironment_launchTemplate (62.07s)
--- PASS: TestAccAWSBatchComputeEnvironment_Tags (107.51s)
--- PASS: TestAccAWSBatchComputeEnvironment_updateComputeEnvironmentName (97.15s)
--- PASS: TestAccAWSBatchComputeEnvironment_updateInstanceType (96.17s)
--- PASS: TestAccAWSBatchComputeEnvironment_UpdateLaunchTemplate (110.14s)
--- PASS: TestAccAWSBatchComputeEnvironment_updateState (92.78s)

--- PASS: TestAccAWSBatchJobDefinition_basic (31.33s)
--- PASS: TestAccAWSBatchJobDefinition_ContainerProperties_Advanced (30.82s)
--- PASS: TestAccAWSBatchJobDefinition_Tags (51.78s)
--- PASS: TestAccAWSBatchJobDefinition_updateForcesNewResource (36.03s)

--- PASS: TestAccAWSBatchJobQueue_basic (142.93s)
--- PASS: TestAccAWSBatchJobQueue_ComputeEnvironments_ExternalOrderUpdate (146.15s)
--- PASS: TestAccAWSBatchJobQueue_disappears (159.82s)
--- PASS: TestAccAWSBatchJobQueue_Priority (182.59s)
--- PASS: TestAccAWSBatchJobQueue_State (146.39s)
--- PASS: TestAccAWSBatchJobQueue_Tags (210.84s)

--- PASS: TestAccDataSourceAwsBatchComputeEnvironment_basic (58.52s)

--- PASS: TestAccDataSourceAwsBatchJobQueue_basic (145.37s)
```

Output from acceptance testing in AWS GovCloud (US):

```
--- PASS: TestAccAWSBatchComputeEnvironment_ComputeResources_DesiredVcpus_Computed (246.11s)
--- PASS: TestAccAWSBatchComputeEnvironment_ComputeResources_MaxVcpus (143.57s)
--- PASS: TestAccAWSBatchComputeEnvironment_ComputeResources_MinVcpus (271.07s)
--- PASS: TestAccAWSBatchComputeEnvironment_createEc2 (52.91s)
--- PASS: TestAccAWSBatchComputeEnvironment_createEc2WithoutComputeResources (24.92s)
--- PASS: TestAccAWSBatchComputeEnvironment_createEc2WithTags (58.91s)
--- PASS: TestAccAWSBatchComputeEnvironment_createSpot (62.24s)
--- PASS: TestAccAWSBatchComputeEnvironment_createSpotWithAllocationStrategy (57.05s)
--- PASS: TestAccAWSBatchComputeEnvironment_createSpotWithoutBidPercentage (22.58s)
--- PASS: TestAccAWSBatchComputeEnvironment_createUnmanaged (51.34s)
--- PASS: TestAccAWSBatchComputeEnvironment_createUnmanagedWithComputeResources (64.74s)
--- PASS: TestAccAWSBatchComputeEnvironment_createWithNamePrefix (51.71s)
--- PASS: TestAccAWSBatchComputeEnvironment_disappears (60.41s)
--- PASS: TestAccAWSBatchComputeEnvironment_launchTemplate (77.36s)
--- PASS: TestAccAWSBatchComputeEnvironment_Tags (121.87s)
--- PASS: TestAccAWSBatchComputeEnvironment_updateComputeEnvironmentName (108.52s)
--- PASS: TestAccAWSBatchComputeEnvironment_updateInstanceType (125.32s)
--- PASS: TestAccAWSBatchComputeEnvironment_UpdateLaunchTemplate (110.03s)
--- PASS: TestAccAWSBatchComputeEnvironment_updateState (91.09s)

--- PASS: TestAccAWSBatchJobDefinition_basic (24.22s)
--- PASS: TestAccAWSBatchJobDefinition_ContainerProperties_Advanced (24.59s)
--- PASS: TestAccAWSBatchJobDefinition_Tags (67.28s)
--- PASS: TestAccAWSBatchJobDefinition_updateForcesNewResource (36.09s)

--- PASS: TestAccAWSBatchJobQueue_basic (150.57s)
--- PASS: TestAccAWSBatchJobQueue_ComputeEnvironments_ExternalOrderUpdate (160.87s)
--- PASS: TestAccAWSBatchJobQueue_disappears (139.57s)
--- PASS: TestAccAWSBatchJobQueue_Priority (155.30s)
--- PASS: TestAccAWSBatchJobQueue_State (182.97s)
--- PASS: TestAccAWSBatchJobQueue_Tags (159.11s)

--- PASS: TestAccDataSourceAwsBatchComputeEnvironment_basic (54.23s)

--- PASS: TestAccDataSourceAwsBatchJobQueue_basic (139.32s)
```

* Update aws/data_source_aws_batch_compute_environment.go

Co-authored-by: Graham Davison <[email protected]>

Co-authored-by: Graham Davison <[email protected]>
@bflad bflad added the linter Pertains to changes to or issues with the various linters. label Oct 28, 2020
@github-actions
Copy link

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

@github-actions github-actions bot added the stale Old or inactive issues managed by automation, if no further action taken these will get closed. label Oct 18, 2022
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Nov 18, 2022
@github-actions
Copy link

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 19, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
linter Pertains to changes to or issues with the various linters. provider Pertains to the provider itself, rather than any interaction with AWS. stale Old or inactive issues managed by automation, if no further action taken these will get closed. technical-debt Addresses areas of the codebase that need refactoring or redesign.
Projects
None yet
Development

No branches or pull requests

2 participants