Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to use custom Docker Image in every stage of the pipelines #382

Closed
stemons opened this issue Aug 25, 2021 · 3 comments
Closed

Unable to use custom Docker Image in every stage of the pipelines #382

stemons opened this issue Aug 25, 2021 · 3 comments
Assignees
@sbkok
Labels
bug Something isn't working
Milestone
v3.2.0

Comments

@stemons
Copy link
Contributor

stemons commented Aug 25, 2021 •
edited
Loading

Hello,

we are trying to use a custom Docker Image in every stage of the pipelines, but it seems that the image is used only in the first build phase using this configuration, while the other stages (plan and apply) use the standard image (STANDARD_5)

pipelines:
  - name: sample-s3-terraform
    default_providers:
      source:
        provider: codecommit
        properties:
          account_id: "111111111111" 
      build:
        provider: codebuild
        properties:
          environment_variables:
            TERRAFORM_VERSION: "1.0.0" # terraform version
          image: 
            repository_arn: arn:aws:ecr:eu-west-1:111111111111:repository/iac
            tag: "0.0.2"
      deploy:
        provider: codebuild
        properties:
          environment_variables:
            # TARGET_ACCOUNTS: 111111111111,222222222222 # target accounts
            TARGET_OUS: /exceptions/sandbox # target OUs
            MASTER_ACCOUNT_ID: 111111111111 # master account
            REGIONS: eu-west-1 # target regions
    params:
      restart_execution_on_update: true
    targets:
      - name: terraform-plan
        properties:
          spec_filename: tf_plan.yml # terraform plan
      - approval # manual approval
      - name: terraform-apply
        properties:
          spec_filename: tf_apply.yml # terraform apply

We tried to use also this configuration by specifying the repository_arn properties in every stage of the pipelines, but in this case we get an error.

pipelines:
  - name: sample-s3-terraform
    default_providers:
      source:
        provider: codecommit
        properties:
          account_id: "111111111111"
      build:
        provider: codebuild
        properties:
          environment_variables:
            TERRAFORM_VERSION: "1.0.0" # terraform version
          image: 
            repository_arn: arn:aws:ecr:eu-west-1:111111111111:repository/iac
            tag: "0.0.2"
      deploy:
        provider: codebuild
        properties:
          environment_variables:
            # TARGET_ACCOUNTS: 111111111111,222222222222 # target accounts
            TARGET_OUS: /exceptions/sandbox # target OUs
            MASTER_ACCOUNT_ID: 111111111111 # master account
            REGIONS: eu-west-1 # target regions
    params:
      restart_execution_on_update: true
    targets:
      - name: terraform-plan
        properties:
          spec_filename: tf_plan.yml # terraform plan
          image: 
            repository_arn: arn:aws:ecr:eu-west-1:111111111111:repository/iac
            tag: "0.0.2"
      - approval # manual approval
      - name: terraform-apply
        properties:
          spec_filename: tf_apply.yml # terraform apply
          image: 
            repository_arn: arn:aws:ecr:eu-west-1:111111111111:repository/iac
            tag: "0.0.2"
[Container] 2021/08/23 09:18:58 Running command cdk synth --app adf-build/cdk/generate_pipeline_stacks.py 1> /dev/null
2021-08-23 09:19:00,524 | INFO | __main__ | ADF Version 3.1.2 | (generate_pipeline_stacks.py:25)
2021-08-23 09:19:00,524 | INFO | __main__ | ADF Log Level is INFO | (generate_pipeline_stacks.py:26)
2021-08-23 09:19:00,529 | INFO | cdk_stacks.main | Pipeline creation/update of sample-s3-terraform commenced | (main.py:31)
jsii.errors.JavaScriptError: 
  Error: There is already a Construct with name 'custom_repo' in Stack [sample-s3-terraform]
      at Node.addChild (/tmp/jsii-kernel-bpFzVV/node_modules/constructs/lib/construct.js:529:19)
      at new Node (/tmp/jsii-kernel-bpFzVV/node_modules/constructs/lib/construct.js:38:28)
      at new ConstructNode (/tmp/jsii-kernel-bpFzVV/node_modules/@aws-cdk/core/lib/construct-compat.js:166:28)
      at Object.createNode (/tmp/jsii-kernel-bpFzVV/node_modules/@aws-cdk/core/lib/construct-compat.js:37:42)
      at new Construct (/tmp/jsii-kernel-bpFzVV/node_modules/constructs/lib/construct.js:576:32)
      at new Construct (/tmp/jsii-kernel-bpFzVV/node_modules/@aws-cdk/core/lib/construct-compat.js:35:9)
      at new Resource (/tmp/jsii-kernel-bpFzVV/node_modules/@aws-cdk/core/lib/resource.js:22:9)
      at new RepositoryBase (/tmp/jsii-kernel-bpFzVV/node_modules/@aws-cdk/aws-ecr/lib/repository.js:16:1)
      at new Import (/tmp/jsii-kernel-bpFzVV/node_modules/@aws-cdk/aws-ecr/lib/repository.js:231:17)
      at Function.fromRepositoryArn (/tmp/jsii-kernel-bpFzVV/node_modules/@aws-cdk/aws-ecr/lib/repository.js:240:16)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "adf-build/cdk/generate_pipeline_stacks.py", line 37, in <module>
    main()
  File "adf-build/cdk/generate_pipeline_stacks.py", line 33, in main
    PipelineStack(app, stack_input)
  File "/codebuild/output/src721917786/src/adf-build/jsii/_runtime.py", line 83, in __call__
    inst = super().__call__(*args, **kwargs)
  File "/codebuild/output/src721917786/src/adf-build/cdk/cdk_stacks/main.py", line 101, in __init__
    adf_codebuild.CodeBuild(
  File "/codebuild/output/src721917786/src/adf-build/jsii/_runtime.py", line 83, in __call__
    inst = super().__call__(*args, **kwargs)
  File "/codebuild/output/src721917786/src/adf-build/cdk/cdk_constructs/adf_codebuild.py", line 40, in __init__
    build_image=CodeBuild.determine_build_image(scope, target, map_params),
  File "/codebuild/output/src721917786/src/adf-build/cdk/cdk_constructs/adf_codebuild.py", line 177, in determine_build_image
    repo_arn = _ecr.Repository.from_repository_arn(
  File "/codebuild/output/src721917786/src/adf-build/aws_cdk/aws_ecr/__init__.py", line 1910, in from_repository_arn
    return jsii.sinvoke(cls, "fromRepositoryArn", [scope, id, repository_arn])
  File "/codebuild/output/src721917786/src/adf-build/jsii/_kernel/__init__.py", line 126, in wrapped
    return _recursize_dereference(kernel, fn(kernel, *args, **kwargs))
  File "/codebuild/output/src721917786/src/adf-build/jsii/_kernel/__init__.py", line 349, in sinvoke
    response = self.provider.sinvoke(
  File "/codebuild/output/src721917786/src/adf-build/jsii/_kernel/providers/process.py", line 371, in sinvoke
    return self._process.send(request, InvokeResponse)
  File "/codebuild/output/src721917786/src/adf-build/jsii/_kernel/providers/process.py", line 335, in send
    raise JSIIError(resp.error) from JavaScriptError(resp.stack)
jsii.errors.JSIIError: There is already a Construct with name 'custom_repo' in Stack [sample-s3-terraform]
Subprocess exited with error 1

[Container] 2021/08/23 09:19:00 Command did not exit successfully cdk synth --app adf-build/cdk/generate_pipeline_stacks.py 1> /dev/null exit status 1
[Container] 2021/08/23 09:19:00 Phase complete: BUILD State: FAILED
[Container] 2021/08/23 09:19:00 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: cdk synth --app adf-build/cdk/generate_pipeline_stacks.py 1> /dev/null. Reason: exit status 1
[Container] 2021/08/23 09:19:00 Entering phase POST_BUILD
[Container] 2021/08/23 09:19:00 Running command python adf-build/cdk/clean_pipelines.py
@StewartW StewartW added the bug Something isn't working label Sep 23, 2021
@stemons
Copy link
Contributor Author

stemons commented Nov 10, 2021

Hello team, any chance to add this in the next release?

@sbkok
Copy link
Collaborator

sbkok commented Nov 10, 2021

Hi @stemons,
I will look into the effort required to assess if this could be added as part of the next release.
I will keep you posted here.

@sbkok sbkok self-assigned this Nov 10, 2021
sbkok added a commit to sbkok/aws-deployment-framework that referenced this issue Aug 25, 2022
@sbkok
Fix use of separate container image per target
fe1778a 
**Why?**

As reported by awslabs#382, setting a specific container image to use by the
CodeBuild provider to deploy to a target resulted in an error.

The CDK stack would fail to synthesize as the 'custom_repo' resource
name was already present in the stack.

**What?**

This fix ensures a unique identifier is used for custom container repositories.
sbkok added a commit to sbkok/aws-deployment-framework that referenced this issue Aug 25, 2022
@sbkok
Fix use of separate container image per target
214a00d 
**Why?**

As reported by awslabs#382, setting a specific container image to use by the
CodeBuild provider to deploy to a target resulted in an error.

The CDK stack would fail to synthesize as the 'custom_repo' resource
name was already present in the stack.

**What?**

This fix ensures a unique identifier is used for custom container repositories.
sbkok added a commit to sbkok/aws-deployment-framework that referenced this issue Aug 25, 2022
@sbkok
Fix use of separate container image per target
84633d2 
**Why?**

As reported by awslabs#382, setting a specific container image to use by the
CodeBuild provider to deploy to a target resulted in an error.

The CDK stack would fail to synthesize as the 'custom_repo' resource
name was already present in the stack.

**What?**

This fix ensures a unique identifier is used for custom container repositories.
@sbkok sbkok mentioned this issue Aug 25, 2022
Merged
@sbkok sbkok added this to the v3.2.0 milestone Aug 26, 2022
sbkok added a commit that referenced this issue Aug 26, 2022
@sbkok
Fix use of separate container image per target (#501)
b9d045a 
**Why?**

As reported by #382, setting a specific container image to use by the
CodeBuild provider to deploy to a target resulted in an error.

The CDK stack would fail to synthesize as the 'custom_repo' resource
name was already present in the stack.

**What?**

This fix ensures a unique identifier is used for custom container repositories.
@sbkok
Copy link
Collaborator

sbkok commented Jan 24, 2023

Thank you for your patience. I am happy to inform you that this issue has been resolved in our latest release v3.2.0 just now.
I'm hereby closing this issue. Please open a new issue if you are experiencing any issues with the latest release.

@sbkok sbkok closed this as completed Jan 24, 2023
@AndreasAugustin AndreasAugustin mentioned this issue Feb 2, 2023
Closed
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sbkok sbkok

Labels
bug Something isn't working
Projects
None yet
Milestone
v3.2.0
Development

No branches or pull requests
3 participants
@sbkok @StewartW @stemons