Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(code pipeline): basic cross-account support #1529

Closed
wants to merge 2 commits into from

Conversation

rsmogura
Copy link
Contributor

@rsmogura rsmogura commented Jan 12, 2019

Support for cross-account deployments, and cross-account artifacts replication.

Overview

The work should enable to construct stacks simillar to one described in
reference architecture
https://github.com/awslabs/aws-refarch-cross-account-pipeline

In such a case foreign role have to have access to stored artifacts
in S3 buckets. However in order to achieve this two things must be achieved

  • foreign account (role) has to be whitelisted on artifacts' bucket policy; and
  • foreign account (role) has to be whitelisted on KMS keys used to encrypt
    artifacts in above bucket (which is impossible to do with AWS managed
    one).

Because names of buckets and KMS key aliases have to be known upfront to pass
those in cross region fashion between underlying Cloud Formation
templates, those are derived from pipeline name (similarly as it's right
for scaffold buckets).

As KMS keys doesn't have name the alias to key is used and it's passed
as configuration to pipeline artifact store, so replicated
artifacts will be encrypted with those keys.

Pipeline tries to whitelist foreign accounts on artifacts stores (buckets
and KMS keys), by adding foreign account root principal to resource
policy.

Changes

Add account attribute to action, and treat it in similar way
as it's right now for region.

Introduced new constructs ArtifactsStore and ImportedArtifactsStore
to encapsulate artifact store (S3 bucket and KMS key), and simplify
managing of stores.

When action is attached to pipeline, check if it's from foreign account,
check if artifact store exists for this region and account. If it doesn't
than create new scaffold stack. Each artifact store created by pipeline
is considered pipeline managed.

If artifact store is pipeline managed whitelist foreign account (add
appropriate resource policy to S3 bucket and KMS key).

When granting permissions to pipeline's artifact bucket
(grantBucketRead/Write), store those n list and grant when pipeline
is rendered. It's driven by two factors:

  • actions may ask (i.e. project build one) for permission before
    all stores has been created;
  • in case of cross region / account calls, such calls have to be
    performed only on imported store to add grants in IAM principal
    policy, and not in resource policy (scaffold stacks are deployed
    before actual role can be deployed)

Changes to S3 allowing grant to ArnPrincipal (would updated only
resource policy)

Sample workflow

Together with #1449 customers should be able
to create on their own roles, and import those in pipeline stack. When

Some parts of this workflow could be automated, however even without it users would
be enabled to create cross-account pipelines.

  • Create stack for deployments in foreign account
  • In pipeline account
    • import both roles (require role ARNs to be known)
    • set imported jump role on Cloud Formation actions;
    • set imported deployment role as Cloud Formation role
    • user may be required to get encryption key for pipeline's
      artifact store and pass it to Project Build action,
      as Project Build by default encrypts output
      with aws/s3 key (even if default key is set on bucket)

TODO

In many places kms:* actions are allowed (those should be narrowed to only
those required for encryption, decryption and alias dereference).

Eventually, use tags and conditional statements to widen access to KMS keys;
Right now all KMS keys are granted for principals (in order to allow
principal access to particular key it's UUID has to be known, alias
can't be used in-place)

Project build action requires passing key from pipeline, otherwise artifacts will not get
replicated, as build action will use default aws/s3 key.

@rsmogura rsmogura requested review from RomainMuller, skinny85 and a team as code owners January 12, 2019 02:30
@rsmogura rsmogura force-pushed the cross-account-support branch 2 times, most recently from 5b4c5c7 to 633b7aa Compare January 14, 2019 06:30
@rsmogura rsmogura changed the title feat(codepipeline): migrate to configurable KMS keys (cross-account w… feat(code pipeline): basic cross-account support Jan 14, 2019
@rsmogura rsmogura force-pushed the cross-account-support branch 2 times, most recently from 886f66f to fe81143 Compare January 14, 2019 17:40
@eladb
Copy link
Contributor

eladb commented Jan 15, 2019

@skinny85 can you please take a look?

@skinny85
Copy link
Contributor

@eladb yep, I'm in communications with @rsmogura on this one

@rsmogura rsmogura force-pushed the cross-account-support branch 2 times, most recently from 4138250 to ea36531 Compare January 17, 2019 20:26
Radoslaw Smogura added 2 commits January 17, 2019 14:39
Use new construct `ArtifactStore` to represent pipeline artifact store.
If this construct has `artifactsKeyAlias` set than use it in output
template as encryption key.

For cross-account deployments artifacts must by encrypted and replicated
using key to which foreign account has an access, to. By default
`aws/s3` key is used which has limited support for setting resource policy.

In addition it’s hard to pass KMS keys between stacks, as KMS keys
are identified by freshly generated UUID, so it’s impossible to generate
scaffold stack and import / export values from it - physical
ARN should be passed to simplify things, so alias is used instead of
KMS key.

This change itself only enables customers to build cross-account pipelines,
but doesn’t support them in automatic managing or creation of keys
(subsequent changes will give more support)
Add account attribute to action, so pipeline could detect
if action should be executed in foreign account, and perform
steps to simplify managing of cross-account deployments.
@rsmogura rsmogura force-pushed the cross-account-support branch from ea36531 to 9fdd5fd Compare January 17, 2019 22:39
@rsmogura
Copy link
Contributor Author

I would like to split this PR in following tasks (commits), it will make all work more transparent, and better organized

  • add ArtifactsStore with KMS support and pass KMS ARNs to CFN templates - ArtifactsStore will encapsulate bucket and KMS key and it's minimum for us, as we will be enabled to start cross-account deployments;
  • add support for account attribute on action;
  • add support for generating and managing KMS keys and S3 buckets, by whitelisting foreign accounts in resource policies;
  • narrowing access to KMS keys (probably using KMS key tags).

@sam-goodwin sam-goodwin added feature-request A feature should be added or improved. @aws-cdk/aws-codepipeline Related to AWS CodePipeline labels Jan 18, 2019
@rsmogura rsmogura closed this Jan 21, 2019
@rsmogura rsmogura deleted the cross-account-support branch January 21, 2019 03:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-codepipeline Related to AWS CodePipeline feature-request A feature should be added or improved.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants