Deploy AWS CDK stacks using CodePipeline

[kennu]

I would like to build my AWS CDK stack in CodeBuild using cdk synth into an artifact, and then deploy the artifact with CodePipeline using its built-in CloudFormation deployment feature.

When I try to do that, I get an error about missing parameters. I can't figure out how to pass the right parameters to CodePipeline. It seems that AWS CDK only generates the required parameters when using cdk deploy.

The parameters are like:

AssetParameters66eaa5211f0e5a7e13085d03b9a2660bca429dd36d6f2ff63caaad265fa82d05ArtifactHash42F81164
AssetParameters66eaa5211f0e5a7e13085d03b9a2660bca429dd36d6f2ff63caaad265fa82d05S3Bucket8B26DF3A
AssetParameters66eaa5211f0e5a7e13085d03b9a2660bca429dd36d6f2ff63caaad265fa82d05S3VersionKeyC231E1D0

I have a project where a single Git repository builds 13 CloudFormation stacks. It would be nice to use CodePipeline to deploy them, instead of trying to handle all the cdk deploy processes in CodeBuild. It seems like the right way to do deploy CloudFormation stacks anyway.

Maybe there already is some way to pass these parameters from CDK to CodePipeline?

---

This is a 🚀 Feature Request

[vineethsoma]

I am currently working through this scenario too. I was thinking of doing the following in Code Pipeline -

1. Do cdk bootstrap against deployment account in a Code Build Action. This will resources to stage your lambda assets.

2. Do cdk deploy <stackName> after bootstrap in a another Code Build Action.

In step 2, I want to see if I can do cdk synth which spits out an assets.json file with assets to templete parameter mappings and templates. And use this infra package with cdk deploy <stackName> to build out my infra.

It would be nice as you mentioned if Code Pipeline has native actions to do CDK deploys.

[kennu]

When thinking about this further, I guess the problem is also that cdk synth does not upload Lambda function code to S3 yet. A perfect integration with CodePipeline would probably output the Lambda function contents as artifacts, and let CodePipeline upload them to S3, and then pass the necessary values to the CloudFormation stack parameters.

[vineethsoma]

I know they are working on the assets publishing capability which is part of the puzzle - https://github.com/aws/aws-cdk-rfcs/blob/master/text/0092-asset-publishing.md

I haven't quite figured out if it can be used yet.

[vineethsoma]

@kennu So I ran into a different issue trying to do cdk deploy of a stack containing Lambdas with API Gateway -
The LambdaIntegration tries to give invoke permissions which requires approval from a user to move forward. I tried using --require-approval=never flag but that just throws -
Stack [myStack] does not exist

I think I am going to take the Code Pipeline approach laid out here and create a Lambda stack and Build Artifact job per lambda.

Hope this helps!

[skinny85]

Hey @kennu and @vineethsoma ,

thanks for opening and commenting on the issue. You've pretty much figured out everything I wanted to say by yourselves, but let me say it anyway 🙂.

The problem you encountered @kennu is indeed because you're using assets in your code, and assets (currently!) are modeled in the CDK using CloudFormation parameters. Like @vineethsoma said, we're working hard on making applications using assets deployable through CodePipeline, but until we release that functionality, you have to forego using assets in all stacks that you want to deploy through CodePipeline. Looking at your asset parameters, I'm guessing you're using assets for the code property of a Lambda function. If that is indeed the case, take a look at this example (that @vineethsoma already mentioned):

https://github.com/skinny85/cdk-codepipeline-and-local-lambda-guidance

Which shows you how to use assets for local development of Lambdas, but then using a different code implementation that's specifically tailored to CodePipeline.

Hope this makes it clear!

Thanks,
Adam

[kennu]

Okay thanks, I will wait for the support to come out. I think until then I can just run cdk deploy in CodeBuild.

I realized, though, that my current use case involves quite a few Lambda functions (total 150 functions in 13 CloudFormation stacks). It would be a lot of work to configure 150 separate CodePipeline S3 artifacts and deployments. Also adding a new Lambda function would require modifying the Pipeline every time. So I'm not sure if that would be the best approach in this case.

Configuring 13 different CloudFormation template artifacts and deployments, on the other hand, seems reasonable. They would stay pretty static.

[skinny85]

Let me ask you this: are all 150 of those Lambdas completely separate? Like, does each of them have their own package.json / pom.xml / setup.py / ... (depending on the programming language they're written in)?

[kennu]

In this particular project there's about 50 separate code asset folders so that 3 Lambda functions share one asset folder. It's a bit special because we are running some tests with many different Lambda runtime configurations (AWS CDK is very convenient for configuring them).

In my typical projects there tends to be only one code asset folder shared by all the Lambda functions in a CloudFormation stack.

[skinny85]

In this particular project there's about 50 separate code asset folders so that 3 Lambda functions share one asset folder. It's a bit special because we are running some tests with many different Lambda runtime configurations (AWS CDK is very convenient for configuring them).

Ok. In that case, you only need 50 CodePipeline Artifacts / CodeBuild Projects ;p Still not great, but better than 150...

[skinny85]

In my typical projects there tends to be only one code asset folder shared by all the Lambda functions in a CloudFormation stack.

For those, you only need a single artifact / CodeBuild Project.

[SomayaB]

Closing this issue for now. Feel free to reopen if you have more questions.

[christophgysin]

This is now possible with the introduction of CDK pipelines:
https://docs.aws.amazon.com/cdk/api/latest/docs/pipelines-readme.html