(servicecatalog): ProductStack does not support Assets

[mackalex]

Describe the feature

The feature is an improvement to the existing ProductStack construct to add support for the use of asset files.

Use Case

I'm always frustrated as a Service Catalog administrator when I try to add a Lambda function to my ProductStack in CDK because I want to reference my Lambda code from an asset file, and CDK throws an error when I attempt to synthesize this. This limitation means that I'm unable to make use of ProductStack when I want to create a Service Catalog product consisting of Lambas that run large amounts of code. This is an example of a product which I would like to deploy to Service Catalog and share with end users across AWS accounts:

class ServerlessProduct extends sc.ProductStack {
  constructor(scope: cdk.Construct, id: string) {
    super(scope, id);

    // Defines an AWS Lambda resource
    const myHandler = new lambda.Function(this, 'Handler', {
      runtime: lambda.Runtime.NODEJS_14_X,
      code: lambda.Code.fromAsset(path.join(__dirname, 'handler')),
      handler: 'index.handler'
    });

    // Defines an API Gateway REST API resource backed by the handler function
    new apigw.LambdaRestApi(this, 'Endpoint', {
      handler: myHandler
    });
  }
}

cdk synth

Error thrown: Service Catalog Product Stacks cannot use Assets

Proposed Solution

Design as of 6/9/22 by @wanjacki, @mackalex

Currently, CDK vends an asset bucket during bootstrap-time to the customer's AWS account. This bucket can be used successfully for enabling file asset support in ProductStack with CFN outputs from the parent stack for both the S3 bucket name and object key. The major issue with this approach is that when sharing a Service Catalog portfolio across accounts, a product that makes use of file assets cannot be provisioned since the parent stack with the aforementioned outputs does not exist in the end-user account.

To solve this, we could implement the usage of a bespoke S3 Bucket to contain asset files from assets used in a Service Catalog ProductStack. The bucket could exist at the Service Catalog Portfolio level which is instantiated in a ProductStack's parent stack. A bespoke bucket for this use case allows us to control the naming of the bucket as well as its permissions. Controlling the bucket name is important, especially at synth-time, since this will be referenced by resources that use assets, such as a Lambda function which references Python code stored in an asset file in the S3 bucket. Controlling permissions on a bucket which contains assets is important within the framework of Service Catalog since the administrator of a Service Catalog portfolio shares this portfolio across AWS accounts with end users who make use of products which reference asset files.

Other Information

Additional design considerations:
We have considered making use of the bootstrap bucket which CDK vends to customers to hold assets used by resources in a ProductStack. This presents issues with cross-account sharing of Service Catalog portfolios since the assets bucket would require permissions for the end-user account to access an asset file used by a Provisioned Product, and appending permissions to an existing S3 bucket policy is nearly infeasible without overwriting the bucket policy. This is not a desirable experience, especially when the bucket policy being overwritten would be the bootstrap bucket used by many components of CDK.

Relates issues:
#20361

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

2.24.1 (build 585f9ca)

Environment details (OS name and version, etc.)

macOS Big Sur Version 11.6.5 (20G527)

[peterwoodworth]

This would be a great addition to make ProductStack more complete. Are you willing to make a PR for this and discuss based on that?

[mackalex]

Yes, a PR will be made for this issue for a more in depth review of the implementation.

[padaszewski]

Hi @mackalex,
Thx for taking care of this. Our team currently overrides the bucket policy and uses an own managed key for cross-account sharing from the assets bucket. This a really awful workaround, which makes the integration and production pipelines to be dependent on the update of the bucket policy and key policy (happens in the dev pipeline).
Do You have any orientation timeline for the PR?

[bendudz]

Hi

Has there been any movement on this feature please? I am trying to produce a Service Catalog product for our org to use a common EKS stack. I didn’t want to implement a workaround if this feature will be ready soon.

Many thanks for your effort on this.

[wanjacki]

@bendudz
We are looking at having a PR out by end of July and merged by mid August.

Edit/Update:
We are shooting for end of September now (We are getting alot of request for this so we are definitely prioritizing this as much as we can)

[philiphorrocks]

Any update on this? Would be great to get this feature as we have products which require lamba functions

[wanjacki]

@philiphorrocks We are working on it, I provided an update:

"We are shooting for end of September now (We are getting alot of request for this so we are definitely prioritizing this as much as we can)"

As a work around if your lambda is small enough you might be able to use the fromInline feature for now

[dennisfleischmann]

Hi is there already an update on that? Is it meanwhile supported? I receive the same issue on the following

`
class SelfserviceVpc(servicecatalog.ProductStack):
def init(self, scope: Construct, id: str, regional_ipam_pools: List[object],):
super().init(scope, id)

[...]
## Get CIDR
self.getcidr = cr.AwsCustomResource(self, "GetCidr",
on_create=cr.AwsSdkCall(
service='EC2',
action='getIpamPoolAllocations',
parameters = {
'IpamPoolAllocationId': cidr_allocation.attr_ipam_pool_allocation_id,
'IpamPoolId': ipam_pool.ref
},
physical_resource_id=cr.PhysicalResourceId.of("GetCidr")
),
policy=cr.AwsCustomResourcePolicy.from_sdk_calls(
resources=cr.AwsCustomResourcePolicy.ANY_RESOURCE
)
)
`

[marcus-j]

Hi there, is there a (new) timeline for this feature?

[wanjacki]

Hi Marcus as of now there's another PR that needs to be merged blocking #22857 . There's no new timeline for this feature as of now, we will try to get it in as soon as possible.

[padaszewski]

Hi @wanjacki, do You think that there might be a chance to merge this till the end of January? We are waiting for this for almost a year now and therefore constantly moving some of our releases. We want of course provide a clean implementation and not our temporary workarounds. It is hard to me from the GitHub conversations find a conclusion or orientation for us, thus the question.

[paulmowat]

I've also just hit this and having to try find workarounds to get it to work. If there is no ETA on when it will be ready. Do you have a recommended/suggested workaround for how to do this at the minute?

[wanjacki]

Well we got an approval from CDK team so hopefully we can get that merged and close this issue soon. End of year should be possible.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.