Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(servicecatalog): Add Product Stack Asset Support #22143

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions packages/@aws-cdk/aws-servicecatalog/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,73 @@ const product = new servicecatalog.CloudFormationProduct(this, 'Product', {
});
```

You can now reference assets in a Product Stack. For example, we can add a handler to a Lambda function or a S3 Asset directly from a local asset file.
wanjacki marked this conversation as resolved.
Show resolved Hide resolved
In this case, a S3 Bucket will automatically be generated for each ProductStack to store your assets based on your environment accountId and region.

```ts
import * as lambda from '@aws-cdk/aws-lambda';
import * as cdk from '@aws-cdk/core';

class LambdaProduct extends servicecatalog.ProductStack {
constructor(scope: Construct, id: string) {
super(scope, id);

new lambda.Function(this, 'LambdaProduct', {
runtime: lambda.Runtime.PYTHON_3_9,
code: lambda.Code.fromAsset("./assets"),
handler: 'index.handler'
});
}
}

const product = new servicecatalog.CloudFormationProduct(this, 'Product', {
productName: "My Product",
owner: "Product Owner",
productVersions: [
{
productVersionName: "v1",
cloudFormationTemplate: servicecatalog.CloudFormationTemplate.fromProductStack(new LambdaProduct(this, 'LambdaFunctionProduct')),
},
],
});
```

You can also create your own ProductStackAssetBucket to store assets, allowing you to provide one bucket for multiple Product Stacks.

```ts
import * as lambda from '@aws-cdk/aws-lambda';
import * as cdk from '@aws-cdk/core';

class LambdaProduct extends servicecatalog.ProductStack {
constructor(scope: Construct, id: string) {
super(scope, id);

new lambda.Function(this, 'LambdaProduct', {
runtime: lambda.Runtime.PYTHON_3_9,
code: lambda.Code.fromAsset("./assets"),
handler: 'index.handler'
});
}
}

const userDefinedBucket = new ProductStackAssetBucket(this, `UserDefinedBucket`, {
assetBucketName: 'user-defined-bucket-for-product-stack-assets',
});

const product = new servicecatalog.CloudFormationProduct(this, 'Product', {
productName: "My Product",
owner: "Product Owner",
productVersions: [
{
productVersionName: "v1",
cloudFormationTemplate: servicecatalog.CloudFormationTemplate.fromProductStack(new LambdaProduct(this, 'LambdaFunctionProduct', {
assetBucket: userDefinedBucket,
})),
},
],
});
```

### Creating a Product from a stack with a history of previous versions

The default behavior of Service Catalog is to overwrite each product version upon deployment.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { IBucket } from '@aws-cdk/aws-s3';
import * as s3_assets from '@aws-cdk/aws-s3-assets';
import { Construct } from 'constructs';
import { hashValues } from './private/util';
Expand Down Expand Up @@ -46,9 +47,16 @@ export abstract class CloudFormationTemplate {
*/
export interface CloudFormationTemplateConfig {
/**
* The http url of the template in S3.
*/
* The http url of the template in S3.
*/
readonly httpUrl: string;

/**
* The S3 bucket containing product stack assets.
* @default - None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does None mean? If this is not provided are assets not supported?

Copy link
Contributor Author

@wanjacki wanjacki Sep 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This class is the result of binding a Artifact Template into a Product.
In the event that it is not binding to a Product Stack (URL or local asset template instead), there is no asset bucket and no asset support.
In the event that they are binding a Product Stack, if they are not using assets, then the asset bucket will not be generated and will be undefined and there will be no asset bucket.
In any other case there is an asset bucket.

I will update to None - no assets are used in this product as well.

*/
readonly assetBucket?: IBucket;

}

/**
Expand Down Expand Up @@ -108,6 +116,7 @@ class CloudFormationProductStackTemplate extends CloudFormationTemplate {
public bind(_scope: Construct): CloudFormationTemplateConfig {
return {
httpUrl: this.productStack._getTemplateUrl(),
assetBucket: this.productStack._getAssetBucket(),
};
}
}
1 change: 1 addition & 0 deletions packages/@aws-cdk/aws-servicecatalog/lib/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ export * from './cloudformation-template';
export * from './portfolio';
export * from './product';
export * from './product-stack';
export * from './product-stack-asset-bucket';
export * from './product-stack-history';
export * from './tag-options';

Expand Down
39 changes: 34 additions & 5 deletions packages/@aws-cdk/aws-servicecatalog/lib/portfolio.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
import * as iam from '@aws-cdk/aws-iam';
import * as sns from '@aws-cdk/aws-sns';
import * as cdk from '@aws-cdk/core';
import { Construct } from 'constructs';
import { Construct, IConstruct } from 'constructs';
import * as iam from '../../aws-iam';
import { IBucket } from '../../aws-s3';
import * as sns from '../../aws-sns';
import * as cdk from '../../core';
wanjacki marked this conversation as resolved.
Show resolved Hide resolved
import { MessageLanguage } from './common';
import {
CloudFormationRuleConstraintOptions, CommonConstraintOptions,
Expand Down Expand Up @@ -105,7 +106,7 @@ export interface IPortfolio extends cdk.IResource {
* @param product A service catalog product.
* @param options options for the constraint.
*/
constrainCloudFormationParameters(product:IProduct, options: CloudFormationRuleConstraintOptions): void;
constrainCloudFormationParameters(product: IProduct, options: CloudFormationRuleConstraintOptions): void;

/**
* Force users to assume a certain role when launching a product.
Expand Down Expand Up @@ -155,6 +156,8 @@ abstract class PortfolioBase extends cdk.Resource implements IPortfolio {
public abstract readonly portfolioArn: string;
public abstract readonly portfolioId: string;
private readonly associatedPrincipals: Set<string> = new Set();
private readonly assetBuckets: Set<IBucket> = new Set<IBucket>();
private readonly sharedAccounts: String[] = [];
wanjacki marked this conversation as resolved.
Show resolved Hide resolved

public giveAccessToRole(role: iam.IRole): void {
this.associatePrincipal(role.roleArn, role.node.addr);
Expand All @@ -169,11 +172,15 @@ abstract class PortfolioBase extends cdk.Resource implements IPortfolio {
}

public addProduct(product: IProduct): void {
if (product.assetBucket) {
this.assetBuckets.add(product.assetBucket);
}
AssociationManager.associateProductWithPortfolio(this, product, undefined);
}

public shareWithAccount(accountId: string, options: PortfolioShareOptions = {}): void {
const hashId = this.generateUniqueHash(accountId);
this.sharedAccounts.push(accountId);
new CfnPortfolioShare(this, `PortfolioShare${hashId}`, {
portfolioId: this.portfolioId,
accountId: accountId,
Expand Down Expand Up @@ -236,6 +243,20 @@ abstract class PortfolioBase extends cdk.Resource implements IPortfolio {
}
}

/**
* Gives access to Asset Buckets to Shared Accounts.
*
* @internal
wanjacki marked this conversation as resolved.
Show resolved Hide resolved
*/
protected _addBucketPermissionsToSharedAccounts() {
if (this.sharedAccounts.length > 0) {
for (const bucket of this.assetBuckets) {
bucket.grantRead(new iam.CompositePrincipal(...this.sharedAccounts.map(account => new iam.AccountPrincipal(account))),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How are you granting access on the other side? The IAM role that gets from these buckets needs
access to be granted access to the bucket in its princial policy.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in my other comment, but there is not much more we can do. We made it as easy as possible but we don't have access to the accounts it is being shared with, it is up to the Admin to configure these spoke accounts.

);
}
}
}

/**
* Create a unique id based off the L1 CfnPortfolio or the arn of an imported portfolio.
*/
Expand Down Expand Up @@ -336,6 +357,14 @@ export class Portfolio extends PortfolioBase {
if (props.tagOptions !== undefined) {
this.associateTagOptions(props.tagOptions);
}

cdk.Aspects.of(this).add({
visit(c: IConstruct) {
if (c instanceof Portfolio) {
c._addBucketPermissionsToSharedAccounts();
};
},
});
}

protected generateUniqueHash(value: string): string {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
import * as cdk from '@aws-cdk/core';
import { ProductStack } from '../product-stack';
import { ProductStackAssetBucket } from '../product-stack-asset-bucket';
import { hashValues } from './util';

/**
* Deployment environment for an AWS Service Catalog product stack.
Expand All @@ -7,6 +10,12 @@ import * as cdk from '@aws-cdk/core';
*/
export class ProductStackSynthesizer extends cdk.StackSynthesizer {
private stack?: cdk.Stack;
private assetBucket?: ProductStackAssetBucket;

constructor(assetBucket?: ProductStackAssetBucket) {
super();
this.assetBucket = assetBucket;
}

public bind(stack: cdk.Stack): void {
if (this.stack !== undefined) {
Expand All @@ -16,7 +25,17 @@ export class ProductStackSynthesizer extends cdk.StackSynthesizer {
}

public addFileAsset(_asset: cdk.FileAssetSource): cdk.FileAssetLocation {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
public addFileAsset(_asset: cdk.FileAssetSource): cdk.FileAssetLocation {
public addFileAsset(asset: cdk.FileAssetSource): cdk.FileAssetLocation {

throw new Error('Service Catalog Product Stacks cannot use Assets');
if (!this.stack) {
throw new Error('You must call bindStack() first');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this message will reach the user. Do you mean to tell your user to call bindStack()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the error we throw in our synthesize method as well. I'm not sure about this, I don't think this is an error the user can act on regardless. I don't think we can ever reach this state, so maybe this error message is just for our own sanity.

}

if (!this.assetBucket) {
const parentStack = (this.stack as ProductStack)._getParentStack();
this.assetBucket = new ProductStackAssetBucket(parentStack, `ProductStackAssetBucket${hashValues(this.stack.stackName)}`);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how I feel about the way we are creating the asset bucket here. What if we just created
the asset bucket by default as part of the product stack?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean we create a ProductStackAssetBucket regardless of if the user is using an asset or not.
I don't that is a good idea since we would be creating resources that will not be needed (S3-deployment also deploys a lambda). Furthermore, a user could be unintentionally create a bunch of S3 Buckets (since it is one per product-stack) and fill up their S3 Bucket limit capacity.


(this.stack as ProductStack)._setAssetBucket(this.assetBucket._getBucket());
return this.assetBucket._addAsset(_asset);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is where we should handle the asset deployment, rather than in the bucket construct.

}

public addDockerImageAsset(_asset: cdk.DockerImageAssetSource): cdk.DockerImageAssetLocation {
Expand Down
114 changes: 114 additions & 0 deletions packages/@aws-cdk/aws-servicecatalog/lib/product-stack-asset-bucket.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
import { IBucket, BlockPublicAccess, Bucket, BucketEncryption } from '@aws-cdk/aws-s3';
import { BucketDeployment, ISource, Source } from '@aws-cdk/aws-s3-deployment';
import * as cdk from '@aws-cdk/core';
import { Construct, IConstruct } from 'constructs';
import { hashValues } from './private/util';
/**
* Product stack asset bucket props.
*/
export interface ProductStackAssetBucketProps {
/**
* Name of s3 asset bucket deployed
*
* @default - None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default is not none... a bucket name is generated if this property is not supplied.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can be more descriptive here since you know what the generated name will be (or atleast
the format).

*/
readonly assetBucketName?: string;
}

/**
* A Service Catalog product stack asset bucket, which is similar in form to an Amazon S3 bucket.
* You can store multiple product stack assets and collectively deploy them to S3.
*/
export class ProductStackAssetBucket extends Construct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should extends s3.Bucket.

private readonly bucketName: string;
private readonly bucket: IBucket;
private readonly assets: ISource[];

constructor(scope: Construct, id: string, props: ProductStackAssetBucketProps = {}) {
super(scope, id);

if (props.assetBucketName) {
this.bucketName = props.assetBucketName;
} else {
this.bucketName = this.generateBucketName(id);
}

this.bucket = new Bucket(scope, `${id}S3Bucket`, {
bucketName: this.bucketName,
blockPublicAccess: BlockPublicAccess.BLOCK_ALL,
encryption: BucketEncryption.KMS,
removalPolicy: cdk.RemovalPolicy.DESTROY,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is DESTROY the correct removalPolicy? If the bucket is removed for any reason all consuming stacks
will break. I think the safer default will be to RETAIN.

autoDeleteObjects: true,
kaizencc marked this conversation as resolved.
Show resolved Hide resolved
});

this.assets = [];

cdk.Aspects.of(this).add({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the reasoning behind using cdk.Aspects to run deploy assets? not questioning, just wondering.

Copy link
Contributor Author

@wanjacki wanjacki Sep 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be done without Aspects I believe, but Aspects fits the use case and allows us to cleanly implement our use case as well as abstract the additional code to the new product-stack-asset-bucket construct rather than modifying and adding additional code to product-stack.

visit(c: IConstruct) {
if (c instanceof ProductStackAssetBucket) {
c._deployAssets();
};
},
});
}

/**
* Fetch the S3 bucket.
*
* @internal
*/
public _getBucket(): IBucket {
return this.bucket;
}

/**
* Generate unique name for S3 bucket.
*
* @internal
*/
private generateBucketName(id: string): string {
wanjacki marked this conversation as resolved.
Show resolved Hide resolved
const accountId = cdk.Stack.of(this).account;
const region = cdk.Stack.of(this).region;
if (cdk.Token.isUnresolved(accountId)) {
throw new Error('CDK Account ID must be defined in the application environment');
}
if (cdk.Token.isUnresolved(region)) {
throw new Error('CDK Region must be defined in the application environment');
}
return `product-stack-asset-bucket-${accountId}-${region}-${hashValues(id)}`;
}

/**
* Fetch the expected S3 location of an asset.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is doing more than fetch right? adds an asset to this.assets

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to be a bit more descriptive.

*
* @internal
*/
public _addAsset(asset: cdk.FileAssetSource): cdk.FileAssetLocation {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the asset management should be done in this bucket construct. These assets should be
handled as normal stack assets.

const assetPath = './cdk.out/' + asset.fileName;
this.assets.push(Source.asset(assetPath));

const bucketName = this.bucketName;
const s3Filename = asset.fileName?.split('.')[1] + '.zip';
const objectKey = `${s3Filename}`;
const s3ObjectUrl = `s3://${bucketName}/${objectKey}`;
const httpUrl = `https://s3.${bucketName}/${objectKey}`;

return { bucketName, objectKey, httpUrl, s3ObjectUrl, s3Url: httpUrl };
}

/**
* Deploy all assets to S3.
*
* @internal
*/
private _deployAssets() {
wanjacki marked this conversation as resolved.
Show resolved Hide resolved
if (this.assets.length > 0) {
new BucketDeployment(this, 'AssetsBucketDeployment', {
sources: this.assets,
destinationBucket: this.bucket,
extract: false,
prune: false,
});
}
}
}
Loading