Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(redshift-alpha): directly add parameters to a parameter group or indirectly through a cluster #20944

Merged
merged 25 commits into from
Sep 9, 2022
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
a8aa2f8
feat(redshift-alpha): methods to directly add parameters to a Cluster…
dontirun Jul 1, 2022
8852e7b
test: integration test updates
dontirun Jul 1, 2022
05727b8
Merge branch 'main' into dynamic-redshift-parameter-groups
dontirun Jul 1, 2022
5c5f98b
refactor: better description for generated Paramater Group
dontirun Jul 1, 2022
f2d5015
Merge branch 'dynamic-redshift-parameter-groups' of github.com:dontir…
dontirun Jul 1, 2022
1c8db26
refactor: spacing, naming, and change fromClusterParameterGroupName t…
dontirun Jul 15, 2022
fe97d47
Merge branch 'main' into dynamic-redshift-parameter-groups
dontirun Jul 26, 2022
c3e5e87
refactor: updating with feedback from the review
dontirun Aug 5, 2022
8ba1e4b
revert: Import base class change
Aug 5, 2022
37b1c9a
Apply suggestions from code review
dontirun Aug 6, 2022
1a7d327
refactor: add changes from code review
dontirun Aug 6, 2022
271e3e2
Merge branch 'main' into dynamic-redshift-parameter-groups
mergify[bot] Aug 6, 2022
e007770
fix: circular reference with parameter group description
Aug 6, 2022
0730a19
test: update integ tests
Aug 6, 2022
ad8140b
Merge branch 'main' into dynamic-redshift-parameter-groups
Aug 25, 2022
87bffb0
chore: update integ tests
Aug 25, 2022
9c3b28b
Merge branch 'main' into dynamic-redshift-parameter-groups
dontirun Aug 27, 2022
220f871
chore: update snapshots
Aug 27, 2022
2191f58
Update packages/@aws-cdk/aws-redshift/lib/cluster.ts
dontirun Sep 3, 2022
2a4fd32
update docs and test
dontirun Sep 3, 2022
b75471a
Merge branch 'main' into dynamic-redshift-parameter-groups
Sep 3, 2022
3621733
chore: update integ tests
Sep 3, 2022
482e9ec
refactor: suggested changes and snapshot updates
Sep 7, 2022
e36ab6b
refactor: error message for parameter group
Sep 7, 2022
fa4f365
Merge branch 'main' into dynamic-redshift-parameter-groups
mergify[bot] Sep 9, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions packages/@aws-cdk/aws-redshift/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -277,6 +277,27 @@ cluster.addRotationMultiUser('MultiUserRotation', {
});
```

## Adding Parameters

You can add a parameter to a parameter group with`ClusterParameterGroup.addParameter()`.

```ts
const params = new ClusterParameterGroup(stack, 'Params', {
description: 'desc',
parameters: {
require_ssl: 'true',
},
});

params.addParameter('enable_user_activity_logging', 'true');
```

Additionally, you can add a parameter to the cluster's associated parameter group with `Cluster.addToParameterGroup()`. If the cluster does not have an associated parameter group, a new parameter group is created.

```ts fixture=cluster
cluster.addToParameterGroup('enable_user_activity_logging', 'true');
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't rendering correctly. I suspect that the issue is the fixture=cluster piece causing it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll fix it for the example in this PR and raise an issue. Looking at the documentation it's also not rendering for the other examples using fixture=cluster

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I think none of the fixtures (even the default) are loading properly for the redshift alpha library

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


## Elastic IP

If you configure your cluster to be publicly accessible, you can optionally select an *elastic IP address* to use for the external IP address. An elastic IP address is a static IP address that is associated with your AWS account. You can use an elastic IP address to connect to your cluster from outside the VPC. An elastic IP address gives you the ability to change your underlying configuration without affecting the IP address that clients use to connect to your cluster. This approach can be helpful for situations such as recovery after a failure.
Expand Down
48 changes: 41 additions & 7 deletions packages/@aws-cdk/aws-redshift/lib/cluster.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import { Duration, IResource, RemovalPolicy, Resource, SecretValue, Token } from
import { Construct } from 'constructs';
import { DatabaseSecret } from './database-secret';
import { Endpoint } from './endpoint';
import { IClusterParameterGroup } from './parameter-group';
import { ClusterParameterGroup, IClusterParameterGroup } from './parameter-group';
import { CfnCluster } from './redshift.generated';
import { ClusterSubnetGroup, IClusterSubnetGroup } from './subnet-group';

Expand Down Expand Up @@ -350,6 +350,7 @@ export interface ClusterProps {
* A new or imported clustered database.
*/
abstract class ClusterBase extends Resource implements ICluster {

/**
* Name of the cluster
*/
Expand All @@ -374,6 +375,7 @@ abstract class ClusterBase extends Resource implements ICluster {
targetType: secretsmanager.AttachmentTargetType.REDSHIFT_CLUSTER,
};
}

dontirun marked this conversation as resolved.
Show resolved Hide resolved
}

/**
Expand All @@ -395,7 +397,6 @@ export class Cluster extends ClusterBase {
public readonly instanceIdentifiers: string[] = [];
public readonly clusterEndpoint = new Endpoint(attrs.clusterEndpointAddress, attrs.clusterEndpointPort);
}

return new Import(scope, id);
}

Expand Down Expand Up @@ -432,13 +433,24 @@ export class Cluster extends ClusterBase {
*/
private readonly vpcSubnets?: ec2.SubnetSelection;

/**
* The underlying CfnCluster
*/
private readonly cluster: CfnCluster;

dontirun marked this conversation as resolved.
Show resolved Hide resolved
/**
* The cluster's parameter group
*/
protected parameterGroup?: IClusterParameterGroup;

constructor(scope: Construct, id: string, props: ClusterProps) {
super(scope, id);

this.vpc = props.vpc;
this.vpcSubnets = props.vpcSubnets ?? {
subnetType: ec2.SubnetType.PRIVATE_WITH_NAT,
};
this.parameterGroup = props.parameterGroup;

const removalPolicy = props.removalPolicy ?? RemovalPolicy.RETAIN;

Expand Down Expand Up @@ -482,7 +494,7 @@ export class Cluster extends ClusterBase {
};
}

const cluster = new CfnCluster(this, 'Resource', {
this.cluster = new CfnCluster(this, 'Resource', {
// Basic
allowVersionUpgrade: true,
automatedSnapshotRetentionPeriod: 1,
Expand Down Expand Up @@ -511,15 +523,15 @@ export class Cluster extends ClusterBase {
elasticIp: props.elasticIp,
});

cluster.applyRemovalPolicy(removalPolicy, {
this.cluster.applyRemovalPolicy(removalPolicy, {
applyToUpdateReplacePolicy: true,
});

this.clusterName = cluster.ref;
this.clusterName = this.cluster.ref;

// create a number token that represents the port of the cluster
const portAttribute = Token.asNumber(cluster.attrEndpointPort);
this.clusterEndpoint = new Endpoint(cluster.attrEndpointAddress, portAttribute);
const portAttribute = Token.asNumber(this.cluster.attrEndpointPort);
this.clusterEndpoint = new Endpoint(this.cluster.attrEndpointAddress, portAttribute);

if (secret) {
this.secret = secret.attach(this);
Expand Down Expand Up @@ -592,4 +604,26 @@ export class Cluster extends ClusterBase {
return nodeCount;
}
}

/**
* Adds a parameter to the Clusters' parameter group
*
* @param name the parameter name
* @param value the parameter name
*/
public addToParameterGroup(name: string, value: string): void {
TheRealAmazonKendra marked this conversation as resolved.
Show resolved Hide resolved
if (!this.parameterGroup) {
const param: { [name: string]: string } = {};
param[name] = value;
this.parameterGroup = new ClusterParameterGroup(this, 'ParameterGroup', {
description: `Parameter Group for the ${this.cluster.clusterIdentifier ?? this.clusterName} Redshift cluster`,
parameters: param,
});
this.cluster.clusterParameterGroupName = this.parameterGroup.clusterParameterGroupName;
} else if (this.parameterGroup instanceof ClusterParameterGroup) {
this.parameterGroup.addParameter(name, value);
} else {
throw new Error('Cannot add a parameter to an imported parameter group.');
}
}
}
42 changes: 36 additions & 6 deletions packages/@aws-cdk/aws-redshift/lib/parameter-group.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ export interface IClusterParameterGroup extends IResource {
* A new cluster or instance parameter group
*/
abstract class ClusterParameterGroupBase extends Resource implements IClusterParameterGroup {

/**
* The name of the parameter group
*/
Expand Down Expand Up @@ -62,17 +63,46 @@ export class ClusterParameterGroup extends ClusterParameterGroupBase {
*/
public readonly clusterParameterGroupName: string;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't a parameter group name?


/**
* The parameters in the parameter group
*/
readonly parameters: { [name: string]: string };

/**
* The underlying CfnClusterParameterGroup
*/
private readonly resource: CfnClusterParameterGroup;

dontirun marked this conversation as resolved.
Show resolved Hide resolved
constructor(scope: Construct, id: string, props: ClusterParameterGroupProps) {
super(scope, id);

const resource = new CfnClusterParameterGroup(this, 'Resource', {
this.parameters = props.parameters;
this.resource = new CfnClusterParameterGroup(this, 'Resource', {
description: props.description || 'Cluster parameter group for family redshift-1.0',
parameterGroupFamily: 'redshift-1.0',
parameters: Object.entries(props.parameters).map(([name, value]) => {
return { parameterName: name, parameterValue: value };
}),
parameters: this.parseParameters(),
});

this.clusterParameterGroupName = resource.ref;
this.clusterParameterGroupName = this.resource.ref;
}
private parseParameters(): any {
return Object.entries(this.parameters).map(([name, value]) => {
return { parameterName: name, parameterValue: value };
});
}
TheRealAmazonKendra marked this conversation as resolved.
Show resolved Hide resolved

/**
* Adds a parameter to the parameter group
*
* @param name the parameter name
* @param value the parameter name
*/
public addParameter(name: string, value: string): void {
TheRealAmazonKendra marked this conversation as resolved.
Show resolved Hide resolved
const existingValue = Object.entries(this.parameters).find(([key, _]) => key === name)?.[1];
if (existingValue === undefined) {
this.parameters[name] = value;
this.resource.parameters = this.parseParameters();
} else if (existingValue !== value) {
throw new Error(`The parameter group already contains the parameter "${name}", but with a different value (Given: ${value}, Existing: ${existingValue}).`);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add the name of the parameter group in here in case the user has an app with multiple.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parameter groups don't have names, but I agree that it makes sense to identify it somehow , so maybe it makes sense to use Names.uniqueResourceName here

  throw new Error(`The parameter group id "${Names.uniqueResourceName(this, {})}" already contains the parameter "${name}", but with a different value (Given: ${value}, Existing: ${existingValue}).`);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my most recent comment below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case it's set to the Instrinsic Ref of CfnClusterParameterGroup, so the error message would contain a Token.

Considering that this is runtime error, I think using Names.uniqueResourceName (along with the stack trace) should be helpful enough to locate the problematic parameter group

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be providing the customer with inaccurate information. Sure, it's an approximation, but it's not actually the name of the resource. I suppose the stack trace will have to provide enough information in conjunction with the key value pair they're trying to add.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted back to this message in the latest commit

}
}
}
83 changes: 66 additions & 17 deletions packages/@aws-cdk/aws-redshift/test/cluster.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -253,27 +253,76 @@ test('create an encrypted cluster with custom KMS key', () => {
},
});
});
describe('parameter group', () => {
test('cluster instantiated with parameter group', () => {
// WHEN
const group = new ClusterParameterGroup(stack, 'Params', {
description: 'bye',
parameters: {
param: 'value',
},
});

new Cluster(stack, 'Redshift', {
masterUser: {
masterUsername: 'admin',
},
vpc,
parameterGroup: group,
});

// THEN
Template.fromStack(stack).hasResourceProperties('AWS::Redshift::Cluster', {
ClusterParameterGroupName: { Ref: 'ParamsA8366201' },
});

test('cluster with parameter group', () => {
// WHEN
const group = new ClusterParameterGroup(stack, 'Params', {
description: 'bye',
parameters: {
param: 'value',
},
});

new Cluster(stack, 'Redshift', {
masterUser: {
masterUsername: 'admin',
},
vpc,
parameterGroup: group,
test('adding to cluster parameter group on cluster not instantiated with parameter group', () => {

// WHEN
const cluster = new Cluster(stack, 'Redshift', {
clusterName: 'foobar',
masterUser: {
masterUsername: 'admin',
},
vpc,
});

cluster.addToParameterGroup('foo', 'bar');

const template = Template.fromStack(stack);
template.hasResourceProperties('AWS::Redshift::Cluster', {
ClusterParameterGroupName: { Ref: Match.anyValue() },
});

template.hasResourceProperties('AWS::Redshift::ClusterParameterGroup', {
Description: 'Parameter Group for the foobar Redshift cluster',
ParameterGroupFamily: 'redshift-1.0',
Parameters: [
{
ParameterName: 'foo',
ParameterValue: 'bar',
},
],
});
});

// THEN
Template.fromStack(stack).hasResourceProperties('AWS::Redshift::Cluster', {
ClusterParameterGroupName: { Ref: 'ParamsA8366201' },
test('Adding a parameter to an IClusterParameterGroup', () => {
// GIVEN
const cluster = new Cluster(stack, 'Redshift', {
clusterName: 'foobar',
parameterGroup: ClusterParameterGroup.fromClusterParameterGroupName(stack, 'Params', 'foo'),
masterUser: {
masterUsername: 'admin',
},
vpc,
});

// WHEN
expect(() => cluster.addToParameterGroup('param', 'value2'))
// THEN
.toThrowError('Cannot add a parameter to an imported parameter group');
});

});
Expand Down Expand Up @@ -372,7 +421,7 @@ test('throws validation error when trying to set encryptionKey without enabling

// THEN
expect(() => {
new Cluster(stack, 'Redshift', props );
new Cluster(stack, 'Redshift', props);
}).toThrowError();

});
Expand Down
Loading