Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Creating Aurora Cluster with Postgresql fails #5126

Closed
vertti opened this issue Nov 20, 2019 · 7 comments · Fixed by #8686
Closed

Creating Aurora Cluster with Postgresql fails #5126

vertti opened this issue Nov 20, 2019 · 7 comments · Fixed by #8686
Assignees
Labels
@aws-cdk/aws-rds Related to Amazon Relational Database effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. in-progress This issue is being actively worked on.

Comments

@vertti
Copy link

vertti commented Nov 20, 2019

I expected that by just changing engine and engineVersion I would get a Postgresql Cluster.

Reproduction Steps

Running CDK with db-stack that contains:

    this.cluster = new DatabaseCluster(this, 'ProductDatabase', {
      defaultDatabaseName: 'ProductDb',
      engine: DatabaseClusterEngine.AURORA_POSTGRESQL,
      engineVersion: '11.4',
      masterUser: {
        username: 'clusteradmin',
      },
      instanceProps: {
        instanceType: InstanceType.of(InstanceClass.BURSTABLE2, InstanceSize.SMALL),
        vpcSubnets: {
          subnetType: SubnetType.ISOLATED,
        },
        vpc: props.vpc,
      },
    })

fails with:

The Parameter Group default.aurora5.6 with DBParameterGroupFamily aurora5.6 cannot be used for this instance. Please use a Parameter Group with DBParameterGroupFamily aurora-postgresql11 (Service: AmazonRDS; Status Code: 400; Error Code: InvalidParameterCombination; Request ID: 4ced8114-0f04-4659-9cde-c214db8638fa)

Clearly it's not supposed to pick aurora5.6 group family with postgresql11 but I have no idea what I'm doing wrong so I expect this to be a bug.

Error Log

  6/14 | 10:02:08 AM | CREATE_FAILED        | AWS::RDS::DBCluster                         | ProductDatabase (ProductDatabase9844B02C) The Parameter Group default.aurora5.6 with DBParameterGroupFamily aurora5.6 cannot be used for this instance. Please use a Parameter Group with DBParameterGroupFamily aurora-postgresql11 (Service: AmazonRDS; Status Code: 400; Error Code: InvalidParameterCombination; Request ID: 4ced8114-0f04-4659-9cde-c214db8638fa)
	new DatabaseCluster (/home/vertti/dev/kesko/k-ruoka-product-service/node_modules/@aws-cdk/aws-rds/lib/cluster.ts:315:21)
	\_ new ProductDbStack (/home/vertti/dev/kesko/k-ruoka-product-service/lib/db-stack.ts:17:20)
	\_ Object.<anonymous> (/home/vertti/dev/kesko/k-ruoka-product-service/bin/product-service.ts:11:17)
	\_ Module._compile (internal/modules/cjs/loader.js:778:30)
	\_ Module.m._compile (/home/vertti/dev/kesko/k-ruoka-product-service/node_modules/ts-node/src/index.ts:530:23)
	\_ Module._extensions..js (internal/modules/cjs/loader.js:789:10)
	\_ Object.require.extensions.(anonymous function) [as .ts] (/home/vertti/dev/kesko/k-ruoka-product-service/node_modules/ts-node/src/index.ts:533:12)
	\_ Module.load (internal/modules/cjs/loader.js:653:32)
	\_ tryModuleLoad (internal/modules/cjs/loader.js:593:12)
	\_ Function.Module._load (internal/modules/cjs/loader.js:585:3)
	\_ Function.Module.runMain (internal/modules/cjs/loader.js:831:12)

Environment

  • CLI Version : 1.5.0
  • Framework Version: 1.5.0
  • OS : Ubuntu
  • Language : Typescript

This is 🐛 Bug Report

@vertti vertti added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Nov 20, 2019
@vertti
Copy link
Author

vertti commented Nov 20, 2019

Adding the folllowing to the cluster definition:

 parameterGroup: ParameterGroup.fromParameterGroupName(this, 'ParameterGroup', 'default.aurora-postgresql11'),

gets you past the error message above. This should really by described in the documentation!

But after adding that I get a new error message:

RDS does not support creating a DB instance with the following combination: DBInstanceClass=db.t2.small, Engine=aurora-postgresql, EngineVersion=11.4, LicenseModel=postgresql-license. For supported combinations of instance class and database engine version, see the documentation. (Service: AmazonRDS; Status Code: 400; Error Code: InvalidParameterCombination; Request ID: dfb07895-5687-486e-a39e-49b0e59a170f)

From what I can find from AWS documentation, 11.4 is supported and db.t2.small is allowed for all Postgreqsql versions.

@vertti
Copy link
Author

vertti commented Nov 20, 2019

Finally found the correct table: according to this, Aurora Postgresql supports only t3.medium instances.

So I would add to the documentation:

  • explanation on using the ParameterGroup for creating Postgresql
  • link to the above tables to see what the instance types should be

@SomayaB SomayaB added the @aws-cdk/aws-rds Related to Amazon Relational Database label Nov 21, 2019
@skinny85
Copy link
Contributor

skinny85 commented Nov 26, 2019

Thanks for reporting @vertti .

@jogold I wonder whether we can improve the DatabaseCluster (EDIT: actually, more DatabaseClusterEngine) construct, so that if parameterGroup was not specified, we will default it based on the engine that was passed - in this example, for instance, we see that the engine is AURORA_POSTGRESQL, and default the parameter group name to default.aurora-postgresql11. We can add a method to the DatabaseClusterEngine class to do that.

Thoughts?

@skinny85 skinny85 added feature-request A feature should be added or improved. and removed bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Nov 26, 2019
@jogold
Copy link
Contributor

jogold commented Nov 27, 2019

@skinny85 related #2213 and #2512

The CFN doc: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-rds-dbcluster.html#cfn-rds-dbcluster-dbclusterparametergroupname

If this argument is omitted, default.aurora5.6 is used. If default.aurora5.6 is used, specifying aurora-mysql or aurora-postgresql for the Engine property might result in an error.

So the right approach seems to be default.<engine><version> but version is optional, do you want to hard code default versions?

@skinny85
Copy link
Contributor

skinny85 commented Dec 8, 2019

So the right approach seems to be default. but version is optional, do you want to hard code default versions?

Yep. I think hard-coding is better in this case than failing at CFN deployment time.

skinny85 added a commit to skinny85/aws-cdk that referenced this issue Jan 8, 2020
The method onStateChange() in Action,
the common superclass for all CDK-vended actions,
incorrectly set the `detailType` of the CloudWatch event rule it created to
'CodePipeline Stage Execution State Change',
when it should be 'CodePipeline Action Execution State Change'.

Fixes aws#5126
@skinny85
Copy link
Contributor

skinny85 commented Jan 8, 2020

I flagged the wrong issues in #5721 , please ignore 🙂

@skinny85 skinny85 assigned nija-at and unassigned skinny85 Feb 6, 2020
@nija-at nija-at added the effort/medium Medium work item – several days of effort label Feb 17, 2020
@konstantinj
Copy link

fyi: I'm having the same issue + also the port is 3306 instead of 5432 when choosing DatabaseClusterEngine.AURORA_POSTGRESQL and adding the mentioned parameterGroup.

skinny85 added a commit to skinny85/aws-cdk that referenced this issue Jun 6, 2020
Change the types of the engine from DatabaseClusterEngine and its subclass,
DatabaseInstanceEngine, to 2 separate interfaces,
IClusterEngine and IInstanceEngine.
Add properties to each of them,
including engineVersion,
and thus stop taking engineVersion separately as a property for instances,
clusters and OptionGroups.
Allow changing the version of an existing engine to an arbitrary string.
Add a bind()-like protocol to both IClusterEngine and IInstanceEngine,
which allows expressing validation and default values logic directly in the engines,
instead of hard-coding them in cluster and instance.
Because of those changes, creating a default cluster with Aurora MySQL or Postgres engines now works out of the box,
instead of failing at deploy time.
We also correctly set the port for Postgres clusters to 5432,
instead of setting it to 3306 (which is the MySQL port).

Fixes aws#2213
Fixes aws#2512
Fixes aws#4150
Fixes aws#5126
Fixes aws#6532
Fixes aws#7072

BREAKING CHANGE: the class DatabaseClusterEngine has been replaced with the interface IClusterEngine in the type of DatabaseClusterProps.engine
* **rds**: the class DatabaseInstanceEngine has been replaced with the interface IInstanceEngine in the type of DatabaseInstanceSourceProps.engine
* **rds**: DatabaseClusterProps.engineVersion has been removed, in favor of the withVersion() method on IClusterEngine
* **rds**: DatabaseInstanceNewProps.instanceClass has been renamed to instanceType
* **rds**: DatabaseInstanceSourceProps.engineVersion has been removed, in favor of the withVersion() method on IInstanceEngine
* **rds**: the engine property can no longer be passed when creating a DatabaseInstanceReadReplica
* **rds**: the property majorEngineVersion can no longer be passed when creating an OptionGroup (instead, the version is taken directly from the passed engine)
@SomayaB SomayaB added the in-progress This issue is being actively worked on. label Jun 8, 2020
skinny85 added a commit to skinny85/aws-cdk that referenced this issue Jun 22, 2020
Change the types of the engine from DatabaseClusterEngine and its subclass,
DatabaseInstanceEngine, to 2 separate interfaces,
IClusterEngine and IInstanceEngine.
Add properties to each of them,
including engineVersion,
and thus stop taking engineVersion separately as a property for instances,
clusters and OptionGroups.
Allow changing the version of an existing engine to an arbitrary string.
Add a bind()-like protocol to both IClusterEngine and IInstanceEngine,
which allows expressing validation and default values logic directly in the engines,
instead of hard-coding them in cluster and instance.
Because of those changes, creating a default cluster with Aurora MySQL or Postgres engines now works out of the box,
instead of failing at deploy time.
We also correctly set the port for Postgres clusters to 5432,
instead of leaving it as the default 3306 (which is the MySQL port).

Fixes aws#2213
Fixes aws#2512
Fixes aws#4150
Fixes aws#5126
Fixes aws#7072

BREAKING CHANGE: the class DatabaseClusterEngine has been replaced with the interface IClusterEngine in the type of DatabaseClusterProps.engine
* **rds**: the class DatabaseInstanceEngine has been replaced with the interface IInstanceEngine in the type of DatabaseInstanceSourceProps.engine
* **rds**: DatabaseClusterProps.engineVersion has been removed; instead, create an IClusterEngine with a specific version using the static factory methods in DatabaseClusterEngine
* **rds**: DatabaseInstanceSourceProps.engineVersion has been removed; instead, create an IInstanceEngine with a specific version using the static factory methods in DatabaseInstanceEngine
* **rds**: the property majorEngineVersion can no longer be passed when creating an OptionGroup; instead, create an IInstanceEngine with a specific version using the static factory methods in DatabaseInstanceEngine
skinny85 added a commit to skinny85/aws-cdk that referenced this issue Jun 22, 2020
Change the types of the engine from DatabaseClusterEngine and its subclass,
DatabaseInstanceEngine, to 2 separate interfaces,
IClusterEngine and IInstanceEngine.
Add properties to each of them,
including engineVersion,
and thus stop taking engineVersion separately as a property for instances,
clusters and OptionGroups.
Allow changing the version of an existing engine to an arbitrary string.
Add a bind()-like protocol to both IClusterEngine and IInstanceEngine,
which allows expressing validation and default values logic directly in the engines,
instead of hard-coding them in cluster and instance.
Because of those changes, creating a default cluster with Aurora MySQL or Postgres engines now works out of the box,
instead of failing at deploy time.
We also correctly set the port for Postgres clusters to 5432,
instead of leaving it as the default 3306 (which is the MySQL port).

Fixes aws#2213
Fixes aws#2512
Fixes aws#4150
Fixes aws#5126
Fixes aws#7072

BREAKING CHANGE: the class DatabaseClusterEngine has been replaced with the interface IClusterEngine in the type of DatabaseClusterProps.engine
* **rds**: the class DatabaseInstanceEngine has been replaced with the interface IInstanceEngine in the type of DatabaseInstanceSourceProps.engine
* **rds**: DatabaseClusterProps.engineVersion has been removed; instead, create an IClusterEngine with a specific version using the static factory methods in DatabaseClusterEngine
* **rds**: DatabaseInstanceSourceProps.engineVersion has been removed; instead, create an IInstanceEngine with a specific version using the static factory methods in DatabaseInstanceEngine
* **rds**: the property majorEngineVersion can no longer be passed when creating an OptionGroup; instead, create an IInstanceEngine with a specific version using the static factory methods in DatabaseInstanceEngine
skinny85 added a commit to skinny85/aws-cdk that referenced this issue Jun 24, 2020
Change the types of the engine from DatabaseClusterEngine and its subclass,
DatabaseInstanceEngine, to 2 separate interfaces,
IClusterEngine and IInstanceEngine.
Add properties to each of them,
including engineVersion,
and thus stop taking engineVersion separately as a property for instances,
clusters and OptionGroups.
Allow changing the version of an existing engine to an arbitrary string.
Add a bind()-like protocol to both IClusterEngine and IInstanceEngine,
which allows expressing validation and default values logic directly in the engines,
instead of hard-coding them in cluster and instance.
Because of those changes, creating a default cluster with Aurora MySQL or Postgres engines now works out of the box,
instead of failing at deploy time.
We also correctly set the port for Postgres clusters to 5432,
instead of leaving it as the default 3306 (which is the MySQL port).

Fixes aws#2213
Fixes aws#2512
Fixes aws#4150
Fixes aws#5126
Fixes aws#7072

BREAKING CHANGE: the class DatabaseClusterEngine has been replaced with the interface IClusterEngine in the type of DatabaseClusterProps.engine
* **rds**: the class DatabaseInstanceEngine has been replaced with the interface IInstanceEngine in the type of DatabaseInstanceSourceProps.engine
* **rds**: DatabaseClusterProps.engineVersion has been removed; instead, create an IClusterEngine with a specific version using the static factory methods in DatabaseClusterEngine
* **rds**: DatabaseInstanceSourceProps.engineVersion has been removed; instead, create an IInstanceEngine with a specific version using the static factory methods in DatabaseInstanceEngine
* **rds**: the property majorEngineVersion can no longer be passed when creating an OptionGroup; instead, create an IInstanceEngine with a specific version using the static factory methods in DatabaseInstanceEngine
skinny85 added a commit to skinny85/aws-cdk that referenced this issue Jun 26, 2020
Change the types of the engine from DatabaseClusterEngine and its subclass,
DatabaseInstanceEngine, to 2 separate interfaces,
IClusterEngine and IInstanceEngine.
Add properties to each of them,
including engineVersion,
and thus stop taking engineVersion separately as a property for instances,
clusters and OptionGroups.
Allow changing the version of an existing engine to an arbitrary string.
Add a bind()-like protocol to both IClusterEngine and IInstanceEngine,
which allows expressing validation and default values logic directly in the engines,
instead of hard-coding them in cluster and instance.
Because of those changes, creating a default cluster with Aurora MySQL or Postgres engines now works out of the box,
instead of failing at deploy time.
We also correctly set the port for Postgres clusters to 5432,
instead of leaving it as the default 3306 (which is the MySQL port).

Fixes aws#2213
Fixes aws#2512
Fixes aws#4150
Fixes aws#5126
Fixes aws#7072

BREAKING CHANGE: the class DatabaseClusterEngine has been replaced with the interface IClusterEngine in the type of DatabaseClusterProps.engine
* **rds**: the class DatabaseInstanceEngine has been replaced with the interface IInstanceEngine in the type of DatabaseInstanceSourceProps.engine
* **rds**: DatabaseClusterProps.engineVersion has been removed; instead, create an IClusterEngine with a specific version using the static factory methods in DatabaseClusterEngine
* **rds**: DatabaseInstanceSourceProps.engineVersion has been removed; instead, create an IInstanceEngine with a specific version using the static factory methods in DatabaseInstanceEngine
* **rds**: the property majorEngineVersion can no longer be passed when creating an OptionGroup; instead, create an IInstanceEngine with a specific version using the static factory methods in DatabaseInstanceEngine
skinny85 added a commit to skinny85/aws-cdk that referenced this issue Jul 7, 2020
Change the types of the engine from DatabaseClusterEngine and its subclass,
DatabaseInstanceEngine, to 2 separate interfaces,
IClusterEngine and IInstanceEngine.
Add properties to each of them,
including engineVersion,
and thus stop taking engineVersion separately as a property for instances,
clusters and OptionGroups.
Allow changing the version of an existing engine to an arbitrary string.
Add a bind()-like protocol to both IClusterEngine and IInstanceEngine,
which allows expressing validation and default values logic directly in the engines,
instead of hard-coding them in cluster and instance.
Because of those changes, creating a default cluster with Aurora MySQL or Postgres engines now works out of the box,
instead of failing at deploy time.
We also correctly set the port for Postgres clusters to 5432,
instead of leaving it as the default 3306 (which is the MySQL port).

Fixes aws#2213
Fixes aws#2512
Fixes aws#4150
Fixes aws#5126
Fixes aws#7072

BREAKING CHANGE: the class DatabaseClusterEngine has been replaced with the interface IClusterEngine in the type of DatabaseClusterProps.engine
* **rds**: the class DatabaseInstanceEngine has been replaced with the interface IInstanceEngine in the type of DatabaseInstanceSourceProps.engine
* **rds**: DatabaseClusterProps.engineVersion has been removed; instead, create an IClusterEngine with a specific version using the static factory methods in DatabaseClusterEngine
* **rds**: DatabaseInstanceSourceProps.engineVersion has been removed; instead, create an IInstanceEngine with a specific version using the static factory methods in DatabaseInstanceEngine
* **rds**: the property majorEngineVersion can no longer be passed when creating an OptionGroup; instead, create an IInstanceEngine with a specific version using the static factory methods in DatabaseInstanceEngine
skinny85 added a commit to skinny85/aws-cdk that referenced this issue Jul 9, 2020
Change the types of the engine from DatabaseClusterEngine and its subclass,
DatabaseInstanceEngine, to 2 separate interfaces,
IClusterEngine and IInstanceEngine.
Add properties to each of them,
including engineVersion,
and thus stop taking engineVersion separately as a property for instances,
clusters and OptionGroups.
Allow changing the version of an existing engine to an arbitrary string.
Add a bind()-like protocol to both IClusterEngine and IInstanceEngine,
which allows expressing validation and default values logic directly in the engines,
instead of hard-coding them in cluster and instance.
Because of those changes, creating a default cluster with Aurora MySQL or Postgres engines now works out of the box,
instead of failing at deploy time.
We also correctly set the port for Postgres clusters to 5432,
instead of leaving it as the default 3306 (which is the MySQL port).

Fixes aws#2213
Fixes aws#2512
Fixes aws#4150
Fixes aws#5126
Fixes aws#7072

BREAKING CHANGE: the class DatabaseClusterEngine has been replaced with the interface IClusterEngine in the type of DatabaseClusterProps.engine
* **rds**: the class DatabaseInstanceEngine has been replaced with the interface IInstanceEngine in the type of DatabaseInstanceSourceProps.engine
* **rds**: DatabaseClusterProps.engineVersion has been removed; instead, create an IClusterEngine with a specific version using the static factory methods in DatabaseClusterEngine
* **rds**: DatabaseInstanceSourceProps.engineVersion has been removed; instead, create an IInstanceEngine with a specific version using the static factory methods in DatabaseInstanceEngine
* **rds**: the property majorEngineVersion can no longer be passed when creating an OptionGroup; instead, create an IInstanceEngine with a specific version using the static factory methods in DatabaseInstanceEngine
@mergify mergify bot closed this as completed in #8686 Jul 9, 2020
mergify bot pushed a commit that referenced this issue Jul 9, 2020
Change the types of the engine from `DatabaseClusterEngine` and its subclass, `DatabaseInstanceEngine`, to 2 separate interfaces, `IClusterEngine` and `IInstanceEngine`. Add properties to each of them, including engineVersion, and thus stop taking `engineVersion` separately as a property for instances, clusters and `OptionGroup`s. Allow creating engines with the version given as an arbitrary string.

Add a `bind()`-like protocol to both `IClusterEngine` and `IInstanceEngine`, which allows expressing validation and default values logic directly in the engines, instead of hard-coding them in cluster and instance.

Because of those changes, creating a default cluster with Aurora MySQL or Postgres engines now works out of the box, instead of failing at deploy time. We also correctly set the port for Postgres clusters to 5432, instead of leaving it as the default 3306 (which is the MySQL port).

Fixes #2213
Fixes #2512
Fixes #4150
Fixes #5126
Fixes #7072

BREAKING CHANGE: the class `DatabaseClusterEngine` has been replaced with the interface `IClusterEngine` in the type of `DatabaseClusterProps.engine`
* **rds**: the class `DatabaseInstanceEngine` has been replaced with the interface `IInstanceEngine` in the type of `DatabaseInstanceSourceProps.engine`
* **rds**: `DatabaseClusterProps.engineVersion` has been removed; instead, create an `IClusterEngine` with a specific version using the static factory methods in `DatabaseClusterEngine`
* **rds**: `DatabaseInstanceSourceProps.engineVersion` has been removed; instead, create an `IInstanceEngine` with a specific version using the static factory methods in `DatabaseInstanceEngine`
* **rds**: the property `majorEngineVersion` can no longer be passed when creating an `OptionGroup`; instead, create an `IInstanceEngine` with a specific version using the static factory methods in `DatabaseInstanceEngine`

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-rds Related to Amazon Relational Database effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. in-progress This issue is being actively worked on.
Projects
None yet
6 participants