Skip to content

Commit

Permalink
feat(efs): allow to specify az for one-zone (#30010)
Browse files Browse the repository at this point in the history
### Issue # (if applicable)

This PR allows users to specify AZ for the one-zone filesystems through the `vpcSubnets.availabilityZones`. Before this PR, `vpcSubnets` is not allowed when `oneZone` is enabled, this PR removes the restriction and takes the following use cases into considerations:

- [x] create a regional EFS file system with a mount target for each AZ
- [x] create a one-zone EFS file system with mount target at the auto-selected AZ
- [x] create a one-zone EFS file system at specified AZ with a single mount target at that AZ
- [x] create a regional EFS file system with a single mount target at the specified AZ

Closes #30005

### Reason for this change



### Description of changes



### Description of how you validated changes

1. Add additional unit tests.
2. I have deployed the code below and validated from my AWS console.

```ts
// create a regional EFS file system with a mount target for each AZ
new FileSystem(stack, 'FileSystem1', {
  vpc,
});

// create a one-zone EFS file system with mount target at the auto-selected AZ
new FileSystem(stack, 'FileSystem2', {
  vpc,
  oneZone: true,
});

// create a one-zone EFS file system at specified AZ with a single mount target at that AZ
new FileSystem(stack, 'FileSystem3', {
  vpc,
  oneZone: true,
  vpcSubnets: {
    availabilityZones: [vpc.availabilityZones[1]],
  },
});

// create a regional EFS file system with a single mount target at the specified AZ
new FileSystem(stack, 'FileSystem4', {
  vpc,
  vpcSubnets: {
    availabilityZones: [vpc.availabilityZones[2]],
  },
});

```


### Checklist
- [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md)

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
pahud authored May 1, 2024
1 parent c389a8b commit cbf130e
Showing 11 changed files with 607 additions and 263 deletions.

This file was deleted.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
@@ -489,7 +489,7 @@
"S3Bucket": {
"Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
},
"S3Key": "4554b47be6f57b68c6c7a7391dcc73894866d2377fe174883351e7639097f292.zip"
"S3Key": "bde7b5c89cb43285f884c94f0b9e17cdb0f5eb5345005114dd60342e0b8a85a1.zip"
},
"Timeout": 900,
"MemorySize": 128,
@@ -589,6 +589,89 @@
"Ref": "VpcPrivateSubnet1Subnet536B997A"
}
}
},
"FileSystem2C84D7800": {
"Type": "AWS::EFS::FileSystem",
"Properties": {
"AvailabilityZoneName": {
"Fn::Select": [
1,
{
"Fn::GetAZs": ""
}
]
},
"Encrypted": true,
"FileSystemPolicy": {
"Statement": [
{
"Action": [
"elasticfilesystem:ClientRootAccess",
"elasticfilesystem:ClientWrite"
],
"Condition": {
"Bool": {
"elasticfilesystem:AccessedViaMountTarget": "true"
}
},
"Effect": "Allow",
"Principal": {
"AWS": "*"
}
}
],
"Version": "2012-10-17"
},
"FileSystemTags": [
{
"Key": "Name",
"Value": "test-efs-one-zone-integ/FileSystem2"
}
]
},
"UpdateReplacePolicy": "Retain",
"DeletionPolicy": "Retain"
},
"FileSystem2EfsSecurityGroup2AF8F758": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "test-efs-one-zone-integ/FileSystem2/EfsSecurityGroup",
"SecurityGroupEgress": [
{
"CidrIp": "0.0.0.0/0",
"Description": "Allow all outbound traffic by default",
"IpProtocol": "-1"
}
],
"Tags": [
{
"Key": "Name",
"Value": "test-efs-one-zone-integ/FileSystem2"
}
],
"VpcId": {
"Ref": "Vpc8378EB38"
}
}
},
"FileSystem2EfsMountTargetPrivateSubnet26E5947D6": {
"Type": "AWS::EFS::MountTarget",
"Properties": {
"FileSystemId": {
"Ref": "FileSystem2C84D7800"
},
"SecurityGroups": [
{
"Fn::GetAtt": [
"FileSystem2EfsSecurityGroup2AF8F758",
"GroupId"
]
}
],
"SubnetId": {
"Ref": "VpcPrivateSubnet2Subnet3788AAA1"
}
}
}
},
"Parameters": {

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -13,6 +13,14 @@ new FileSystem(stack, 'FileSystem', {
oneZone: true,
});

new FileSystem(stack, 'FileSystem2', {
vpc,
oneZone: true,
vpcSubnets: {
availabilityZones: [vpc.availabilityZones[1]],
},
});

new integ.IntegTest(app, 'test-efs-one-zone-integ-test', {
testCases: [stack],
});
17 changes: 15 additions & 2 deletions packages/aws-cdk-lib/aws-efs/README.md
Original file line number Diff line number Diff line change
@@ -69,12 +69,25 @@ new efs.FileSystem(this, 'OneZoneFileSystem', {
⚠️ One Zone file systems are not compatible with the MAX_IO performance mode.

⚠️ When `oneZone` is enabled, the file system is automatically placed in the first availability zone of the VPC.
It is not currently possible to specify a different availability zone.
To specify a different availability zone:

```ts
declare const vpc: ec2.Vpc;

new efs.FileSystem(this, 'OneZoneFileSystem', {
vpc,
oneZone: true,
vpcSubnets: {
availabilityZones: ['us-east-1b'],
},
})
```

⚠️ When `oneZone` is enabled, mount targets will be created only in the specified availability zone.
This is to prevent deployment failures due to cross-AZ configurations.

⚠️ When `oneZone` is enabled, `vpcSubnets` cannot be specified.
⚠️ When `oneZone` is enabled, `vpcSubnets` can be specified with
`availabilityZones` that contains exactly one single zone.

### Replicating file systems

38 changes: 33 additions & 5 deletions packages/aws-cdk-lib/aws-efs/lib/efs-file-system.ts
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@ import { CfnFileSystem, CfnMountTarget } from './efs.generated';
import * as ec2 from '../../aws-ec2';
import * as iam from '../../aws-iam';
import * as kms from '../../aws-kms';
import { ArnFormat, FeatureFlags, Lazy, RemovalPolicy, Resource, Size, Stack, Tags } from '../../core';
import { ArnFormat, FeatureFlags, Lazy, RemovalPolicy, Resource, Size, Stack, Tags, Token } from '../../core';
import * as cxapi from '../../cx-api';

/**
@@ -721,19 +721,21 @@ export class FileSystem extends FileSystemBase {

private readonly _mountTargetsAvailable = new DependencyGroup();

private readonly props: FileSystemProps;

/**
* Constructor for creating a new EFS FileSystem.
*/
constructor(scope: Construct, id: string, props: FileSystemProps) {
super(scope, id);

this.props = props;

if (props.performanceMode === PerformanceMode.MAX_IO && props.oneZone) {
throw new Error('performanceMode MAX_IO is not supported for One Zone file systems.');
}

if (props.oneZone && props.vpcSubnets) {
throw new Error('vpcSubnets cannot be specified when oneZone is enabled.');
}
if (props.oneZone) { this.oneZoneValidation(); }

if (props.throughputMode === ThroughputMode.PROVISIONED && props.provisionedThroughputPerSecond === undefined) {
throw new Error('Property provisionedThroughputPerSecond is required when throughputMode is PROVISIONED');
@@ -767,7 +769,10 @@ export class FileSystem extends FileSystemBase {
lifecyclePolicies.push({ transitionToArchive: props.transitionToArchivePolicy });
}

const oneZoneAzName = props.vpc.availabilityZones[0];
// if props.vpcSubnets.availabilityZones is defined, select the first one as the zone otherwise
// the first AZ of the VPC.
const oneZoneAzName = props.vpcSubnets?.availabilityZones ?
props.vpcSubnets.availabilityZones[0] : props.vpc.availabilityZones[0];

const fileSystemProtection = props.replicationOverwriteProtection !== undefined ? {
replicationOverwriteProtection: props.replicationOverwriteProtection,
@@ -880,6 +885,29 @@ export class FileSystem extends FileSystemBase {
this.mountTargetsAvailable = this._mountTargetsAvailable;
}

private oneZoneValidation() {
// validate when props.oneZone is enabled
if (this.props.vpcSubnets && !this.props.vpcSubnets.availabilityZones) {
throw new Error('When oneZone is enabled and vpcSubnets defined, vpcSubnets.availabilityZones can not be undefined.');
}
// when vpcSubnets.availabilityZones is defined
if (this.props.vpcSubnets && this.props.vpcSubnets.availabilityZones) {
// it has to be only one az
if (this.props.vpcSubnets.availabilityZones?.length !== 1) {
throw new Error('When oneZone is enabled, vpcSubnets.availabilityZones should exactly have one zone.');
}
// it has to be in availabilityZones
// but we only check this when vpc.availabilityZones are valid(not dummy values nore unresolved tokens)
const isNotUnresolvedToken = (x: string) => !Token.isUnresolved(x);
const isNotDummy = (x: string) => !x.startsWith('dummy');
if (this.props.vpc.availabilityZones.every(isNotUnresolvedToken) &&
this.props.vpc.availabilityZones.every(isNotDummy) &&
!this.props.vpc.availabilityZones.includes(this.props.vpcSubnets.availabilityZones[0])) {
throw new Error('vpcSubnets.availabilityZones specified is not in vpc.availabilityZones.');
}
}
}

/**
* create access point from this filesystem
*/
65 changes: 63 additions & 2 deletions packages/aws-cdk-lib/aws-efs/test/efs-file-system.test.ts
Original file line number Diff line number Diff line change
@@ -932,15 +932,76 @@ test('one zone file system with MAX_IO performance mode is not supported', () =>
}).toThrow(/performanceMode MAX_IO is not supported for One Zone file systems./);
});

test('one zone file system with vpcSubnets is not supported', () => {
test('one zone file system with vpcSubnets but availabilityZones undefined is not supported', () => {
// THEN
expect(() => {
new FileSystem(stack, 'EfsFileSystem', {
vpc,
oneZone: true,
vpcSubnets: { subnetType: ec2.SubnetType.PUBLIC },
});
}).toThrow(/vpcSubnets cannot be specified when oneZone is enabled./);
}).toThrow(/When oneZone is enabled and vpcSubnets defined, vpcSubnets.availabilityZones can not be undefined./);
});

test('one zone file system with vpcSubnets but availabilityZones not in the vpc', () => {
// THEN
expect(() => {
// vpc with defined AZs
const vpc2 = new ec2.Vpc(stack, 'Vpc2', { availabilityZones: ['zonea', 'zoneb', 'zonec'] });
new FileSystem(stack, 'EfsFileSystem', {
vpc: vpc2,
oneZone: true,
vpcSubnets: { availabilityZones: ['not-exist-zone'] },
});
}).toThrow(/vpcSubnets.availabilityZones specified is not in vpc.availabilityZones./);
});

test('one zone file system with vpcSubnets but vpc.availabilityZones are dummy or unresolved tokens', () => {
// THEN
// this should not throw because vpc.availabilityZones are unresolved or dummy values
expect(() => {
new FileSystem(stack, 'EfsFileSystem', {
vpc,
oneZone: true,
vpcSubnets: { availabilityZones: ['not-exist-zone'] },
});
}).not.toThrow();
});

test('one zone file system with vpcSubnets.availabilityZones having 1 AZ.', () => {
// THEN
new FileSystem(stack, 'EfsFileSystem', {
vpc,
oneZone: true,
vpcSubnets: { availabilityZones: ['us-east-1a'] },
});
// THEN
Template.fromStack(stack).hasResourceProperties('AWS::EFS::FileSystem', {
AvailabilityZoneName: 'us-east-1a',
});

});

test('one zone file system with vpcSubnets.availabilityZones having more than 1 AZ.', () => {
// THEN
expect(() => {
new FileSystem(stack, 'EfsFileSystem', {
vpc,
oneZone: true,
vpcSubnets: { availabilityZones: ['mock-az1', 'mock-az2'] },
});
}).toThrow(/When oneZone is enabled, vpcSubnets.availabilityZones should exactly have one zone./);
});

test('one zone file system with vpcSubnets.availabilityZones empty.', () => {
// THEN
expect(() => {
new FileSystem(stack, 'EfsFileSystem', {
vpc,
oneZone: true,
vpcSubnets: { availabilityZones: [] },
});
}).toThrow(/When oneZone is enabled, vpcSubnets.availabilityZones should exactly have one zone./);
});

test.each([

0 comments on commit cbf130e

Please sign in to comment.