Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove EBS Attach for RFS in GovCloud #1216

Merged
merged 1 commit into from
Jan 6, 2025

Conversation

AndreKurait
Copy link
Member

@AndreKurait AndreKurait commented Jan 3, 2025

Description

We introduced support for larger shards by using ECS EBS Attach on the tasks, this is not supported in govcloud:

How Amazon Elastic Container Service differs for AWS GovCloud (US)
Attaching Amazon EBS volumes to Amazon ECS tasks is not supported.
[Source]

This pr introduces a short-term fix (before EKS) to go back to ephemeral storage for GovCloud. This will limit our max shard size back to 80gb.

  • When deployed in govcloud region, RFS stack does not attempt ebs attach
  • When deployed in govcloud, RFS Stack throws error if attempting to make a ethemeral volume over 200gb
  • Unit tests added in cdk (verify govcloud throws error for max shard size exceeding limit, verify no ebs attach present)

Issues Resolved

MIGRATIONS-2293

Testing

Added unit tests for govcloud expected behavior

Deployed to govcloud successfully
image

Verified the error message is appropriate
image

Check List

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link

codecov bot commented Jan 3, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.49%. Comparing base (f6b0c4c) to head (f731072).
Report is 9 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##               main    #1216   +/-   ##
=========================================
  Coverage     80.49%   80.49%           
  Complexity     3079     3079           
=========================================
  Files           421      421           
  Lines         15650    15650           
  Branches       1057     1057           
=========================================
  Hits          12598    12598           
  Misses         2408     2408           
  Partials        644      644           
Flag Coverage Δ
unittests 80.49% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@AndreKurait AndreKurait marked this pull request as ready for review January 3, 2025 21:59
Copy link
Member

@chelma chelma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One question I'd like to understand, but otherwise LGTM

@@ -61,7 +61,7 @@ export class MigrationServiceCore extends Stack {
props.taskRolePolicies?.forEach(policy => this.serviceTaskRole.addToPolicy(policy))

const serviceTaskDef = new FargateTaskDefinition(this, "ServiceTaskDef", {
ephemeralStorageGiB: props.ephemeralStorageGiB ? props.ephemeralStorageGiB : 75,
ephemeralStorageGiB: Math.max(props.ephemeralStorageGiB ? props.ephemeralStorageGiB : 75, 21), // valid values 21 - 200
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this change in behavior affect the user experience? Can you briefly list all the scenarios and the before/after outcome? I ask because I'm not entirely sure it does change things.

Copy link
Member Author

@AndreKurait AndreKurait Jan 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also ensures for RFS GovCloud we don't attempt to set this less than 21GB which will otherwise result in an deployment error

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it - so we're preventing a deploy time error by overriding a too-small value with the actual minimum. Thanks!

1
)

if (volumeSizeGB > 16000) {
if (shardVolumeSizeGiB > (16*1024)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for cleaning up this logic a bit

// ECS EBS attachment is not supported in GovCloud
// https://docs.aws.amazon.com/govcloud-us/latest/UserGuide/govcloud-ecs.html#govcloud-ecs-diffs
// Use Ephemeral Storage instead, adding size for shard
ephemeralStorageGiB = Math.ceil(shardVolumeSizeGiB + systemStorageGiB)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Lots of math, that logic is easier to test if it was refactored into a stand-alone function(s)

Copy link
Member

@chelma chelma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - Thanks Andre!

@AndreKurait AndreKurait merged commit 9781a9b into opensearch-project:main Jan 6, 2025
22 checks passed
@AndreKurait AndreKurait deleted the FixRFSEBSGovCloud branch January 6, 2025 21:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants