Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add missing instances to instance store volumes table #1966

Merged

Conversation

ConnorJC3
Copy link
Contributor

@ConnorJC3 ConnorJC3 commented Mar 14, 2024

Is this a bug fix or adding new feature?

Bug fix

What is this PR about? / Why do we need it?

Adds missing instance store instances to the table
Also documents command to generate this table
Also adds a script to generate this table

What testing is done?

Confirmed manually against table on AWS docs, as well as previous list to ensure no types were removed/changed

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Mar 14, 2024
Copy link

Code Coverage Diff

This PR does not change the code coverage

@torredil
Copy link
Member

/retest

Copy link
Contributor

@AndrewSirenko AndrewSirenko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spot checked 20 different instance types. Looks good. A few comments but then will approve.

// / IMDS does not provide NVMe instance store data; we'll just list all instances here
// / TODO: See if we can get these values from DescribeInstanceTypes API
// Generated by command, requires aws cli and jq:
// aws ec2 describe-instance-types --region us-east-1 --filters "Name=instance-storage-supported,Values=true" --query "InstanceTypes[].[InstanceType, InstanceStorageInfo]" | jq -r 'map("\"" + .[0] + "\": " + (.[1].Disks | map(.Count) | add | tostring) + ",") | .[]' | sort
var nvmeInstanceStoreVolumes = map[string]int{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason we don't support hpc6id.32xlarge instance?

4 instance store volumes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, it's only available in us-east-2 so when I go looking in us-east-1 I don't find it. Be ready for the insane kludge of v2

Copy link
Contributor

@AndrewSirenko AndrewSirenko Mar 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow that's unintuitive. Thanks for letting us know the reason and adding in this new script. Really removes some operator toil!

pkg/cloud/volume_limits.go Outdated Show resolved Hide resolved
pkg/cloud/volume_limits.go Outdated Show resolved Hide resolved
@AndrewSirenko
Copy link
Contributor

/retest

2 similar comments
@torredil
Copy link
Member

/retest

@AndrewSirenko
Copy link
Contributor

/retest

@ConnorJC3 ConnorJC3 force-pushed the update-instance-store branch from d2bde02 to aa15f76 Compare March 14, 2024 21:49
Copy link
Member

@torredil torredil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

$ ./check.sh
Success! no instances were removed
original="https://raw.githubusercontent.com/kubernetes-sigs/aws-ebs-csi-driver/master/pkg/cloud/volume_limits.go"
updated="https://raw.githubusercontent.com/kubernetes-sigs/aws-ebs-csi-driver/aa15f7650d453e80b262587bd0bcc62242601177/pkg/cloud/volume_limits.go"

extract_table() {
  curl -s "$1" | sed -n '/^var nvmeInstanceStoreVolumes = map\[string\]int{/,/^}/p' | grep -v "^var" | grep -v "^}"
}

original_table=$(extract_table "$original")
updated_table=$(extract_table "$updated")

original_keys=$(echo "$original_table" | grep -o '^[^:]*' | sort)
updated_keys=$(echo "$updated_table" | grep -o '^[^:]*' | sort)

removed_instances=$(diff <(echo "$original_keys") <(echo "$updated_keys") | grep "^<" | cut -d ' ' -f 2)

if [ -n "$removed_instances" ]; then
  echo "Warning!!! the following instances were incorrectly removed from the table"
  echo "$removed_instances"
else
  echo "Success! no instances were removed"
fi

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 14, 2024
@AndrewSirenko
Copy link
Contributor

/retest

Copy link
Contributor

@AndrewSirenko AndrewSirenko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! Thanks for the initiative with the script.

@AndrewSirenko
Copy link
Contributor

/retest aws-ebs-csi-driver-verify

@k8s-ci-robot
Copy link
Contributor

@AndrewSirenko: The /retest command does not accept any targets.
The following commands are available to trigger required jobs:

  • /test pull-aws-ebs-csi-driver-e2e-multi-az
  • /test pull-aws-ebs-csi-driver-e2e-single-az
  • /test pull-aws-ebs-csi-driver-external-test
  • /test pull-aws-ebs-csi-driver-external-test-eks
  • /test pull-aws-ebs-csi-driver-test-e2e-external-eks-windows
  • /test pull-aws-ebs-csi-driver-test-helm-chart
  • /test pull-aws-ebs-csi-driver-unit
  • /test pull-aws-ebs-csi-driver-verify

The following commands are available to trigger optional jobs:

  • /test pull-aws-ebs-csi-driver-external-test-arm64

Use /test all to run all jobs.

In response to this:

/retest aws-ebs-csi-driver-verify

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@AndrewSirenko
Copy link
Contributor

/retest pull-aws-ebs-csi-driver-verify

@k8s-ci-robot
Copy link
Contributor

@AndrewSirenko: The /retest command does not accept any targets.
The following commands are available to trigger required jobs:

  • /test pull-aws-ebs-csi-driver-e2e-multi-az
  • /test pull-aws-ebs-csi-driver-e2e-single-az
  • /test pull-aws-ebs-csi-driver-external-test
  • /test pull-aws-ebs-csi-driver-external-test-eks
  • /test pull-aws-ebs-csi-driver-test-e2e-external-eks-windows
  • /test pull-aws-ebs-csi-driver-test-helm-chart
  • /test pull-aws-ebs-csi-driver-unit
  • /test pull-aws-ebs-csi-driver-verify

The following commands are available to trigger optional jobs:

  • /test pull-aws-ebs-csi-driver-external-test-arm64

Use /test all to run all jobs.

In response to this:

/retest pull-aws-ebs-csi-driver-verify

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@AndrewSirenko
Copy link
Contributor

/test pull-aws-ebs-csi-driver-verify

@AndrewSirenko
Copy link
Contributor

/approve

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 15, 2024
@ConnorJC3 ConnorJC3 force-pushed the update-instance-store branch from aa15f76 to 380b07f Compare March 15, 2024 15:04
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 15, 2024
@ConnorJC3 ConnorJC3 force-pushed the update-instance-store branch from 380b07f to 60807fe Compare March 15, 2024 15:05
Copy link
Member

@torredil torredil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 15, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: AndrewSirenko, torredil

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [AndrewSirenko,torredil]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 416b5b8 into kubernetes-sigs:master Mar 15, 2024
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants