Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update interruption-handling.md #7552

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Update interruption-handling.md #7552

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 29 additions & 27 deletions designs/interruption-handling.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,15 +29,17 @@ There are two ways in-which Spot interruption notifications and Rebalance Recomm
EC2 IMDS is an HTTP API that can only be locally accessed from an EC2 instance.

```
`curl 169.254.169.254/latest/meta-data/spot/instance-action
# Termination Check
curl 169.254.169.254/latest/meta-data/spot/instance-action
{
"action": "terminate",
"time": "2022-07-11T17:11:44Z"
}

curl 169.254.169.254``/``latest``/``meta``-``data``/``events``/``recommendations``/``rebalance`
`{`
` ``"noticeTime"``:`` ``"2022-07-16T19:18:24Z"`
# Rebalance Check
curl 169.254.169.254/latest/meta-data/events/recommendations/rebalance
{
"noticeTime": "2022-07-16T19:18:24Z"
}

```
Expand All @@ -47,19 +49,19 @@ curl 169.254.169.254``/``latest``/``meta``-``data``/``events``/``recommendations
EventBridge is an Event Bus service within AWS that allows users to set rules on events to capture and then target destinations for those events. Relevant targets for Spot interruption notifications include SQS, Lambda, and EC2-Terminate-Instance.

```
`# Example spot interruption notification EventBridge rule`
`$ aws events put``-``rule \`
` ``--``name ``MyK8sSpotTermRule`` \`
` ``--``event``-``pattern ``"{\"source\": [\"aws.ec2\"],\"detail-type\": [\"EC2 Spot Instance Interruption\"]}"`

`# Example rebalance recommendation EventBridge rule``
$ aws events put-rule \
--name MyK8sRebalanceRule \
--event-pattern "{\"source\": [\"aws.ec2\"],\"detail-type\": [\"EC2 Instance Rebalance Recommendation\"]}"
`` `
`# Example targeting an SQS queue`
`$ aws events put``-``targets ``--``rule ``MyK8sSpotTermRule`` \`
` ``--``targets ``"Id"``=``"1"``,``"Arn"``=``"arn:aws:sqs:us-east-1:123456789012:MyK8sTermQueue"`` `
# Example spot interruption notification EventBridge rule
aws events put-rule \
--name MyK8sSpotTermRule \
--event-pattern "{\"source\": [\"aws.ec2\"],\"detail-type\": [\"EC2 Spot Instance Interruption\"]}"

# Example rebalance recommendation EventBridge rule
aws events put-rule \
--name MyK8sRebalanceRule \
--event-pattern "{\"source\": [\"aws.ec2\"],\"detail-type\": [\"EC2 Instance Rebalance Recommendation\"]}"

# Example targeting an SQS queue
aws events put-targets --rule MyK8sSpotTermRule \
--targets "Id=1,Arn=arn:aws:sqs:us-east-1:123456789012:MyK8sTermQueue"
```


Expand Down Expand Up @@ -113,17 +115,17 @@ SQS exposes a VPC Endpoint which will fulfill the isolated VPC use-case.
Dynamically creating the SQS infrastructure and EventBridge rules means that Karpenter’s IAM role would need permissions to SQS and EventBridge:

```
`"sqs:GetQueueUrl",`
`"sqs:ListQueues"``,`
`"sqs:ReceiveMessage"``,`
`"sqs:CreateQueue"``,`
`"sqs:DeleteMessage"``,`
`"events:ListRules",`
"`events:DescribeRule`",
"events:PutRule",
"sqs:GetQueueUrl",
"sqs:ListQueues",
"sqs:ReceiveMessage",
"sqs:CreateQueue",
"sqs:DeleteMessage",
"events:ListRules",
"events:DescribeRule",
"events:PutRule",
"events:PutTargets",
"`events:DeleteRule`",
`"events:RemoveTargets"`
"events:DeleteRule",
"events:RemoveTargets"
```

The policy can be setup with a predefined name based on the cluster name. For example, `karpenter-events-${CLUSTER_NAME}` which would allow for a more constrained resource policy.
Expand Down