From a9279cbc2d19c7d24add8face18811fa4e0c93dd Mon Sep 17 00:00:00 2001 From: Geoffrey Cline Date: Fri, 21 Jan 2022 18:22:31 -0600 Subject: [PATCH 1/3] add desc of capacity fallback --- website/content/en/preview/faq.md | 6 ++++++ website/content/en/preview/provisioner.md | 6 ++++-- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/website/content/en/preview/faq.md b/website/content/en/preview/faq.md index a739e6dfdf1c..5689d05106af 100644 --- a/website/content/en/preview/faq.md +++ b/website/content/en/preview/faq.md @@ -95,6 +95,12 @@ If the instance type is unavailable for some reason, then fleet will move on to If you are using the spot capacity type, Karpenter uses the capacity-optimized-prioritized allocation strategy which tells fleet to find the instance type that EC2 has the most capacity of which will decrease the probability of a spot interruption happening in the near term. See [Choose the appropriate allocation strategy](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-fleet-allocation-strategy.html#ec2-fleet-allocation-use-cases) for information on fleet optimization. +### What if there is no spot capacity? Will Karpenter fallback to on-demand? + +Karpenter will fallback to on-demand, if your provisioner specifies both spot and on-demand. + +More specifically, Karpenter maintains a concept of "offerings" which is a combination of instance type, zone, and purchase type. Spot offerings are prioritized, if they're available. Whenever the fleet API returns an insufficient capacity for spot, that offering is removed from the list for 5 minutes. Since fleet gives expressive responses about which offerings are unavilable, it results in at worst 2 API calls. The retry will happen immediately within milliseconds. + ## Workloads ### How can someone deploying pods take advantage of Karpenter? diff --git a/website/content/en/preview/provisioner.md b/website/content/en/preview/provisioner.md index c8ecdc194c29..7d0432ca8ef2 100644 --- a/website/content/en/preview/provisioner.md +++ b/website/content/en/preview/provisioner.md @@ -139,11 +139,13 @@ Karpenter supports `amd64` nodes, and `arm64` nodes. ☁️ **AWS** - values - - `spot` (default) - - `on-demand` + - `spot` + - `on-demand` (default) Karpenter supports specifying capacity type, which is analogous to [EC2 purchase options](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-purchasing-options.html). +Karpenter prioritizes spot offerings. If the provider API (e.g., EC2 fleet API) indicates spot capacity is unavailable, Karpenter caches that result for five minutes. Next, Karpenter will provision on-demand instances, generally within milliseconds. + ## spec.kubeletConfiguration From bd1e656a433e9a0371ab1769a4232ef302dd8a65 Mon Sep 17 00:00:00 2001 From: Geoffrey Cline Date: Fri, 28 Jan 2022 14:58:44 -0600 Subject: [PATCH 2/3] Apply suggestions from code review Co-authored-by: Elton Pinto --- website/content/en/preview/faq.md | 2 +- website/content/en/preview/provisioner.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/website/content/en/preview/faq.md b/website/content/en/preview/faq.md index 5689d05106af..fb5555c8b885 100644 --- a/website/content/en/preview/faq.md +++ b/website/content/en/preview/faq.md @@ -99,7 +99,7 @@ See [Choose the appropriate allocation strategy](https://docs.aws.amazon.com/AWS Karpenter will fallback to on-demand, if your provisioner specifies both spot and on-demand. -More specifically, Karpenter maintains a concept of "offerings" which is a combination of instance type, zone, and purchase type. Spot offerings are prioritized, if they're available. Whenever the fleet API returns an insufficient capacity for spot, that offering is removed from the list for 5 minutes. Since fleet gives expressive responses about which offerings are unavilable, it results in at worst 2 API calls. The retry will happen immediately within milliseconds. +More specifically, Karpenter maintains a concept of "offerings" for each instance type, which is a combination of zone and capacity type (equivalent in the AWS cloud provider to an EC2 purchase option). Spot offerings are prioritized, if they're available. Whenever the Fleet API returns an insufficient capacity error for Spot instances, those particular offerings are temporarily removed from consideration (across the entire provisioner) for 45 seconds so that Karpenter can make forward progress through fallback. The retry will happen immediately within milliseconds. ## Workloads diff --git a/website/content/en/preview/provisioner.md b/website/content/en/preview/provisioner.md index 7d0432ca8ef2..5a99b7983e57 100644 --- a/website/content/en/preview/provisioner.md +++ b/website/content/en/preview/provisioner.md @@ -144,7 +144,7 @@ Karpenter supports `amd64` nodes, and `arm64` nodes. Karpenter supports specifying capacity type, which is analogous to [EC2 purchase options](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-purchasing-options.html). -Karpenter prioritizes spot offerings. If the provider API (e.g., EC2 fleet API) indicates spot capacity is unavailable, Karpenter caches that result for five minutes. Next, Karpenter will provision on-demand instances, generally within milliseconds. +Karpenter prioritizes Spot offerings if the provisioner allows Spot and on-demand instances. If the provider API (e.g. EC2 Fleet's API) indicates Spot capacity is unavailable, Karpenter caches that result across all attempts to provision EC2 capacity for that instance type and zone for the next 45 seconds. If there are no other possible offerings available for Spot, Karpenter will attempt to provision on-demand instances, generally within milliseconds. ## spec.kubeletConfiguration From b93ef357f787219624a2cdf7cb442e675dfc648c Mon Sep 17 00:00:00 2001 From: Geoffrey Cline Date: Fri, 28 Jan 2022 14:59:27 -0600 Subject: [PATCH 3/3] remove details of cache time --- website/content/en/preview/faq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/en/preview/faq.md b/website/content/en/preview/faq.md index fb5555c8b885..f97d344b34c4 100644 --- a/website/content/en/preview/faq.md +++ b/website/content/en/preview/faq.md @@ -99,7 +99,7 @@ See [Choose the appropriate allocation strategy](https://docs.aws.amazon.com/AWS Karpenter will fallback to on-demand, if your provisioner specifies both spot and on-demand. -More specifically, Karpenter maintains a concept of "offerings" for each instance type, which is a combination of zone and capacity type (equivalent in the AWS cloud provider to an EC2 purchase option). Spot offerings are prioritized, if they're available. Whenever the Fleet API returns an insufficient capacity error for Spot instances, those particular offerings are temporarily removed from consideration (across the entire provisioner) for 45 seconds so that Karpenter can make forward progress through fallback. The retry will happen immediately within milliseconds. +More specifically, Karpenter maintains a concept of "offerings" for each instance type, which is a combination of zone and capacity type (equivalent in the AWS cloud provider to an EC2 purchase option). Spot offerings are prioritized, if they're available. Whenever the Fleet API returns an insufficient capacity error for Spot instances, those particular offerings are temporarily removed from consideration (across the entire provisioner) so that Karpenter can make forward progress through fallback. The retry will happen immediately within milliseconds. ## Workloads