diff --git a/Dockerfile b/Dockerfile index adb02531..7e60545b 100644 --- a/Dockerfile +++ b/Dockerfile @@ -6,8 +6,8 @@ RUN wget https://github.com/gohugoio/hugo/releases/download/v0.92.0/hugo_0.92.0_ tar -xf hugo_0.92.0_Linux-64bit.tar.gz hugo && \ rm -rf hugo_0.92.0_Linux-64bit.tar.gz && \ cp hugo /usr/bin/hugo -WORKDIR /www -RUN git clone https://github.com/matcornic/hugo-theme-learn/ themes/learn +WORKDIR /www/ COPY . /www/ +RUN rm -rf themes/learn && git clone https://github.com/matcornic/hugo-theme-learn/ themes/learn ENTRYPOINT ["hugo", "server", "--bind", "0.0.0.0"] CMD [""] diff --git a/content/karpenter/040_karpenter/advanced_provisioner.md b/content/karpenter/040_karpenter/advanced_provisioner.md index 49705422..b25514aa 100644 --- a/content/karpenter/040_karpenter/advanced_provisioner.md +++ b/content/karpenter/040_karpenter/advanced_provisioner.md @@ -116,17 +116,17 @@ kubectl describe provisioners default ## (Optional Read) Customizing AMIs and Node Bootstrapping {{% notice info %}} -In this workshop we will stick to the default AMI's used by Karpenter. This section does not contain any exercise or command. The section describes how the AMI and node bootsrapping can be adapted when needed. If you want to deep dive into this topic you can [read the following karpenter documentation link](https://karpenter.sh/v0.6.5/aws/launch-templates/) +In this workshop we will stick to the default AMI's used by Karpenter. This section does not contain any exercise or command. The section describes how the AMI and node bootsrapping can be adapted when needed. If you want to deep dive into this topic you can [read the following karpenter documentation link](https://karpenter.sh/v0.10.0/aws/launch-templates/) {{% /notice %}} -By default, Karpenter generates [launch templates](https://docs.aws.amazon.com/autoscaling/ec2/userguide/LaunchTemplates.html) that use [EKS Optimized AMI](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html) for nodes. Often, users need to customize the node image to integrate with existing infrastructure, meet compliance requirements, add extra storage, etc. Karpenter supports custom node images and bootsrapping through Launch Templates. If you need to customize the node, then you need a custom launch template. +By default, Karpenter generates [launch templates](https://docs.aws.amazon.com/autoscaling/ec2/userguide/LaunchTemplates.html) that use [EKS Optimized AMI](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html) and Encrypted EBS root volumes with the default (AWS Managed) KMS key for nodes. Often, users need to customize the node image to integrate with existing infrastructure, meet compliance requirements, add extra storage, etc. Karpenter supports custom node images and bootsrapping through Launch Templates. If you need to customize the node, then you need a custom launch template. {{%notice note %}} Using custom launch templates prevents multi-architecture support, the ability to automatically upgrade nodes, and securityGroup discovery. Using launch templates may also cause confusion because certain fields are duplicated within Karpenter’s provisioners while others are ignored by Karpenter, e.g. subnets and instance types. {{% /notice%}} {{% notice warning %}} -By customizing the image, you are taking responsibility for maintaining it, including security updates. In the default configuration, Karpenter will use the latest version of the EKS optimized AMI, which is maintained by AWS. +By customizing the image, you are taking responsibility for maintaining it, including security updates. In the default configuration, Karpenter will use the latest version of the EKS optimized AMI, which is maintained by AWS. {{% /notice %}} @@ -150,6 +150,3 @@ when building a custom base image. Specifying max-pods can break Karpenter's bin-packing logic (it has no way to know what this setting is). If Karpenter attempts to pack more than this number of pods, the instance may be oversized, and additional pods will reschedule. {{% /notice %}} - - - diff --git a/content/karpenter/040_karpenter/automatic_node_provisioning.md b/content/karpenter/040_karpenter/automatic_node_provisioning.md index 3a6fb325..7ca60c5b 100644 --- a/content/karpenter/040_karpenter/automatic_node_provisioning.md +++ b/content/karpenter/040_karpenter/automatic_node_provisioning.md @@ -96,19 +96,24 @@ echo type: $(kubectl describe node --selector=intent=apps | grep "beta.kubernete There is something even more interesting to learn about how the node was provisioned. Check out Karpenter logs and look at the new Karpenter created. The lines should be similar to the ones below ```bash -2021-11-15T11:09:10.574Z INFO controller.allocation.provisioner/default Waiting to batch additional pods {"commit": "6468992"} -2021-11-15T11:09:11.976Z INFO controller.allocation.provisioner/default Found 1 provisionable pods {"commit": "6468992"} -2021-11-15T11:09:13.037Z INFO controller.allocation.provisioner/default Computed packing for 1 pod(s) with instance type option(s) [t3.medium c6i.large c5.large t3a.medium c5ad.large c4.large c5a.large c3.large c5d.large c5n.large t3a.large m5a.large t3.large m5ad.large m5.large m6i.large m3.large m4.large m5zn.large m5dn.large] {"commit": "6468992"} -2021-11-15T11:09:15.185Z INFO controller.allocation.provisioner/default Launched instance: i-09ba099d68f7c982c, hostname: xxxxxxxxxxxxx.compute.internal, type: t3.medium, zone: eu-west-1a, capacityType: spot {"commit": "6468992"} -2021-11-15T11:09:15.202Z INFO controller.allocation.provisioner/default Bound 1 pod(s) to node xxxxxxxxxxxxx.compute.internal {"commit": "6468992"} -2021-11-15T11:09:15.202Z INFO controller.allocation.provisioner/default Starting provisioning loop {"commit": "6468992"} +2022-05-12T03:38:15.698Z INFO controller Batched 1 pod(s) in 1.000075807s {"commit": "00661aa"} +2022-05-12T03:38:16.485Z DEBUG controller Discovered 401 EC2 instance types {"commit": "00661aa"} +2022-05-12T03:38:16.601Z DEBUG controller Discovered EC2 instance types zonal offerings {"commit": "00661aa"} +2022-05-12T03:38:16.768Z DEBUG controller Discovered subnets: [subnet-0204b1b3b885ca98d (eu-west-1a) subnet-037d1d97a6a473fd1 (eu-west-1b) subnet-04c2ca248972479e7 (eu-west-1b) subnet-063d5c7ba912986d5 (eu-west-1a)] {"commit": "00661aa"} +2022-05-12T03:38:16.953Z DEBUG controller Discovered security groups: [sg-03ab1d5d49b00b596 sg-06e7e2ca961ab3bed] {"commit": "00661aa", "provisioner": "default"} +2022-05-12T03:38:16.955Z DEBUG controller Discovered kubernetes version 1.21 {"commit": "00661aa", "provisioner": "default"} +2022-05-12T03:38:17.046Z DEBUG controller Discovered ami-0440c10a3f77514d8 for query "/aws/service/eks/optimized-ami/1.21/amazon-linux-2/recommended/image_id" {"commit": "00661aa", "provisioner": "default"} +2022-05-12T03:38:17.213Z DEBUG controller Created launch template, Karpenter-eksworkshop-eksctl-7600085100718942941 {"commit": "00661aa", "provisioner": "default"} +2022-05-12T03:38:19.400Z INFO controller Launched instance: i-0f47f9dc3fa486c35, hostname: ip-192-168-37-165.eu-west-1.compute.internal, type: t3.medium, zone: eu-west-1b, capacityType: spot {"commit": "00661aa", "provisioner": "default"} +2022-05-12T03:38:19.412Z INFO controller Created node with 1 pods requesting {"cpu":"1125m","memory":"1536Mi","pods":"3"} from types c4.large, c6a.large, t3a.medium, c5a.large, c6i.large and 306 other(s) {"commit": "00661aa", "provisioner": "default"} +2022-05-12T03:38:19.426Z INFO controller Waiting for unschedulable pods {"commit": "00661aa"} ``` We explained earlier on about group-less cluster scalers and how that simplifies operations and maintenance. Let's deep dive for a second into this concept. Notice how Karpenter picks up the instance from a diversified selection of instances. In this case it selected the following instances: ``` -t3.medium c6i.large c5.large t3a.medium c5ad.large c4.large c5a.large c3.large c5d.large c5n.large t3a.large m5a.large t3.large m5ad.large m5.large m6i.large m3.large m4.large m5zn.large m5dn.large +c4.large, c6a.large, t3a.medium, c5a.large, c6i.large and 306 other(s) ``` {{% notice note %}} @@ -118,7 +123,7 @@ Instances types might be different depending on the region selected. All this instances are the suitable instances that reduce the waste of resources (memory and CPU) for the pod submitted. If you are interested in Algorithms, internally Karpenter is using a [First Fit Decreasing (FFD)](https://en.wikipedia.org/wiki/Bin_packing_problem#First_Fit_Decreasing_(FFD)) approach. Note however this can change in the future. -We did set Karpenter Provisioner to use [EC2 Spot instances](https://aws.amazon.com/ec2/spot/), and there was no `instance-types` [requirement section in the Provisioner to filter the type of instances](https://karpenter.sh/v0.6.2/provisioner/#instance-types). This means that Karpenter will use the default value of instances types to use. The default value includes all instance types with the exclusion of metal (non-virtualized), [non-HVM](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/virtualization_types.html), and GPU instances.Internally Karpenter used **EC2 Fleet in Instant mode** to provision the instances. You can read more about EC2 Fleet Instant mode [**here**](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instant-fleet.html). Here are a few properties to mention about EC2 Fleet instant mode that are key for Karpenter. +We did set Karpenter Provisioner to use [EC2 Spot instances](https://aws.amazon.com/ec2/spot/), and there was no `instance-types` [requirement section in the Provisioner to filter the type of instances](https://karpenter.sh/v0.10.0/provisioner/#instance-types). This means that Karpenter will use the default value of instances types to use. The default value includes all instance types with the exclusion of metal (non-virtualized), [non-HVM](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/virtualization_types.html), and GPU instances.Internally Karpenter used **EC2 Fleet in Instant mode** to provision the instances. You can read more about EC2 Fleet Instant mode [**here**](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instant-fleet.html). Here are a few properties to mention about EC2 Fleet instant mode that are key for Karpenter. * EC2 Fleet instant mode provides a synchronous call to procure instances, including EC2 Spot, this simplifies and avoid error when provisioning instances. For those of you familiar with [Cluster Autoscaler on AWS](https://github.com/kubernetes/autoscaler/blob/c4b56ea56136681e8a8ff654dfcd813c0d459442/cluster-autoscaler/cloudprovider/aws/auto_scaling_groups.go#L33-L36), you may know about how it uses `i-placeholder` to coordinate instances that have been created in asynchronous ways. @@ -224,27 +229,24 @@ This will set a few pods pending. Karpenter will get the pending pod signal and ```bash -2021-11-15T12:33:14.976Z INFO controller.allocation.provisioner/default Found 5 provisionable pods {"commit": "6468992"} -2021-11-15T12:33:16.324Z INFO controller.allocation.provisioner/default Computed packing for 5 pod(s) with instance type option(s) [c3.2xlarge c4.2xlarge c5ad.2xlarge c6i.2xlarge c5a.2xlarge c5d.2xlarge c5.2xlarge c5n.2xlarge m3.2xlarge t3a.2xlarge m5ad.2xlarge m4.2xlarge t3.2xlarge m5n.2xlarge m5d.2xlarge m6i.2xlarge m5a.2xlarge m5zn.2xlarge m5.2xlarge m5dn.2xlarge] {"commit": "6468992"} -2021-11-15T12:33:18.774Z INFO controller.allocation.provisioner/default Launched instance: i-0c1fc34e7527358f0, hostname: xxxxxxxxxxxxx.compute.internal, type: t3.2xlarge, zone: eu-west-1a, capacityType: spot {"commit": "6468992"} -2021-11-15T12:33:18.802Z INFO controller.allocation.provisioner/default Bound 5 pod(s) to node xxxxxxxxxxxxx.compute.internal {"commit": "6468992"} -2021-11-15T12:33:18.802Z INFO controller.allocation.provisioner/default Starting provisioning loop {"commit": "6468992"} +2022-05-12T03:47:17.907Z INFO controller Batched 5 pod(s) in 1.056343494s {"commit": "00661aa"} +2022-05-12T03:47:18.692Z DEBUG controller Discovered 401 EC2 instance types {"commit": "00661aa"} +2022-05-12T03:47:18.848Z DEBUG controller Discovered EC2 instance types zonal offerings {"commit": "00661aa"} +2022-05-12T03:47:19.011Z DEBUG controller Discovered subnets: [subnet-0204b1b3b885ca98d (eu-west-1a) subnet-037d1d97a6a473fd1 (eu-west-1b) subnet-04c2ca248972479e7 (eu-west-1b) subnet-063d5c7ba912986d5 (eu-west-1a)] {"commit": "00661aa"}2022-05-12T03:47:19.094Z DEBUG controller Discovered security groups: [sg-03ab1d5d49b00b596 sg-06e7e2ca961ab3bed] {"commit": "00661aa", "provisioner": "default"} +2022-05-12T03:47:19.097Z DEBUG controller Discovered kubernetes version 1.21 {"commit": "00661aa", "provisioner": "default"} +2022-05-12T03:47:19.134Z DEBUG controller Discovered ami-0440c10a3f77514d8 for query "/aws/service/eks/optimized-ami/1.21/amazon-linux-2/recommended/image_id" {"commit": "00661aa", "provisioner": "default"} +2022-05-12T03:47:19.175Z DEBUG controller Discovered launch template Karpenter-eksworkshop-eksctl-7600085100718942941 {"commit": "00661aa", "provisioner": "default"} +2022-05-12T03:47:21.199Z INFO controller Launched instance: i-066971cf53a56a2f7, hostname: ip-192-168-38-49.eu-west-1.compute.internal, type: t3.2xlarge, zone: eu-west-1b, capacityType: spot {"commit": "00661aa", "provisioner": "default"} +2022-05-12T03:47:21.208Z INFO controller Created node with 5 pods requesting {"cpu":"5125m","memory":"7680Mi","pods":"7"} from types c4.2xlarge, c6i.2xlarge, c6a.2xlarge, c5a.2xlarge, c5.2xlarge and 222 other(s) {"commit": "00661aa", "provisioner": "default"} +2022-05-12T03:47:21.236Z INFO controller Waiting for unschedulable pods {"commit": "00661aa"} ``` Indeed the instances selected this time are larger ! The instances selected in this example were: ```bash -c3.2xlarge c4.2xlarge c5ad.2xlarge c6i.2xlarge c5a.2xlarge c5d.2xlarge c5.2xlarge c5n.2xlarge m3.2xlarge t3a.2xlarge m5ad.2xlarge m4.2xlarge t3.2xlarge m5n.2xlarge m5d.2xlarge m6i.2xlarge m5a.2xlarge m5zn.2xlarge m5.2xlarge m5dn.2xlarge. +c4.2xlarge, c6i.2xlarge, c6a.2xlarge, c5a.2xlarge, c5.2xlarge and 222 other(s) ``` -There is one last thing that we have not mentioned until now. Check out this line in Karpenter log. - -```bash -2021-11-15T12:33:18.802Z INFO controller.allocation.provisioner/default Bound 5 pod(s) to node ip-192-168-89-216.eu-west-1.compute.internal {"commit": "6468992"} -``` - -The line and message **Bound 5 pod(s)** is important. Karpenter Provisioners attempt to schedule pods when they are in state `type=PodScheduled,reason=Unschedulable`. In this case, Karpenter will make a provisioning decision, launch new capacity, and proactively **bind pods to the provisioned nodes**. Unlike the Cluster Autoscaler, Karpenter does not wait for the [Kube Scheduler](https://kubernetes.io/docs/concepts/scheduling-eviction/kube-scheduler) to make a scheduling decision, as the decision is already made during the provisioning time. The objective of this operation is to speed up the placement of the pods to the new nodes. - Finally to check out the configuration of the `intent=apps` node execute again: ``` @@ -280,9 +282,3 @@ In this section we have learned: * Karpenter can scale up from zero and scale in to zero. -* Karpenter binds Pods directly with newly created nodes thus reducing the total time for the pods to be placed and available. - - - - - diff --git a/content/karpenter/040_karpenter/ec2_spot_deployments.md b/content/karpenter/040_karpenter/ec2_spot_deployments.md index 03c13c62..e34575a4 100644 --- a/content/karpenter/040_karpenter/ec2_spot_deployments.md +++ b/content/karpenter/040_karpenter/ec2_spot_deployments.md @@ -24,7 +24,7 @@ To deploy the Node Termination Handler run the following command: helm repo add eks https://aws.github.io/eks-charts helm install aws-node-termination-handler \ --namespace kube-system \ - --version 0.16.0 \ + --version 0.18.3 \ --set nodeSelector."karpenter\\.sh/capacity-type"=spot \ eks/aws-node-termination-handler ``` diff --git a/content/karpenter/040_karpenter/multiple_architectures.md b/content/karpenter/040_karpenter/multiple_architectures.md index 4d154ed0..cc04d245 100644 --- a/content/karpenter/040_karpenter/multiple_architectures.md +++ b/content/karpenter/040_karpenter/multiple_architectures.md @@ -124,12 +124,15 @@ The output should show something similar to the lines below ```bash ... -2021-11-16T00:43:22.773Z INFO controller.allocation.provisioner/default Starting provisioning loop {"commit": "6468992"} -2021-11-16T00:43:22.773Z INFO controller.allocation.provisioner/default Waiting to batch additional pods {"commit": "6468992"} -2021-11-16T00:43:23.976Z INFO controller.allocation.provisioner/default Found 2 provisionable pods {"commit": "6468992"} -2021-11-16T00:43:25.034Z INFO controller.allocation.provisioner/default Computed packing for 2 pod(s) with instance type option(s) [c5a.xlarge c5d.xlarge c3.xlarge c4.xlarge c5ad.xlarge c5.xlarge c6i.xlarge c1.xlarge c5n.xlarge m1.xlarge m3.xlarge t3.xlarge t3a.xlarge m5dn.xlarge m5ad.xlarge m4.xlarge m5zn.xlarge m5n.xlarge m6i.xlarge m5d.xlarge] {"commit": "6468992"} -2021-11-16T00:43:27.038Z INFO controller.allocation.provisioner/default Launched instance: i-0e4b58e5fbdc8eeb8, hostname: xxxxxxxxxxxxxxxxxxx.compute.internal, type: t3a.xlarge, zone: eu-west-1a, capacityType: on-demand {"commit": "6468992"} -2021-11-16T00:43:27.057Z INFO controller.allocation.provisioner/default Bound 2 pod(s) to node xxxxxxxxxxxxxxxxxxx.compute.internal {"commit": "6468992"} +2022-05-12T04:15:10.593Z INFO controller Batched 2 pod(s) in 1.009818195s {"commit": "00661aa"} +2022-05-12T04:15:10.770Z DEBUG controller Discovered subnets: [subnet-0204b1b3b885ca98d (eu-west-1a) subnet-037d1d97a6a473fd1 (eu-west-1b) subnet-04c2ca248972479e7 (eu-west-1b) subnet-063d5c7ba912986d5 (eu-west-1a)] {"commit": "00661aa"} +2022-05-12T04:15:10.851Z DEBUG controller Discovered security groups: [sg-03ab1d5d49b00b596 sg-06e7e2ca961ab3bed]{"commit": "00661aa", "provisioner": "default"} +2022-05-12T04:15:10.854Z DEBUG controller Discovered kubernetes version 1.21 {"commit": "00661aa", "provisioner": "default"} +2022-05-12T04:15:10.940Z DEBUG controller Discovered ami-0440c10a3f77514d8 for query "/aws/service/eks/optimized-ami/1.21/amazon-linux-2/recommended/image_id" {"commit": "00661aa", "provisioner": "default"} +2022-05-12T04:15:10.977Z DEBUG controller Discovered launch template Karpenter-eksworkshop-eksctl-13001267661656074018 {"commit": "00661aa", "provisioner": "default"} +2022-05-12T04:15:13.081Z INFO controller Launched instance: i-0d19d3eeb2d59578d, hostname: ip-192-168-22-165.eu-west-1.compute.internal, type: m4.xlarge, zone: eu-west-1a, capacityType: spot {"commit": "00661aa", "provisioner": "default"} +2022-05-12T04:15:13.093Z INFO controller Created node with 2 pods requesting {"cpu":"2125m","memory":"512M","pods":"5"} from types c4.xlarge, c6i.xlarge, c5a.xlarge, c5.xlarge, c6a.xlarge and 267 other(s) {"commit": "00661aa", "provisioner": "default"} +2022-05-12T04:15:13.103Z INFO controller Waiting for unschedulable pods {"commit": "00661aa"} ... ``` @@ -199,10 +202,17 @@ The output should show something similar to the lines below ```bash ... -2021-11-30T19:21:52.786Z INFO controller.provisioning Batched 2 pods in 1.030167637s {"commit": "84b683b", "provisioner": "default"} -2021-11-30T19:21:52.791Z INFO controller.provisioning Computed packing of 1 node(s) for 2 pod(s) with instance type option(s) [c6g.xlarge] {"commit": "84b683b", "provisioner": "default"} -2021-11-30T19:21:55.764Z INFO controller.provisioning Launched instance: i-0fe8ef259d1e0bf83, hostname: ip-192-168-77-232.us-west-2.compute.internal, type: c6g.xlarge, zone: us-west-2a, capacityType: on-demand {"commit": "84b683b", "provisioner": "default"} -2021-11-30T19:21:55.865Z INFO controller.provisioning Bound 2 pod(s) to node ip-192-168-77-232.us-west-2.compute.internal {"commit": "84b683b", "provisioner": "default"} +2022-05-12T04:20:51.896Z INFO controller Batched 2 pod(s) in 1.014499438s {"commit": "00661aa"} +2022-05-12T04:20:52.665Z DEBUG controller Discovered 401 EC2 instance types {"commit": "00661aa"} +2022-05-12T04:20:52.790Z DEBUG controller Discovered EC2 instance types zonal offerings {"commit": "00661aa"} +2022-05-12T04:20:52.960Z DEBUG controller Discovered subnets: [subnet-0204b1b3b885ca98d (eu-west-1a) subnet-037d1d97a6a473fd1 (eu-west-1b) subnet-04c2ca248972479e7 (eu-west-1b) subnet-063d5c7ba912986d5 (eu-west-1a)] {"commit": "00661aa"} +2022-05-12T04:20:53.055Z DEBUG controller Discovered security groups: [sg-03ab1d5d49b00b596 sg-06e7e2ca961ab3bed]{"commit": "00661aa", "provisioner": "default"} +2022-05-12T04:20:53.071Z DEBUG controller Discovered kubernetes version 1.21 {"commit": "00661aa", "provisioner": "default"} +2022-05-12T04:20:53.140Z DEBUG controller Discovered ami-05dc8c3028bc33fd6 for query "/aws/service/eks/optimized-ami/1.21/amazon-linux-2-arm64/recommended/image_id" {"commit": "00661aa", "provisioner": "default"} +2022-05-12T04:20:53.310Z DEBUG controller Created launch template, Karpenter-eksworkshop-eksctl-13655185630813568172 {"commit": "00661aa", "provisioner": "default"} +2022-05-12T04:20:55.201Z INFO controller Launched instance: i-013c73aa33c952ace, hostname: ip-192-168-60-157.eu-west-1.compute.internal, type: c6g.xlarge, zone: eu-west-1b, capacityType: spot {"commit": "00661aa", "provisioner": "default"} +2022-05-12T04:20:55.214Z INFO controller Created node with 2 pods requesting {"cpu":"2125m","memory":"512M","pods":"5"} from types c6g.xlarge {"commit": "00661aa", "provisioner": "default"} +2022-05-12T04:20:55.229Z INFO controller Waiting for unschedulable pods {"commit": "00661aa"} ... ``` diff --git a/content/karpenter/040_karpenter/set_up_the_environment.md b/content/karpenter/040_karpenter/set_up_the_environment.md index 91f3880a..6966d475 100644 --- a/content/karpenter/040_karpenter/set_up_the_environment.md +++ b/content/karpenter/040_karpenter/set_up_the_environment.md @@ -12,10 +12,10 @@ Before we install Karpenter, there are a few things that we will need to prepare Instances launched by Karpenter must run with an InstanceProfile that grants permissions necessary to run containers and configure networking. Karpenter discovers the InstanceProfile using the name `KarpenterNodeRole-${ClusterName}`. ``` -export KARPENTER_VERSION=v0.6.4 +export KARPENTER_VERSION=v0.10.0 echo "export KARPENTER_VERSION=${KARPENTER_VERSION}" >> ~/.bash_profile TEMPOUT=$(mktemp) -curl -fsSL https://karpenter.sh/"${KARPENTER_VERSION}"/getting-started/cloudformation.yaml > $TEMPOUT \ +curl -fsSL https://karpenter.sh/"${KARPENTER_VERSION}"/getting-started/getting-started-with-eksctl/cloudformation.yaml > $TEMPOUT \ && aws cloudformation deploy \ --stack-name Karpenter-${CLUSTER_NAME} \ --template-file ${TEMPOUT} \ @@ -24,7 +24,7 @@ curl -fsSL https://karpenter.sh/"${KARPENTER_VERSION}"/getting-started/cloudform ``` {{% notice tip %}} -This step may take about 2 minutes. In the meantime, you can [download the file](https://karpenter.sh/v0.6.4/getting-started/cloudformation.yaml) and check the content of the CloudFormation Stack. Check how the stack defines a policy, a role and and Instance profile that will be used to associate to the instances launched. You can also head to the **CloudFormation** console and check which resources does the stack deploy. +This step may take about 2 minutes. In the meantime, you can [download the file](https://karpenter.sh/v0.10.0/getting-started/getting-started-with-eksctl/cloudformation.yaml) and check the content of the CloudFormation Stack. Check how the stack defines a policy, a role and and Instance profile that will be used to associate to the instances launched. You can also head to the **CloudFormation** console and check which resources does the stack deploy. {{% /notice %}} Second, grant access to instances using the profile to connect to the cluster. This command adds the Karpenter node role to your aws-auth configmap, allowing nodes with this role to connect to the cluster. @@ -78,6 +78,3 @@ This step is only necessary if this is the first time you’re using EC2 Spot in ``` aws iam create-service-linked-role --aws-service-name spot.amazonaws.com ``` - - - diff --git a/content/karpenter/040_karpenter/set_up_the_provisioner.md b/content/karpenter/040_karpenter/set_up_the_provisioner.md index 4b21b7f0..f952bc68 100644 --- a/content/karpenter/040_karpenter/set_up_the_provisioner.md +++ b/content/karpenter/040_karpenter/set_up_the_provisioner.md @@ -60,7 +60,7 @@ The configuration for this provider is quite simple. We will change in the futur {{% notice info %}} -Karpenter has been designed to be generic and support other Cloud and Infrastructure providers. At the moment of writing this workshop (Karpenter 0.6.4) main implementation and Provisioner available is on AWS. You can read more about the **[configuration available for the AWS Provisioner here](https://karpenter.sh/v0.6.5/aws/)** +Karpenter has been designed to be generic and support other Cloud and Infrastructure providers. At the moment of writing this workshop (**Karpenter 0.10.0**) main implementation and Provisioner available is on AWS. You can read more about the **[configuration available for the AWS Provisioner here](https://karpenter.sh/v0.10.0/aws/)** {{% /notice %}} ## Displaying Karpenter Logs diff --git a/content/karpenter/040_karpenter/using_alternative_provisioners.md b/content/karpenter/040_karpenter/using_alternative_provisioners.md index 2119289d..8df0b02e 100644 --- a/content/karpenter/040_karpenter/using_alternative_provisioners.md +++ b/content/karpenter/040_karpenter/using_alternative_provisioners.md @@ -51,7 +51,7 @@ spec: - labelSelector: matchLabels: app: inflate-team1 - maxSkew: 100 + maxSkew: 1 topologyKey: topology.kubernetes.io/zone whenUnsatisfiable: ScheduleAnyway EOF @@ -132,15 +132,19 @@ The output of Karpenter should look similar to the one below ``` ... -2021-11-16T05:41:30.292Z INFO controller.allocation.provisioner/team1 Starting provisioning loop {"commit": "6468992"} -2021-11-16T05:41:30.293Z INFO controller.allocation.provisioner/team1 Waiting to batch additional pods {"commit": "6468992"} -2021-11-16T05:41:31.476Z INFO controller.allocation.provisioner/team1 Found 4 provisionable pods {"commit": "6468992"} -2021-11-16T05:41:32.490Z INFO controller.allocation.provisioner/team1 Computed packing for 2 pod(s) with instance type option(s) [c5.xlarge c6i.xlarge c3.xlarge c5a.xlarge c5d.xlarge c4.xlarge c1.xlarge c5n.xlarge m3.xlarge m1.xlarge m6i.xlarge m5zn.xlarge m5.xlarge m4.xlarge m5ad.xlarge m5d.xlarge t3.xlarge m5dn.xlarge m5a.xlarge t3a.xlarge] {"commit": "6468992"} -2021-11-16T05:41:32.509Z INFO controller.allocation.provisioner/team1 Computed packing for 2 pod(s) with instance type option(s) [c6i.xlarge c5.xlarge c4.xlarge c5a.xlarge c5ad.xlarge c5d.xlarge c3.xlarge c1.xlarge c5n.xlarge m1.xlarge m3.xlarge m5zn.xlarge m5.xlarge m6i.xlarge t3a.xlarge m5ad.xlarge m5a.xlarge t3.xlarge m5dn.xlarge m5n.xlarge] {"commit": "6468992"} -2021-11-16T05:41:34.216Z INFO controller.allocation.provisioner/team1 Launched instance: i-0f92acfe9a25a80c4, hostname: xxxxxxxxxxxxxxxxxxxxxxxxxxx.compute.internal, type: t3a.xlarge, zone: eu-west-1a, capacityType: on-demand {"commit": "6468992"} -2021-11-16T05:41:34.242Z INFO controller.allocation.provisioner/team1 Bound 2 pod(s) to node xxxxxxxxxxxxxxxxxxxxxxxxxxx.compute.internal {"commit": "6468992"} -2021-11-16T05:41:34.305Z INFO controller.allocation.provisioner/team1 Launched instance: i-02adc32c53c19fcec, hostname: xxxxxxxxxxxxxxxxxxxxxxxxxxxx.compute.internal, type: t3a.xlarge, zone: eu-west-1b, capacityType: on-demand {"commit": "6468992"} -2021-11-16T05:41:34.335Z INFO controller.allocation.provisioner/team1 Bound 2 pod(s) to node xxxxxxxxxxxxxxxxxxxxxxxxxxxx.compute.internal {"commit": "6468992"} +2022-05-12T04:33:28.625Z INFO controller Batched 4 pod(s) in 1.038819076s {"commit": "00661aa"} +2022-05-12T04:33:29.361Z DEBUG controller Discovered 401 EC2 instance types {"commit": "00661aa"} +2022-05-12T04:33:29.482Z DEBUG controller Discovered EC2 instance types zonal offerings {"commit": "00661aa"} +2022-05-12T04:33:29.638Z DEBUG controller Discovered subnets: [subnet-0204b1b3b885ca98d (eu-west-1a) subnet-037d1d97a6a473fd1 (eu-west-1b) subnet-04c2ca248972479e7 (eu-west-1b) subnet-063d5c7ba912986d5 (eu-west-1a)] {"commit": "00661aa"} +2022-05-12T04:33:29.728Z DEBUG controller Discovered security groups: [sg-03ab1d5d49b00b596 sg-06e7e2ca961ab3bed] {"commit": "00661aa", "provisioner": "team1"} +2022-05-12T04:33:29.732Z DEBUG controller Discovered kubernetes version 1.21 {"commit": "00661aa", "provisioner": "team1"} +2022-05-12T04:33:29.796Z DEBUG controller Discovered ami-0440c10a3f77514d8 for query "/aws/service/eks/optimized-ami/1.21/amazon-linux-2/recommended/image_id" {"commit": "00661aa", "provisioner": "team1"} +2022-05-12T04:33:29.971Z DEBUG controller Created launch template, Karpenter-eksworkshop-eksctl-2228580094236845875 {"commit": "00661aa", "provisioner": "team1"} +2022-05-12T04:33:31.885Z INFO controller Launched instance: i-02df8ea1e99895e78, hostname: ip-192- 68-14-13.eu-west-1.compute.internal, type: t3a.xlarge, zone: eu-west-1a, capacityType: on-demand {"commit": "00661aa", "provisioner": "team1"} +2022-05-12T04:33:31.896Z INFO controller Created node with 2 pods requesting {"cpu":"2125m","memory":"512M","pods":"4"} from types c4.xlarge, c6a.xlarge, c6i.xlarge, c5.xlarge, c5a.xlarge and 263 other(s) {"commit": "00661aa", "provisioner": "team1"} +2022-05-12T04:33:32.443Z INFO controller Launched instance: i-0b6984823f26d5b15, hostname: ip-192-168-39-218.eu-west-1.compute.internal, type: t3a.xlarge, zone: eu-west-1b, capacityType: on-demand {"commit": "00661aa", "provisioner": "team1"} +2022-05-12T04:33:32.453Z INFO controller Created node with 2 pods requesting {"cpu":"2125m","memory":"512M","pods":"4"} from types c4.xlarge, c6a.xlarge, c6i.xlarge, c5.xlarge, c5a.xlarge and 267 other(s) {"commit": "00661aa", "provisioner": "team1"} +2022-05-12T04:33:32.464Z INFO controller Waiting for unschedulable pods {"commit": "00661aa"} ... ```