Skip to content

Commit

Permalink
Merge pull request #29 from airbnb/anton_kirillov--cluster-autoscaler…
Browse files Browse the repository at this point in the history
…-patch-cluster-autoscaler-1.21.3-airbnb0

cluster autoscaler patch cluster autoscaler 1.21.3 airbnb0
  • Loading branch information
akirillov authored Oct 27, 2022
2 parents c95c261 + fa16c5e commit c6bada7
Show file tree
Hide file tree
Showing 408 changed files with 112,071 additions and 62,462 deletions.
6 changes: 6 additions & 0 deletions cluster-autoscaler/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -630,6 +630,12 @@ would match the cluster size. This expander is described in more details

* `priority` - selects the node group that has the highest priority assigned by the user. It's configuration is described in more details [here](expander/priority/readme.md)


Multiple expanders may be passed, i.e.
`.cluster-autoscaler --expander=priority,least-waste`

This will cause the `least-waste` expander to be used as a fallback in the event that the priority expander selects multiple node groups. In general, a list of expanders can be used, where the output of one is passed to the next and the final decision by randomly selecting one. An expander must not appear in the list more than once.

### Does CA respect node affinity when selecting node groups to scale up?

CA respects `nodeSelector` and `requiredDuringSchedulingIgnoredDuringExecution` in nodeAffinity given that you have labelled your node groups accordingly. If there is a pod that cannot be scheduled with either `nodeSelector` or `requiredDuringSchedulingIgnoredDuringExecution` specified, CA will only consider node groups that satisfy those requirements for expansion.
Expand Down
11 changes: 5 additions & 6 deletions cluster-autoscaler/cloudprovider/alicloud/alicloud_manager.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ import (
"k8s.io/autoscaler/cluster-autoscaler/cloudprovider"
"k8s.io/autoscaler/cluster-autoscaler/cloudprovider/alicloud/alibaba-cloud-sdk-go/services/ess"
klog "k8s.io/klog/v2"
kubeletapis "k8s.io/kubelet/pkg/apis"
"math/rand"
"time"
)
Expand Down Expand Up @@ -237,13 +236,13 @@ func (m *AliCloudManager) buildNodeFromTemplate(sg *Asg, template *sgTemplate) (

func buildGenericLabels(template *sgTemplate, nodeName string) map[string]string {
result := make(map[string]string)
result[kubeletapis.LabelArch] = cloudprovider.DefaultArch
result[kubeletapis.LabelOS] = cloudprovider.DefaultOS
result[apiv1.LabelArchStable] = cloudprovider.DefaultArch
result[apiv1.LabelOSStable] = cloudprovider.DefaultOS

result[apiv1.LabelInstanceType] = template.InstanceType.instanceTypeID
result[apiv1.LabelInstanceTypeStable] = template.InstanceType.instanceTypeID

result[apiv1.LabelZoneRegion] = template.Region
result[apiv1.LabelZoneFailureDomain] = template.Zone
result[apiv1.LabelTopologyRegion] = template.Region
result[apiv1.LabelTopologyZone] = template.Zone
result[apiv1.LabelHostname] = nodeName

// append custom node labels
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ func TestBuildGenericLabels(t *testing.T) {
}
nodeName := "virtual-node"
labels := buildGenericLabels(template, nodeName)
assert.Equal(t, labels[apiv1.LabelInstanceType], template.InstanceType.instanceTypeID)
assert.Equal(t, labels[apiv1.LabelInstanceTypeStable], template.InstanceType.instanceTypeID)
}

func TestExtractLabelsFromAsg(t *testing.T) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ Note: The keys for the tags that you entered don't have values. Cluster Autoscal
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:DescribeAutoScalingInstances",
"autoscaling:DescribeLaunchConfigurations",
"autoscaling:DescribeScalingActivities",
"autoscaling:DescribeTags",
"autoscaling:SetDesiredCapacity",
"autoscaling:TerminateInstanceInAutoScalingGroup"
Expand Down
50 changes: 50 additions & 0 deletions cluster-autoscaler/cloudprovider/aws/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,42 @@ by specifying Auto Scaling Group ARNs in the `Resource` list of the policy. More
information can be found
[here](https://docs.aws.amazon.com/autoscaling/latest/userguide/control-access-using-iam.html#policy-auto-scaling-resources).

*NOTE:* The below policies/arguments to the Cluster Autoscaler need to be modified as appropriate
for the names of your ASGs, as well as account ID and AWS region before being used.

The following policy provides the minimum privileges necessary for Cluster Autoscaler to run.
When using this policy, you cannot use autodiscovery of ASGs. In addition, it restricts the
IAM permissions to the node groups the Cluster Autoscaler is configured to scale.

This in turn means that you must pass the following arguments to the Cluster Autoscaler
binary, replacing min and max node counts and the ASG:

```bash
--aws-use-static-instance-list=false
--nodes=1:100:exampleASG1
--nodes=1:100:exampleASG2
```

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:DescribeAutoScalingInstances",
"autoscaling:DescribeLaunchConfigurations",
"autoscaling:DescribeScalingActivities",
"autoscaling:SetDesiredCapacity",
"autoscaling:TerminateInstanceInAutoScalingGroup"
],
"Resource": ["arn:aws:autoscaling:${YOUR_CLUSTER_AWS_REGION}:${YOUR_AWS_ACCOUNT_ID}:autoScalingGroup:*:autoScalingGroupName/${YOUR_ASG_NAME}"]
}
]
}
```

### Using OIDC Federated Authentication
OIDC federated authentication allows your service to assume an IAM role and interact with AWS services without having to store credentials as environment variables. For an example of how to use AWS IAM OIDC with the Cluster Autoscaler please see [here](CA_with_AWS_IAM_OIDC.md).

Expand Down Expand Up @@ -144,6 +180,20 @@ be labeled or tainted when they join the cluster, such as:
* `k8s.io/cluster-autoscaler/node-template/label/foo`: `bar`
* `k8s.io/cluster-autoscaler/node-template/taint/dedicated`: `NoSchedule`

ASG labels can specify autoscaling options, overriding the global cluster-autoscaler
settings for the labeled ASGs. Those labels takes the same values format as the
cluster-autoscaler command line flags they override (a float or a duration, encoded
as string). Currently supported autoscaling options (and example values) are:

* `k8s.io/cluster-autoscaler/node-template/autoscaling-options/scaledownutilizationthreshold`: `0.5`
(overrides `--scale-down-utilization-threshold` value for that specific ASG)
* `k8s.io/cluster-autoscaler/node-template/autoscaling-options/scaledowngpuutilizationthreshold`: `0.5`
(overrides `--scale-down-gpu-utilization-threshold` value for that specific ASG)
* `k8s.io/cluster-autoscaler/node-template/autoscaling-options/scaledownunneededtime`: `10m0s`
(overrides `--scale-down-unneeded-time` value for that specific ASG)
* `k8s.io/cluster-autoscaler/node-template/autoscaling-options/scaledownunreadytime`: `20m0s`
(overrides `--scale-down-unready-time` value for that specific ASG)

**NOTE:** It is your responsibility to ensure such labels and/or taints are
applied via the node's kubelet configuration at startup.

Expand Down
263 changes: 0 additions & 263 deletions cluster-autoscaler/cloudprovider/aws/auto_scaling.go

This file was deleted.

Loading

0 comments on commit c6bada7

Please sign in to comment.