Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Rancher as a cloud provider #4041

Closed
wants to merge 8 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion cluster-autoscaler/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,13 @@ You should also take a look at the notes and "gotchas" for your specific cloud p
* [CloudStack](./cloudprovider/cloudstack/README.md)
* [HuaweiCloud](./cloudprovider/huaweicloud/README.md)
* [Hetzner](./cloudprovider/hetzner/README.md)
* [Packet](./cloudprovider/packet/README.md#notes)
* [Packet](./cloudprovider/packet/README.md#notes)
* [IonosCloud](./cloudprovider/ionoscloud/README.md)
* [OVHcloud](./cloudprovider/ovhcloud/README.md)
* [Linode](./cloudprovider/linode/README.md)
* [ClusterAPI](./cloudprovider/clusterapi/README.md)
* [BizflyCloud](./cloudprovider/bizflycloud/README.md)
* [Rancher](./cloudprovider/rancher/README.md)

# Releases

Expand Down Expand Up @@ -159,3 +160,4 @@ Supported cloud providers:
* Linode https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/linode/README.md
* Hetzner https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/hetzner/README.md
* Cluster API https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/clusterapi/README.md
* Rancher https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/rancher/README.md
6 changes: 5 additions & 1 deletion cluster-autoscaler/cloudprovider/builder/builder_all.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// +build !gce,!aws,!azure,!kubemark,!alicloud,!magnum,!digitalocean,!clusterapi,!huaweicloud,!ionoscloud,!linode,!hetzner,!bizflycloud
// +build !gce,!aws,!azure,!kubemark,!alicloud,!magnum,!digitalocean,!clusterapi,!huaweicloud,!ionoscloud,!linode,!hetzner,!bizflycloud,!rancher

/*
Copyright 2018 The Kubernetes Authors.
Expand Down Expand Up @@ -37,6 +37,7 @@ import (
"k8s.io/autoscaler/cluster-autoscaler/cloudprovider/magnum"
"k8s.io/autoscaler/cluster-autoscaler/cloudprovider/ovhcloud"
"k8s.io/autoscaler/cluster-autoscaler/cloudprovider/packet"
"k8s.io/autoscaler/cluster-autoscaler/cloudprovider/rancher"
"k8s.io/autoscaler/cluster-autoscaler/config"
)

Expand All @@ -58,6 +59,7 @@ var AvailableCloudProviders = []string{
cloudprovider.IonoscloudProviderName,
cloudprovider.LinodeProviderName,
cloudprovider.BizflyCloudProviderName,
cloudprovider.RancherProviderName,
}

// DefaultCloudProvider is GCE.
Expand Down Expand Up @@ -99,6 +101,8 @@ func buildCloudProvider(opts config.AutoscalingOptions, do cloudprovider.NodeGro
return ionoscloud.BuildIonosCloud(opts, do, rl)
case cloudprovider.LinodeProviderName:
return linode.BuildLinode(opts, do, rl)
case cloudprovider.RancherProviderName:
return rancher.BuildRancher(opts, do, rl)
}
return nil
}
42 changes: 42 additions & 0 deletions cluster-autoscaler/cloudprovider/builder/builder_rancher.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
// +build rancher

/*
Copyright 2021 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package builder

import (
"k8s.io/autoscaler/cluster-autoscaler/cloudprovider"
"k8s.io/autoscaler/cluster-autoscaler/cloudprovider/rancher"
"k8s.io/autoscaler/cluster-autoscaler/config"
)

// AvailableCloudProviders supported by the rancher provider builder.
var AvailableCloudProviders = []string{
cloudprovider.RancherProviderName,
}

// DefaultCloudProvider for do-only build is rancher.
const DefaultCloudProvider = cloudprovider.RancherProviderName

func buildCloudProvider(opts config.AutoscalingOptions, do cloudprovider.NodeGroupDiscoveryOptions, rl *cloudprovider.ResourceLimiter) cloudprovider.CloudProvider {
switch opts.CloudProviderName {
case cloudprovider.RancherProviderName:
return rancher.BuildRancher(opts, do, rl)
}

return nil
}
2 changes: 2 additions & 0 deletions cluster-autoscaler/cloudprovider/cloud_provider.go
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,8 @@ const (
OVHcloudProviderName = "ovhcloud"
// LinodeProviderName gets the provider name of linode
LinodeProviderName = "linode"
// RancherProviderName gets the provider name of rancher
RancherProviderName = "rancher"
)

// CloudProvider contains configuration info and functions for interacting with
Expand Down
54 changes: 54 additions & 0 deletions cluster-autoscaler/cloudprovider/rancher/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Cluster Autoscaler for Rancher

The cluster autoscaler for Rancher scales nodes within any specified Rancher Kubernetes Engine cluster's node pool.

# Requirements

Rancher version >= 2.5.6

# Configuration

The cluster-autoscaler for Rancher needs a configuration file to work by using --cloud-config parameter.

Here an [example](examples/autoscaler-config-example.yaml).

```yaml
apiVersion: v1
kind: Secret
metadata:
name: cluster-autoscaler-cloud-config
namespace: kube-system
type: Opaque
stringData:
cloud-config: |-
[Global]
url=https://rancherapi.com/v3
access=your-token
secret=your-secret
cluster-id=c-abcdef
autoscaler_node_arg: "2:6:c-abcdef:np-abcde" # Your NodePool ID
```

You have to create a new API Key from your Rancher Dashboard to get the `access` and `secret` values to use the Autoscaler.

# Development

Make sure you're inside the root path of the [autoscaler repository](https://github.com/kubernetes/autoscaler)

1.) Build the `cluster-autoscaler` binary:

```
make build-in-docker
```

2.) Build the docker image:

```
docker build -t rancher/cluster-autoscaler:dev .
```

3.) Push the docker image to Docker hub:

```
docker push rancher/cluster-autoscaler:dev
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add some information about contacts of maintainers here?

It would be helpful for maintainers to reach out to the cloud provider owners for release coordination when necessary.

Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
apiVersion: v1
kind: Secret
metadata:
name: cluster-autoscaler-cloud-config
namespace: kube-system
type: Opaque
stringData:
cloud-config: |-
[Global]
url=https://rancherapi.com/v3
access=token-abcdef
secret=ksjdhfiusdhfkjsdfhisudhfnskjdfhskjdfhksdjfhksjdfhksdjf
cluster-id=c-abcdef
autoscaler_node_arg: "2:6:c-abcdef:np-abcde"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: new line

Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
apiVersion: rbac.authorization.k8s.io/v1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems it ends with txt. is it used for rendering templates?

kind: ClusterRole
metadata:
name: cluster-autoscaler-role
rules:
- apiGroups: [""]
resources: ["events", "endpoints"]
verbs: ["create", "patch"]
- apiGroups: [""]
resources: ["pods/eviction"]
verbs: ["create"]
- apiGroups: [""]
resources: ["pods/status"]
verbs: ["update"]
- apiGroups: [""]
resources: ["endpoints"]
resourceNames: ["cluster-autoscaler"]
verbs: ["get", "update"]
- apiGroups: [""]
resources: ["nodes"]
verbs: ["watch", "list", "get", "update"]
- apiGroups: [""]
resources:
- "pods"
- "services"
- "replicationcontrollers"
- "persistentvolumeclaims"
- "persistentvolumes"
verbs: ["watch", "list", "get"]
- apiGroups: ["batch"]
resources: ["jobs"]
verbs: ["watch", "list", "get"]
- apiGroups: ["policy"]
resources: ["poddisruptionbudgets"]
verbs: ["watch", "list"]
- apiGroups: ["apps"]
resources: ["daemonsets", "replicasets", "statefulsets"]
verbs: ["watch", "list", "get"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses", "csinodes"]
verbs: ["watch", "list", "get"]
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["create","list","watch"]
- apiGroups: [""]
resources: ["configmaps"]
resourceNames: ["cluster-autoscaler-status", "cluster-autoscaler-priority-expander"]
verbs: ["delete", "get", "update"]
- apiGroups: ["coordination.k8s.io"]
resources: ["leases"]
verbs: ["watch", "list", "get", "create", "update", "patch", "delete", "deletecollection"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: cluster-autoscaler-rolebinding
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-autoscaler-role
subjects:
- kind: ServiceAccount
name: cluster-autoscaler-account
namespace: kube-system

---
apiVersion: v1
kind: ServiceAccount
metadata:
name: cluster-autoscaler-account
namespace: kube-system

---
kind: Deployment
apiVersion: apps/v1
metadata:
name: cluster-autoscaler
namespace: kube-system
labels:
app: cluster-autoscaler
spec:
replicas: 1
selector:
matchLabels:
app: cluster-autoscaler
template:
metadata:
namespace: kube-system
labels:
app: cluster-autoscaler
spec:
serviceAccountName: cluster-autoscaler-account
containers:
- name: cluster-autoscaler
image: us.gcr.io/gns-hosting-dev/cluster-autoscaler-amd64:{{ env "TAG" }}
imagePullPolicy: Always
command:
- ./cluster-autoscaler
- --alsologtostderr
- --cloud-config=/config/cloud-config
- --cloud-provider=rancher
- --nodes=$(NODE_ARG)
volumeMounts:
- name: cloud-config
mountPath: /config
readOnly: true
env:
- name: NODE_ARG
valueFrom:
secretKeyRef:
name: cluster-autoscaler-cloud-config
key: autoscaler_node_arg
volumes:
- name: cloud-config
secret:
secretName: cluster-autoscaler-cloud-config
Loading