Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename packet to equinix metal #4580

Merged
merged 3 commits into from
Jan 13, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion cluster-autoscaler/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ this document:
* [CA doesn’t work, but it used to work yesterday. Why?](#ca-doesnt-work-but-it-used-to-work-yesterday-why)
* [How can I check what is going on in CA ?](#how-can-i-check-what-is-going-on-in-ca-)
* [What events are emitted by CA?](#what-events-are-emitted-by-ca)
* [My cluster is below minimum / above maximum number of nodes, but CA did not fix that! Why?](#my-cluster-is-below-minimum--above-maximum-number-of-nodes-but-ca-did-not-fix-that-why)
* [What happens in scale-up when I have no more quota in the cloud provider?](#what-happens-in-scale-up-when-i-have-no-more-quota-in-the-cloud-provider)
* [Developer](#developer)
* [How can I run e2e tests?](#how-can-i-run-e2e-tests)
Expand Down Expand Up @@ -669,7 +670,7 @@ after scale-up. This is useful when you have different classes of nodes, for exa

* `price` - select the node group that will cost the least and, at the same time, whose machines
would match the cluster size. This expander is described in more details
[HERE](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/proposals/pricing.md). Currently it works only for GCE, GKE and Packet (patches welcome.)
[HERE](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/proposals/pricing.md). Currently it works only for GCE, GKE and Equinix Metal (patches welcome.)

* `priority` - selects the node group that has the highest priority assigned by the user. It's configuration is described in more details [here](expander/priority/readme.md)

Expand Down
4 changes: 2 additions & 2 deletions cluster-autoscaler/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ You should also take a look at the notes and "gotchas" for your specific cloud p
* [CloudStack](./cloudprovider/cloudstack/README.md)
* [HuaweiCloud](./cloudprovider/huaweicloud/README.md)
* [Hetzner](./cloudprovider/hetzner/README.md)
* [Packet](./cloudprovider/packet/README.md#notes)
* [Equinix Metal](./cloudprovider/packet/README.md#notes)
* [IonosCloud](./cloudprovider/ionoscloud/README.md)
* [OVHcloud](./cloudprovider/ovhcloud/README.md)
* [Linode](./cloudprovider/linode/README.md)
Expand Down Expand Up @@ -162,7 +162,7 @@ Supported cloud providers:
* DigitalOcean https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/digitalocean/README.md
* CloudStack https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/cloudstack/README.md
* Exoscale https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/exoscale/README.md
* Packet https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/packet/README.md
* Equinix Metal https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/packet/README.md
* OVHcloud https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/ovhcloud/README.md
* Linode https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/linode/README.md
* Hetzner https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/hetzner/README.md
Expand Down
38 changes: 20 additions & 18 deletions cluster-autoscaler/cloudprovider/packet/README.md
Original file line number Diff line number Diff line change
@@ -1,42 +1,44 @@
<!--TODO: Remove "previously referred to as master" references from this doc once this terminology is fully removed from k8s-->
# Cluster Autoscaler for Packet
# Cluster Autoscaler for Equinix Metal

The cluster autoscaler for [Packet](https://packet.com) worker nodes performs
The cluster autoscaler for [Equinix Metal](https://metal.equinix.com) worker nodes performs
autoscaling within any specified nodepools. It will run as a `Deployment` in
your cluster. The nodepools are specified using tags on Packet.
your cluster. The nodepools are specified using tags on Equinix Metal.

Note: Packet was acquired by Equinix in 2020 and renamed to Equinix Metal.

This README will go over some of the necessary steps required to get
the cluster autoscaler up and running.

## Permissions and credentials

The autoscaler needs a `ServiceAccount` with permissions for Kubernetes and
requires credentials for interacting with Packet.
requires credentials for interacting with Equinix Metal.

An example `ServiceAccount` is given in [examples/cluster-autoscaler-svcaccount.yaml](examples/cluster-autoscaler-svcaccount.yaml).

The credentials for authenticating with Packet are stored in a secret and
The credentials for authenticating with Equinix Metal are stored in a secret and
provided as an env var to the container. [examples/cluster-autoscaler-secret](examples/cluster-autoscaler-secret.yaml)
In the above file you can modify the following fields:

| Secret | Key | Value |
|---------------------------------|-------------------------|------------------------------------------------------------------------------------------------------------------------------------|
| cluster-autoscaler-packet | authtoken | Your Packet API token. It must be base64 encoded. |
| cluster-autoscaler-cloud-config | Global/project-id | Your Packet project id |
| cluster-autoscaler-packet | authtoken | Your Equinix Metal API token. It must be base64 encoded. |
| cluster-autoscaler-cloud-config | Global/project-id | Your Equinix Metal project id |
| cluster-autoscaler-cloud-config | Global/api-server | The ip:port for you cluster's k8s api (e.g. K8S_MASTER_PUBLIC_IP:6443) |
| cluster-autoscaler-cloud-config | Global/facility | The Packet facility for the devices in your nodepool (eg: ams1) |
| cluster-autoscaler-cloud-config | Global/plan | The Packet plan (aka size/flavor) for new nodes in the nodepool (eg: t1.small.x86) |
| cluster-autoscaler-cloud-config | Global/facility | The Equinix Metal facility for the devices in your nodepool (eg: sv15) |
| cluster-autoscaler-cloud-config | Global/plan | The Equinix Metal plan (aka size/flavor) for new nodes in the nodepool (eg: c3.small.x86) |
| cluster-autoscaler-cloud-config | Global/billing | The billing interval for new nodes (default: hourly) |
| cluster-autoscaler-cloud-config | Global/os | The OS image to use for new nodes (default: ubuntu_18_04). If you change this also update cloudinit. |
| cluster-autoscaler-cloud-config | Global/cloudinit | The base64 encoded [user data](https://support.packet.com/kb/articles/user-data) submitted when provisioning devices. In the example file, the default value has been tested with Ubuntu 18.04 to install Docker & kubelet and then to bootstrap the node into the cluster using kubeadm. The kubeadm, kubelet, kubectl are pinned to version 1.17.4. For a different base OS or bootstrap method, this needs to be customized accordingly|
| cluster-autoscaler-cloud-config | Global/cloudinit | The base64 encoded [user data](https://metal.equinix.com/developers/docs/servers/user-data/) submitted when provisioning devices. In the example file, the default value has been tested with Ubuntu 18.04 to install Docker & kubelet and then to bootstrap the node into the cluster using kubeadm. The kubeadm, kubelet, kubectl are pinned to version 1.17.4. For a different base OS or bootstrap method, this needs to be customized accordingly|
| cluster-autoscaler-cloud-config | Global/reservation | The values "require" or "prefer" will request the next available hardware reservation for new devices in selected facility & plan. If no hardware reservations match, "require" will trigger a failure, while "prefer" will launch on-demand devices instead (default: none) |
| cluster-autoscaler-cloud-config | Global/hostname-pattern | The pattern for the names of new Packet devices (default: "k8s-{{.ClusterName}}-{{.NodeGroup}}-{{.RandString8}}" ) |
| cluster-autoscaler-cloud-config | Global/hostname-pattern | The pattern for the names of new Equinix Metal devices (default: "k8s-{{.ClusterName}}-{{.NodeGroup}}-{{.RandString8}}" ) |

You can always update the secret with more nodepool definitions (with different plans etc.) as shown in the example, but you should always provide a default nodepool configuration.

## Configure nodepool and cluster names using Packet tags
## Configure nodepool and cluster names using Equinix Metal tags

The Packet API does not yet have native support for groups or pools of devices. So we use tags to specify them. Each Packet device that's a member of the "cluster1" cluster should have the tag k8s-cluster-cluster1. The devices that are members of the "pool1" nodepool should also have the tag k8s-nodepool-pool1. Once you have a Kubernetes cluster running on Packet, use the Packet Portal or API to tag the nodes accordingly.
The Equinix Metal API does not yet have native support for groups or pools of devices. So we use tags to specify them. Each Equinix Metal device that's a member of the "cluster1" cluster should have the tag k8s-cluster-cluster1. The devices that are members of the "pool1" nodepool should also have the tag k8s-nodepool-pool1. Once you have a Kubernetes cluster running on Equinix Metal, use the Equinix Metal Portal or API to tag the nodes accordingly.

## Autoscaler deployment

Expand All @@ -48,7 +50,7 @@ to match your cluster.
|-----------------------|------------------------------------------------------------------------------------------------------------|
| --cluster-name | The name of your Kubernetes cluster. It should correspond to the tags that have been applied to the nodes. |
| --nodes | Of the form `min:max:NodepoolName`. For multiple nodepools you can add the same argument multiple times. E.g. for pool1, pool2 you would add `--nodes=0:10:pool1` and `--nodes=0:10:pool2`. In addition, each node provisioned by the autoscaler will have a label with key: `pool` and with value: `NodepoolName`. These labels can be useful when there is a need to target specific nodepools. |
| --expander=price | This is an optional argument which allows the cluster-autoscaler to take into account the pricing of the Packet nodes when scaling with multiple nodepools. |
| --expander=price | This is an optional argument which allows the cluster-autoscaler to take into account the pricing of the Equinix Metal nodes when scaling with multiple nodepools. |

## Target Specific Nodepools (New!)

Expand All @@ -66,7 +68,7 @@ affinity:
values:
- pool3
```
Target a nodepool with a specific Packet instance:
Target a nodepool with a specific Equinix Metal instance:
```
affinity:
nodeAffinity:
Expand All @@ -76,7 +78,7 @@ affinity:
- key: beta.kubernetes.io/instance-type
operator: In
values:
- t1.small.x86
- c3.small.x86
```

## CCM and Controller node labels
Expand All @@ -100,7 +102,7 @@ field `.spec.ProviderID`.

Autoscaler assumes that control plane nodes in your cluster are identified by the label
`node-role.kubernetes.io/master`. If for some reason, this assumption is not true in your case, then set the
envirnment variable in the deployment:
environment variable in the deployment:

```
env:
Expand All @@ -120,7 +122,7 @@ the control plane (previously referred to as master) node.

### Changes

1. It is now possible to use multiple nodepools, scale nodepools to 0 nodes and prioritize scaling of specific nodepools by taking into account the pricing of the Packet instances.
1. It is now possible to use multiple nodepools, scale nodepools to 0 nodes and prioritize scaling of specific nodepools by taking into account the pricing of the Equinix Metal instances.

2. In order to take advantage of the new features mentioned above, you might need to update the cloud-config and the autoscaler deployment as shown in the examples. For example, the default/global cloud-config is applied to all the nodepools and if you want to override it for a specific nodepool you have to modify the cloud-config according to the examples.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
apiVersion: v1
data:
authtoken: YOUR_PACKET_API_TOKEN
authtoken: YOUR_EQUINIX_METAL_API_TOKEN
kind: Secret
metadata:
name: cluster-autoscaler-packet
Expand All @@ -21,22 +21,22 @@ stringData:
# In the cloud-config you must always have a valid default nodegroup
cloud-config: |-
[nodegroupdef "default"]
project-id=YOUR_PACKET_PROJECT_ID
project-id=YOUR_EQUINIX_METAL_PROJECT_ID
api-server-endpoint=YOUR_KUBERNETES_API_IP_ADDRESS:YOUR_KUBERNETES_API_PORT
facility=ams1
facility=sv15
os=ubuntu_18_04
plan=t1.small.x86
plan=c3.small.x86
billing=hourly
cloudinit=IyEvYmluL2Jhc2gKZXhwb3J0IERFQklBTl9GUk9OVEVORD1ub25pbnRlcmFjdGl2ZQphcHQtZ2V0IHVwZGF0ZSAmJiBhcHQtZ2V0IGluc3RhbGwgLXkgYXB0LXRyYW5zcG9ydC1odHRwcyBjYS1jZXJ0aWZpY2F0ZXMgY3VybCBzb2Z0d2FyZS1wcm9wZXJ0aWVzLWNvbW1vbgpjdXJsIC1mc1NMIGh0dHBzOi8vZG93bmxvYWQuZG9ja2VyLmNvbS9saW51eC91YnVudHUvZ3BnIHwgYXB0LWtleSBhZGQgLQpjdXJsIC1zIGh0dHBzOi8vcGFja2FnZXMuY2xvdWQuZ29vZ2xlLmNvbS9hcHQvZG9jL2FwdC1rZXkuZ3BnIHwgYXB0LWtleSBhZGQgLQpjYXQgPDxFT0YgPi9ldGMvYXB0L3NvdXJjZXMubGlzdC5kL2t1YmVybmV0ZXMubGlzdApkZWIgaHR0cHM6Ly9hcHQua3ViZXJuZXRlcy5pby8ga3ViZXJuZXRlcy14ZW5pYWwgbWFpbgpFT0YKYWRkLWFwdC1yZXBvc2l0b3J5ICAgImRlYiBbYXJjaD1hbWQ2NF0gaHR0cHM6Ly9kb3dubG9hZC5kb2NrZXIuY29tL2xpbnV4L3VidW50dSAgICQobHNiX3JlbGVhc2UgLWNzKSAgIHN0YWJsZSIKYXB0LWdldCB1cGRhdGUKYXB0LWdldCB1cGdyYWRlIC15CmFwdC1nZXQgaW5zdGFsbCAteSBrdWJlbGV0PTEuMTcuNC0wMCBrdWJlYWRtPTEuMTcuNC0wMCBrdWJlY3RsPTEuMTcuNC0wMAphcHQtbWFyayBob2xkIGt1YmVsZXQga3ViZWFkbSBrdWJlY3RsCmN1cmwgLWZzU0wgaHR0cHM6Ly9kb3dubG9hZC5kb2NrZXIuY29tL2xpbnV4L3VidW50dS9ncGcgfCBhcHQta2V5IGFkZCAtCmFkZC1hcHQtcmVwb3NpdG9yeSAiZGViIFthcmNoPWFtZDY0XSBodHRwczovL2Rvd25sb2FkLmRvY2tlci5jb20vbGludXgvdWJ1bnR1IGJpb25pYyBzdGFibGUiCmFwdCB1cGRhdGUKYXB0IGluc3RhbGwgLXkgZG9ja2VyLWNlPTE4LjA2LjJ+Y2V+My0wfnVidW50dQpjYXQgPiAvZXRjL2RvY2tlci9kYWVtb24uanNvbiA8PEVPRgp7CiAgImV4ZWMtb3B0cyI6IFsibmF0aXZlLmNncm91cGRyaXZlcj1zeXN0ZW1kIl0sCiAgImxvZy1kcml2ZXIiOiAianNvbi1maWxlIiwKICAibG9nLW9wdHMiOiB7CiAgICAibWF4LXNpemUiOiAiMTAwbSIKICB9LAogICJzdG9yYWdlLWRyaXZlciI6ICJvdmVybGF5MiIKfQpFT0YKbWtkaXIgLXAgL2V0Yy9zeXN0ZW1kL3N5c3RlbS9kb2NrZXIuc2VydmljZS5kCnN5c3RlbWN0bCBkYWVtb24tcmVsb2FkCnN5c3RlbWN0bCByZXN0YXJ0IGRvY2tlcgpzd2Fwb2ZmIC1hCm12IC9ldGMvZnN0YWIgL2V0Yy9mc3RhYi5vbGQgJiYgZ3JlcCAtdiBzd2FwIC9ldGMvZnN0YWIub2xkID4gL2V0Yy9mc3RhYgpjYXQgPDxFT0YgfCB0ZWUgL2V0Yy9kZWZhdWx0L2t1YmVsZXQKS1VCRUxFVF9FWFRSQV9BUkdTPS0tY2xvdWQtcHJvdmlkZXI9ZXh0ZXJuYWwKRU9GCmt1YmVhZG0gam9pbiAtLWRpc2NvdmVyeS10b2tlbi11bnNhZmUtc2tpcC1jYS12ZXJpZmljYXRpb24gLS10b2tlbiB7ey5Cb290c3RyYXBUb2tlbklEfX0ue3suQm9vdHN0cmFwVG9rZW5TZWNyZXR9fSB7ey5BUElTZXJ2ZXJFbmRwb2ludH19Cg==
reservation=prefer
hostname-pattern=k8s-{{.ClusterName}}-{{.NodeGroup}}-{{.RandString8}}

[nodegroupdef "pool2"]
project-id=YOUR_PACKET_PROJECT_ID
project-id=YOUR_EQUINIX_METAL_PROJECT_ID
api-server-endpoint=YOUR_KUBERNETES_API_IP_ADDRESS:YOUR_KUBERNETES_API_PORT
facility=ams1
facility=sv15
os=ubuntu_18_04
plan=t1.small.x86
plan=c3.small.x86
billing=hourly
cloudinit=IyEvYmluL2Jhc2gKZXhwb3J0IERFQklBTl9GUk9OVEVORD1ub25pbnRlcmFjdGl2ZQphcHQtZ2V0IHVwZGF0ZSAmJiBhcHQtZ2V0IGluc3RhbGwgLXkgYXB0LXRyYW5zcG9ydC1odHRwcyBjYS1jZXJ0aWZpY2F0ZXMgY3VybCBzb2Z0d2FyZS1wcm9wZXJ0aWVzLWNvbW1vbgpjdXJsIC1mc1NMIGh0dHBzOi8vZG93bmxvYWQuZG9ja2VyLmNvbS9saW51eC91YnVudHUvZ3BnIHwgYXB0LWtleSBhZGQgLQpjdXJsIC1zIGh0dHBzOi8vcGFja2FnZXMuY2xvdWQuZ29vZ2xlLmNvbS9hcHQvZG9jL2FwdC1rZXkuZ3BnIHwgYXB0LWtleSBhZGQgLQpjYXQgPDxFT0YgPi9ldGMvYXB0L3NvdXJjZXMubGlzdC5kL2t1YmVybmV0ZXMubGlzdApkZWIgaHR0cHM6Ly9hcHQua3ViZXJuZXRlcy5pby8ga3ViZXJuZXRlcy14ZW5pYWwgbWFpbgpFT0YKYWRkLWFwdC1yZXBvc2l0b3J5ICAgImRlYiBbYXJjaD1hbWQ2NF0gaHR0cHM6Ly9kb3dubG9hZC5kb2NrZXIuY29tL2xpbnV4L3VidW50dSAgICQobHNiX3JlbGVhc2UgLWNzKSAgIHN0YWJsZSIKYXB0LWdldCB1cGRhdGUKYXB0LWdldCB1cGdyYWRlIC15CmFwdC1nZXQgaW5zdGFsbCAteSBrdWJlbGV0PTEuMTcuNC0wMCBrdWJlYWRtPTEuMTcuNC0wMCBrdWJlY3RsPTEuMTcuNC0wMAphcHQtbWFyayBob2xkIGt1YmVsZXQga3ViZWFkbSBrdWJlY3RsCmN1cmwgLWZzU0wgaHR0cHM6Ly9kb3dubG9hZC5kb2NrZXIuY29tL2xpbnV4L3VidW50dS9ncGcgfCBhcHQta2V5IGFkZCAtCmFkZC1hcHQtcmVwb3NpdG9yeSAiZGViIFthcmNoPWFtZDY0XSBodHRwczovL2Rvd25sb2FkLmRvY2tlci5jb20vbGludXgvdWJ1bnR1IGJpb25pYyBzdGFibGUiCmFwdCB1cGRhdGUKYXB0IGluc3RhbGwgLXkgZG9ja2VyLWNlPTE4LjA2LjJ+Y2V+My0wfnVidW50dQpjYXQgPiAvZXRjL2RvY2tlci9kYWVtb24uanNvbiA8PEVPRgp7CiAgImV4ZWMtb3B0cyI6IFsibmF0aXZlLmNncm91cGRyaXZlcj1zeXN0ZW1kIl0sCiAgImxvZy1kcml2ZXIiOiAianNvbi1maWxlIiwKICAibG9nLW9wdHMiOiB7CiAgICAibWF4LXNpemUiOiAiMTAwbSIKICB9LAogICJzdG9yYWdlLWRyaXZlciI6ICJvdmVybGF5MiIKfQpFT0YKbWtkaXIgLXAgL2V0Yy9zeXN0ZW1kL3N5c3RlbS9kb2NrZXIuc2VydmljZS5kCnN5c3RlbWN0bCBkYWVtb24tcmVsb2FkCnN5c3RlbWN0bCByZXN0YXJ0IGRvY2tlcgpzd2Fwb2ZmIC1hCm12IC9ldGMvZnN0YWIgL2V0Yy9mc3RhYi5vbGQgJiYgZ3JlcCAtdiBzd2FwIC9ldGMvZnN0YWIub2xkID4gL2V0Yy9mc3RhYgpjYXQgPDxFT0YgfCB0ZWUgL2V0Yy9kZWZhdWx0L2t1YmVsZXQKS1VCRUxFVF9FWFRSQV9BUkdTPS0tY2xvdWQtcHJvdmlkZXI9ZXh0ZXJuYWwKRU9GCmt1YmVhZG0gam9pbiAtLWRpc2NvdmVyeS10b2tlbi11bnNhZmUtc2tpcC1jYS12ZXJpZmljYXRpb24gLS10b2tlbiB7ey5Cb290c3RyYXBUb2tlbklEfX0ue3suQm9vdHN0cmFwVG9rZW5TZWNyZXR9fSB7ey5BUElTZXJ2ZXJFbmRwb2ludH19Cg==
reservation=prefer
Expand All @@ -53,7 +53,7 @@ metadata:
name: bootstrap-token-cluster-autoscaler-packet
namespace: kube-system
stringData:
description: "The default bootstrap token used by cluster-autoscaler on Packet."
description: "The default bootstrap token used by cluster-autoscaler on Equinix Metal."

# Token ID and secret. Required if using bootstrap tokens in cloudinit (e.g. with kubeadm).
# token-id must match the regular expression [a-z0-9]{6}
Expand Down