add arch, os, and provisioner anti-affinities for the karpenter deployment #468
Conversation
| nodeAffinity: | ||
| requiredDuringSchedulingIgnoredDuringExecution: | ||
| nodeSelectorTerms: | ||
| - matchExpressions: |
There was a problem hiding this comment.
I don't actually think we want this. Imagine if someone wants to run karpenter in a management cluster, who's nodes are themselves managed by a separate karpenter installation.
There was a problem hiding this comment.
Interesting, I hadn't considered the case of a management cluster using karpenter... But a karpenter installation should never be hosted on a node provisioned by the installation, right? I can remove this one since it's not super important for people trying it out right now, only slightly annoying when developing.
There was a problem hiding this comment.
I went ahead and removed this one.
| - key: kubernetes.io/arch | ||
| operator: In | ||
| values: | ||
| - amd64 |
There was a problem hiding this comment.
Don't we currently build arm images w/ ko?
There was a problem hiding this comment.
The makefile has the flag to generate for all platforms, but I only see the amd64 one in the public registry.
There was a problem hiding this comment.
It might be that ecr public just doesn't show the other arch ones ... the manifest appears to have the hashes listed, let me do some digging real quick.
There was a problem hiding this comment.
I tested on an arm64 instance and it does appear to pull down an arm64 variant, so I removed the arch affinity.
| - key: kubernetes.io/os | ||
| operator: In | ||
| values: | ||
| - linux |
Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Issue #, if available:
N/A
Description of changes:
There are a few ways that karpenter pods can get scheduled on bad compute nodes:
This PR adds a default anti-affinity for the karpenter provisioner name label (
provisioner.karpenter.sh/name) and affinity to linux and amd64 nodes.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.