Topology now factors in affinity and global requirements (subnets) when computing spread

[ellistarn]

1. Issue, if available:

2. Description of changes:

1. Topology and Affinity no longer assume a subnet exists for all zones. GetInstanceTypes now returns the set of instance type offerings limited by subnet AZ availability rather than assuming. These offerings are then used to compute WellKnownLabel values (including zones). This is roughly the same behavior, except it happens once per provisioning loop (+caching) rather than once at process startup.
2. Support for https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/#interaction-with-node-affinity-and-node-selectors
3. WellKnownLabel values are not longer validated at provisioner creation time. This is because valid zones can change over time and based on subnet creation ordering w.r.t provision creation. Rather than trying to be perfect at creation time, we enforce the correctness at runtime.
4. Cleaned up some scheduling code since I was already making significant changes
5. Cleaned up the allocator code since I was already making changes there
6. Cleaned up some metrics code

3. Does this change impact docs?

• Yes, PR includes docs updates
• Yes, issue opened: link to issue
• No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[netlify]

✔️ Deploy Preview for karpenter-docs-prod canceled.

🔨 Explore the source changes: ac5daa3

🔍 Inspect the deploy log: https://app.netlify.com/sites/karpenter-docs-prod/deploys/617c2710a881010007d19839

[bwagner5]

I did a quick pass through. Nice refactor and bug fix :)

[bwagner5]

remove 2.