Add support for tags on AWS managed nodegroups to indicate resources

[tbalzer]

What type of PR is this?

/kind feature

What this PR does / why we need it:

This PR introduces a feature that allows handling of ephemeral storage and custom resources in a scale-from-zero scenario when using EKS nodegroups.

It does this via allowing tags in the form of k8s.io/cluster-autoscaler/resources/<resource>: <quantity> analogous to how they are allowed directly on an ASG.
As the EKS-managed ASGs in case of managed nodegroups don't support custom tags for those resources they will be read from the managed nodegroup and processed accordingly.

Which issue(s) this PR fixes:

NONE

Special notes for your reviewer:

I have modeled this after the implementation for ASGs and tried to be as clean as possible, passing through the Tags in the managed node group cache. I don't expect anybody will find it unexpected that these specific annotations influence the node templates.

As for documentation regarding this I didn't immediately find a place where one could mention that the ASG annotations also work for managed nodegroups, however I initially just assumed they would until I looked at the code.

Does this PR introduce a user-facing change?

Allow k8s.io/cluster-autoscaler/resources/<resource>:<quantity> tags on AWS managed nodegroups (in addition to ASGs) to indicate availability of custom resources or ephemeral storage in a scale-from-zero scenario

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

NONE

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: tbalzer / name: Tim Balzer (fb32b84)

[k8s-ci-robot]

Welcome @tbalzer!

It looks like this is your first PR to kubernetes/autoscaler 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/autoscaler has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[tbalzer]

Hi @jaypipes @drmorr0,

Any chance to get a review on this? We're willing to contribute more time into this as needed, as this missing feature prevented us from using scale-from-zero when having pending pods with ephemeral-storage requests on AWS with managed nodegroups and we think that's a perfectly valid usecase that's missing after the support for Labels/Taints being extracted from managed nodegroups in a scale-from-zero scenario.

Currently we're running our clusters with our self-hosted cluster-autoscaler image that has this included, but would obviously prefer to be able to use upstream again. So far I can say it's worked flawlessly.

[jaypipes]

@tbalzer thanks for this :) Left a couple questions/suggestions inline for you but I think the code makes sense.

[tbalzer]

@jaypipes : Thanks for the feedback and attention. I've updated the code according to your suggestions and replaced the commit. PTAL again.

[jaypipes]

Yes, I'm happy with this. Thank you @tbalzer for your work! :)

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jaypipes, tbalzer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• cluster-autoscaler/cloudprovider/aws/OWNERS [jaypipes]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[tbalzer]

@jaypipes : Can you also add an lgtm here or should I ping someone else for this?

[jaypipes]

@jaypipes : Can you also add an lgtm here or should I ping someone else for this?

@tbalzer I'd prefer another approver have a quick look. @gjtempleton can you have a quick looksie? Thanks!

[gjtempleton]

/assign @gjtempleton

[tbalzer]

@gjtempleton : Any chance to get a review? This is getting more and more stale.

[gjtempleton]

@tbalzer apologies, I'll give this a review this week.

[gjtempleton]

Thanks for this.

/lgtm