feat: support HPA in helm chart

[Dean-Coakley]

What type of PR is this?

• Adds support for HorizontalPodAutoscaler support to helm chart

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #4929

Release Notes: Yes

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 66.86%. Comparing base (0c68ae7) to head (e958829).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5127      +/-   ##
==========================================
- Coverage   66.87%   66.86%   -0.02%     
==========================================
  Files         210      210              
  Lines       32977    32977              
==========================================
- Hits        22054    22050       -4     
- Misses       9588     9590       +2     
- Partials     1335     1337       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[arkodg]

thanks @Dean-Coakley this looks good ! can you add a test in https://github.com/envoyproxy/gateway/tree/main/test/helm

[guydc]

Prefer to keep this entirely in values (similar to behavior) and not force end users to rely only on CPU/Mem. For example, custom metrics could be relevant here, e.g. size of the XDS cache, number of connected envoy proxies, etc.

[Dean-Coakley]

Ok, I will make this change.

Interesting, I was implementing this as an array initially: #4929 (comment)

But after looking around, I saw this different approach seemed more common so I thought I must be wrong:

• https://github.com/nginx/kubernetes-ingress/blob/main/charts/nginx-ingress/templates/controller-hpa.yaml#L24
• https://github.com/bitnami/charts/blob/main/bitnami/apache/templates/hpa.yaml#L23
• https://github.com/criblio/helm-charts/blob/master/helm-chart-sources/logstream-workergroup/templates/hpa.yaml#L28

But moving back to [] makes sense to me. And I have seen one example of doing similar:

• https://github.com/traefik/traefik-helm-chart/blob/master/traefik/templates/hpa.yaml#L28

If you get reports of usability issues you could follow a similar approach to traefik. Seems nice: https://github.com/traefik/traefik-helm-chart/blob/master/EXAMPLES.md#install-with-auto-scaling