add custom metric to hpa template #5704

oleksandrsemak · 2020-06-11T22:40:10Z

What this PR does / why we need it:

Add custom metrics support to HPA template
Increase resources request for reducing useless scaling HPA

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Which issue/s this PR fixes

Low resources request triggered HPA scaling even without any load

How Has This Been Tested?

Tested on K8s 1.16.10

Checklist:

My change requires a change to the documentation.
I have updated the documentation accordingly.
I've read the CONTRIBUTION guide
I have added tests to cover my changes.
All new and existing tests passed.

k8s-ci-robot · 2020-06-11T22:40:13Z

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.

If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
If you have done the above and are still having issues with the CLA being reported as unsigned, please log a ticket with the Linux Foundation Helpdesk: https://support.linuxfoundation.org/
Should you encounter any issues with the Linux Foundation Helpdesk, send a message to the backup e-mail support address at: [email protected]

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

k8s-ci-robot · 2020-06-11T22:40:18Z

Hi @alexandrsemak. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

oleksandrsemak · 2020-06-11T23:07:52Z

I signed it

charts/ingress-nginx/values.yaml

aledbf · 2020-06-12T00:42:08Z

charts/ingress-nginx/values.yaml

@@ -577,4 +588,4 @@ tcp: {}
 # Ref: https://github.com/kubernetes/contrib/tree/master/ingress/controllers/nginx/examples/udp
 ##
 udp: {}
-#  53: "kube-system/kube-dns:53"
+#  53: "kube-system/kube-dns:53"


please don't remove the newline

aledbf · 2020-06-12T00:42:19Z

/ok-to-test

ChiefAlexander · 2020-06-15T14:27:42Z

charts/ingress-nginx/values.yaml

@@ -226,10 +226,10 @@ controller:
  resources:
  #  limits:
  #    cpu: 100m
-  #    memory: 90Mi
+  #    memory: 200Mi


why are we changing these values?

@ChiefAlexander low memory value triggered hpa even w/o any load, so if you install chart with 90Mi memory it will scale hpa to maxreplicas

I see. As we don't default the HPA to enabled I'd rather we leave this alone and instead make a note of that in our Charts Readme. What do you think?

I don't mind. I will back this value

ChiefAlexander · 2020-06-16T14:23:33Z

/assign @ChiefAlexander

ChiefAlexander · 2020-06-16T14:37:30Z

This is looking good. Please squash your commits. https://github.com/kubernetes/community/blob/master/contributors/guide/pull-requests.md#6-squashing

ChiefAlexander · 2020-06-16T16:12:31Z

charts/ingress-nginx/Chart.yaml

@@ -1,6 +1,6 @@
 apiVersion: v1
 name: ingress-nginx
-version: 2.4.0
+version: 2.5.0


Suggested change

version: 2.5.0

version: 2.6.0

Another change got in before yours. Please bump the chart version and squash the commit. After that this lgtm

Sorry about this. Yet another change was merged in before yours again. Gonna need to bump the minor version and rebase your changes.

do you mean chart version 2.6.1 ?

So we should bump to 2.7.0

Kubernetes follows semver, k8s docs. So Major.Minor.Patch

gotcha, done

oleksandrsemak · 2020-06-17T14:46:21Z

/retest

oleksandrsemak · 2020-06-18T11:57:49Z

@ChiefAlexander please could you resolve conflict?

ChiefAlexander · 2020-06-18T14:29:56Z

Your PR needs a rebase against the changes in origin.

k8s-ci-robot · 2020-06-18T15:20:36Z

@alexandrsemak: The following tests failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
pull-ingress-nginx-e2e-1-18	a8258d9ad10878c12f1a97f32c94770766c840fe	link	`/test pull-ingress-nginx-e2e-1-18`
pull-ingress-nginx-e2e-1-15	a8258d9ad10878c12f1a97f32c94770766c840fe	link	`/test pull-ingress-nginx-e2e-1-15`
pull-ingress-nginx-e2e-1-17	a8258d9ad10878c12f1a97f32c94770766c840fe	link	`/test pull-ingress-nginx-e2e-1-17`
pull-ingress-nginx-lualint	a8258d9ad10878c12f1a97f32c94770766c840fe	link	`/test pull-ingress-nginx-lualint`
pull-ingress-nginx-test-lua	a8258d9ad10878c12f1a97f32c94770766c840fe	link	`/test pull-ingress-nginx-test-lua`
pull-ingress-nginx-e2e-1-16	a8258d9ad10878c12f1a97f32c94770766c840fe	link	`/test pull-ingress-nginx-e2e-1-16`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

oleksandrsemak · 2020-06-18T15:43:45Z

@ChiefAlexander done

ChiefAlexander · 2020-06-18T15:47:38Z

/lgtm
Thanks! Sorry we had to jump through so many hoops 😆

k8s-ci-robot · 2020-06-18T15:47:54Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alexandrsemak, ChiefAlexander

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~charts/ingress-nginx/OWNERS~~ [ChiefAlexander]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

oleksandrsemak · 2020-06-18T15:50:33Z

yeah no problem, this is much better than support own chart

sslavic · 2020-07-13T06:34:20Z

charts/ingress-nginx/values.yaml

+  # - type: Pods
+  #   pods:
+  #     metric:
+  #       name: nginx_ingress_controller_nginx_process_requests_total


@alexandrsemak nginx_ingress_controller_nginx_process_requests_total metric is a counter IMO making it not appropriate metrics for HPA; other than being "reset" when container is restarted and pod cycled, just its rate may get reduced when additional replicas get added.

Maybe nginx_ingress_controller_nginx_process_connections would be more appropriate. WDYT?

@sslavic sorry for confusing by this metric it was provided just for example the main goal was add ability to add any of custom metric. BTW kubernetes use timestamp for calculate averagevalue even with counter metric it will works correct, but you right we can change it to nginx_ingress_controller_nginx_process_connections or wrap nginx_ingress_controller_nginx_process_requests_total in rate

k8s-ci-robot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Jun 11, 2020

k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jun 11, 2020

k8s-ci-robot requested review from ChiefAlexander and ElvinEfendi June 11, 2020 22:40

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Jun 11, 2020

aledbf reviewed Jun 12, 2020

View reviewed changes

charts/ingress-nginx/values.yaml Outdated Show resolved Hide resolved

aledbf reviewed Jun 12, 2020

View reviewed changes

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jun 12, 2020

ChiefAlexander reviewed Jun 15, 2020

View reviewed changes

k8s-ci-robot assigned ChiefAlexander Jun 16, 2020

ChiefAlexander reviewed Jun 16, 2020

View reviewed changes

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 17, 2020

add custom metric to hpa template

baed5c6

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 18, 2020

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 18, 2020

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 18, 2020

k8s-ci-robot merged commit 6048e63 into kubernetes:master Jun 18, 2020

sslavic reviewed Jul 13, 2020

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add custom metric to hpa template #5704

add custom metric to hpa template #5704

oleksandrsemak commented Jun 11, 2020

k8s-ci-robot commented Jun 11, 2020

k8s-ci-robot commented Jun 11, 2020

oleksandrsemak commented Jun 11, 2020

aledbf Jun 12, 2020

aledbf commented Jun 12, 2020

ChiefAlexander Jun 15, 2020

oleksandrsemak Jun 15, 2020

ChiefAlexander Jun 16, 2020

oleksandrsemak Jun 16, 2020

ChiefAlexander commented Jun 16, 2020

ChiefAlexander commented Jun 16, 2020

ChiefAlexander Jun 16, 2020

oleksandrsemak Jun 16, 2020

ChiefAlexander Jun 17, 2020

oleksandrsemak Jun 17, 2020

ChiefAlexander Jun 17, 2020

oleksandrsemak Jun 17, 2020

oleksandrsemak commented Jun 17, 2020

oleksandrsemak commented Jun 18, 2020

ChiefAlexander commented Jun 18, 2020

k8s-ci-robot commented Jun 18, 2020 •

edited

Loading

oleksandrsemak commented Jun 18, 2020

ChiefAlexander commented Jun 18, 2020

k8s-ci-robot commented Jun 18, 2020

oleksandrsemak commented Jun 18, 2020

sslavic Jul 13, 2020

oleksandrsemak Jul 13, 2020

add custom metric to hpa template #5704

add custom metric to hpa template #5704

Conversation

oleksandrsemak commented Jun 11, 2020

What this PR does / why we need it:

Types of changes

Which issue/s this PR fixes

How Has This Been Tested?

Checklist:

k8s-ci-robot commented Jun 11, 2020

k8s-ci-robot commented Jun 11, 2020

oleksandrsemak commented Jun 11, 2020

Choose a reason for hiding this comment

aledbf commented Jun 12, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ChiefAlexander commented Jun 16, 2020

ChiefAlexander commented Jun 16, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oleksandrsemak commented Jun 17, 2020

oleksandrsemak commented Jun 18, 2020

ChiefAlexander commented Jun 18, 2020

k8s-ci-robot commented Jun 18, 2020 • edited Loading

oleksandrsemak commented Jun 18, 2020

ChiefAlexander commented Jun 18, 2020

k8s-ci-robot commented Jun 18, 2020

oleksandrsemak commented Jun 18, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

k8s-ci-robot commented Jun 18, 2020 •

edited

Loading