-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bumped updateCh limit to fix #2022 #2081
Conversation
For larger clusters, updateCh with 1024 is not enough and prevents nginx from starting. Ideally this will be auto sized or adjusted to start popping items off during original startup.
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: azweb76 Assign the PR to them by writing The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these OWNERS Files:
Approvers can indicate their approval by writing |
@azweb76 can you share the number of services and endpoints in your cluster? |
496 services, 499 endpoints |
486 ingress objects |
This is likely a ticking time bomb for those that are about to exceed the 1024 limit. Once their object count exceeds that limit, the pod will fail to start. This will at least extend the time to provide a long term fix. |
@azweb76 I am fixing this issue right now using a ring buffer to never block writes. I will open the PR and a temporal image in a couple of minites |
sweet! I can help test if you'd like. |
@azweb76 please use |
That appears to have fixed it. |
What this PR does / why we need it:
For larger clusters, updateCh with 1024 is not enough and prevents nginx from starting. I bumped updateCh limit to 99999. Ideally this will be auto sized or adjusted to start popping items off during original startup.
Which issue this PR fixes (optional, in fixes #(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #2022
Special notes for your reviewer:
Ideally a longer term solution should exist to support even larger clusters.