Support enabling mimalloc and turn it on by default #54
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This was supported in https://github.com/kubernetes/ingress-nginx/releases/tag/nginx-0.26.0 and recently in upstream chart too. We should turn it on to reduce memory utilization - expected at least 10% reduction. Memory utilization increased with recent tuning of max-worker-connections #50 optimization for high performance scenarios.
Even without max-worker-connections switched to 0, some tenant clusters where HPA trials were run had large amount of Ingress resource definitions, resulting in high memory utilization especially during nginx configuration reloads. To better support these use cases out of the box, together with enabling HPA default memory requests for most non-trivial clusters were increased to 2.5GB. Hopefully with applying mimalloc enabled we can lower the utilization. Smaller nginx memory footprint is good for both cost efficiency, faster scheduling and better elasticity - in production use cases like flash sale spikes nginx can scale out / add same capacity faster while creating less pressure on worker nodes scale; with larger nginx it is more likely that additional worker nodes have to be added to place new nginx replicas, and it takes time for cluster-autoscaler to kick in and nodes to be provisioned.
In addition to making mimalloc optional and enabled by default, this PR adds support for configuring additional environment variables for the nginx controller container, like in upstream chart. This among other things enables configuring additional mimalloc options if we find needed.