Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: increase default resource cpu & memory limits #312

Merged
merged 1 commit into from
Mar 9, 2023

Conversation

Rakshith-R
Copy link
Member

We now have 100 workers by default and watch a lot of resources such as PVCs, VolumeAttachments and owned CRs. This leads to Out-Of-Memory kills(OOMKILLs) with just limit of 128Mi.
Therefore, raising limit to 512Mi memory and 1000m cpu./

Note: I am noticing increase of 50Mi usage with 1000 PVC with reclaimspace annotation. This new limit should give us headroom, while the requested resource is still the same.

@nixpanic
Copy link
Collaborator

nixpanic commented Mar 8, 2023

512Mi sounds rather a lot? Could you share how many PVCs you used for testing?

I wonder if this should be documented, with some results for reference.

Verified

This commit was signed with the committer’s verified signature. The key has expired.
addaleax Anna Henningsen
We now have 100 workers by default and watch a lot of
resources such as PVCs,  VolumeAttachments and owned CRs.
This leads to Out-Of-Memory kills(OOMKILLs) with just
limit of 128Mi.
Therefore, raising limit to 512Mi memory and 1000m cpu.

Signed-off-by: Rakshith R <[email protected]>
@Rakshith-R Rakshith-R force-pushed the increase-resources branch from 64ff680 to 7f23457 Compare March 9, 2023 07:57
@Rakshith-R
Copy link
Member Author

512Mi sounds rather a lot? Could you share how many PVCs you used for testing?

I've used about 3000 PVCs with reclaimspace annotation(not mounted) and observed increase of about 50Mi per 1000 PVC
approximately.
Considering the usage depends on PVCs, Pods(volumeattachements), Nodes, and CSI-Addons CRs, we will easily breach the limit of 128Mi on a production cluster.

I wonder if this should be documented, with some results for reference.

I've added a section for this.
I don't think we can definitely add results without extensive testing.
The resource request is untouched, while only increasing the limit on the resource, therefore we should be fine.
The user can tweak it based on the cluster requirements and parameters.

Thanks!

@nixpanic nixpanic added documentation Improvements or additions to documentation enhancement New feature or request labels Mar 9, 2023
@mergify mergify bot merged commit 666074b into csi-addons:main Mar 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants