You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In environments where we are running a lot of pods we see the external resizer going out of memory.
In these environments we typically only run 10's of pvc's, but 1000's of pods.
I believe the pod informer might be the culprit.
Doesn't starting an informer launch a cache? This means all pods are currently cached on the resizer, which leads to excessive memory use. The code isn't actually using the cache, I think the code is only interested in the Add, Update, Delete part and not in the cache.
If this is correct I am willing to create a PR to fix this
The text was updated successfully, but these errors were encountered:
stijndehaes
changed the title
Out of memory when there are a lot of pods
Out of memory when running a lot of pods
Jun 17, 2022
The whole handleInUse thing is behind a flag and only applicable for use cases where a driver needs external-resizer to enforce offline only expansion strongly. If it is enabled in EKS with external-resizer, then that seems like a problem. cc @wongma7
In environments where we are running a lot of pods we see the external resizer going out of memory.
In these environments we typically only run 10's of pvc's, but 1000's of pods.
I believe the pod informer might be the culprit.
Doesn't starting an informer launch a cache? This means all pods are currently cached on the resizer, which leads to excessive memory use. The code isn't actually using the cache, I think the code is only interested in the
Add
,Update
,Delete
part and not in the cache.If this is correct I am willing to create a PR to fix this
The text was updated successfully, but these errors were encountered: