reflector going OOM caused by spiky behaviour

[aeimer]

Hi guys,

my reflector pod just exited hard due to an out-of-memory event.
There are also regular errors popping up in the log.

Instana event:

Instana pod details:

Logs:

These kind of logs repeat every few hours.

So one thing that came up, maybe the pod just needs more RAM than you specified here: https://github.com/emberstack/kubernetes-reflector/blob/master/src/helm/reflector/values.yaml#L58
But in general this works quite good, but I don't know where the spikes come from.
I have the second smaller cluster with reflector running, the RAM fills up there also, but not that fast and I couldn't see any OOMs so far.

Thank you for helping.

BR
Alex

[stale]

Automatically marked as stale due to no recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[aeimer]

@winromulus can you have a look into? What can I do about it?

[stale]

Removed stale label.

[stale]

Automatically marked as stale due to no recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[aeimer]

/nostale

…

On Sat, 19 Jun 2021, 01:48 stale[bot], ***@***.***> wrote: Automatically marked as stale due to no recent activity. It will be closed if no further activity occurs. Thank you for your contributions. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#187 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF7ADLUG2NMLBJTCGA6NSQLTTPLNNANCNFSM454WQWOQ> .

[stale]

Removed stale label.

[scarby]

@aeimer Are you using the reflector to reflect certificates from LetsEncrypt?

[aeimer]

@scarby Yes, I actually do. Maybe #191 is the cause for this issue. Good point.

[stale]

Automatically marked as stale due to no recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[aeimer]

@klimisa is there any chance that you can have a look into this issue?

[stale]

Removed stale label.

[stale]

Automatically marked as stale due to no recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[brokenjacobs]

Is the issue just the memory limit? Or is there some larger issue at play here causing a problem?

[stale]

Removed stale label.

[aeimer]

@brokenjacobs AFAIK everyhting seems to work. We have cert-manager with LE running.

Edit: @scarby suspected a link to #191

[stale]

Automatically marked as stale due to no recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[aeimer]

no stale. Problem seems to persist

[stale]

Removed stale label.

[stale]

Automatically marked as stale due to no recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

Automatically closed stale item.