Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reuse NewResourceMetadataEnricher to reduce memory utilisation #35591

Closed
ChrsMark opened this issue May 26, 2023 · 1 comment
Closed

Reuse NewResourceMetadataEnricher to reduce memory utilisation #35591

ChrsMark opened this issue May 26, 2023 · 1 comment
Labels
Team:Cloudnative-Monitoring Label for the Cloud Native Monitoring team

Comments

@ChrsMark
Copy link
Member

ChrsMark commented May 26, 2023

In the kubernetes module for every metricset enabled we are creating a different MetadataEnricher by calling the NewResourceMetadataEnricher method:

func NewResourceMetadataEnricher(

This means that many k8s resources like Nodes, Namespaces are being retrieved multiple times through Watchers.
Same for Pods that are being retrieved from pod, container, state_pod and state_container metricsets.

This has apparently impact on the memory utilization of Metricbeat and can lead in Metricbeat Pod requiring a lot of memory resources to avoid being OOMKilled. We can improve this by establishing a common "pool" of Enrichers by type for the whole module and then each metricset can re-use the proper Enricher accordingly.

This also affects Elastic Agent since it runs Metricbeat under the hood.

cc: @gizas @tommyers-elastic @bturquet

@MichaelKatsoulis
Copy link
Contributor

Duplicate of #37243

@MichaelKatsoulis MichaelKatsoulis marked this as a duplicate of #37243 Jan 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Cloudnative-Monitoring Label for the Cloud Native Monitoring team
Projects
None yet
Development

No branches or pull requests

2 participants