Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit keyword not release memory after delete oldest metrics. #763

Open
zhengtianbao opened this issue Sep 4, 2023 · 2 comments
Open

Limit keyword not release memory after delete oldest metrics. #763

zhengtianbao opened this issue Sep 4, 2023 · 2 comments
Labels
bug This is considered a bug and shall get fixed mtail-Language/VM Issues related to the mtail language, compiler, or VM

Comments

@zhengtianbao
Copy link
Contributor

Hi @jaqx0r, I set the Limit keyword so mtail will remove the oldest metrics, but I noticed the memory usage of mtail process will increase and not decrease after GC.

https://github.com/google/mtail/blob/e35f4c5d7b53b9947336850b4e11d10a78428dfe/internal/metrics/metric.go#L215C20-L215C20

m.labelValuesMap type is map[string]*LabelValue, the delete will not shrink the map size. see golang/go#20135

If someone uses the Limit keyword, it means that the number of metrics he wants to limit may be very large. Do we need to consider the memory consumption of this part?

I changed the m.labelValuesMap type to map[[256]byte]*LabelValue, because golang will allocates data more than 128 bytes from heap and save pointers in the bucket of map, and after GC pointers at map will be replaced by nil pointer, so the memory pointed will be recollected. This solution may have some unconsidered issues, just for reference.

@zhengtianbao

This comment was marked as off-topic.

@jaqx0r
Copy link
Contributor

jaqx0r commented Apr 21, 2024

The second comment is probably a bug, can you open a new issue about it?

Thanks for the tip about the first issue.

@jaqx0r jaqx0r added bug This is considered a bug and shall get fixed mtail-Language/VM Issues related to the mtail language, compiler, or VM labels Apr 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This is considered a bug and shall get fixed mtail-Language/VM Issues related to the mtail language, compiler, or VM
Projects
None yet
Development

No branches or pull requests

2 participants