Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#573 optimize filtering algorithm for DropwizardExports #601

Closed
wants to merge 2 commits into from
Closed

#573 optimize filtering algorithm for DropwizardExports #601

wants to merge 2 commits into from

Conversation

vladimir-bukhtoyarov
Copy link

The intention of this optimization has been described there #573 (comment)

@brian-brazil
Copy link
Contributor

Do you have benchmarks to demonstrate how useful this is?

Also please use explicit imports, it's hard to follow what is what otherwise.

Signed-off-by: vladimir.bukhtoyarov <[email protected]>
@vladimir-bukhtoyarov
Copy link
Author

vladimir-bukhtoyarov commented Nov 16, 2020

I can not imagine how to write the benchmark here. The penalty of filtering depends on two factors:

  • Complexity of filter itself.
  • Amount of metrics in the registry.

This pull request allows applying filter only one for each metric, filter does not need to be applied on each reporting cycle. For example, if we have 3 regular expressions in the filter, and 5000 metrics in the registry, then 15000 regexps are not executed on each reporting cycle, because pre-filtering of registry makes this work unnecessary. A huge metric-registry with thousands of metrics is a typical case for systems like Cassandra or Kafka, and in the same time usually we need to implement complex filtering for this kind of applications because we do not want to report so huge dataset to the monitoring database.

@brian-brazil
Copy link
Contributor

It'd be good to have some rough idea, for example if it turns out that other processing is substantially larger then there's not much point in having this from a complexity standpoint.

I also presume it's safe to assume that filters don't vary over time?

@vladimir-bukhtoyarov
Copy link
Author

Looks I can just continue to wrap the registry by decorator before passing to the collector.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants