Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support Dogstatsd 1.1 protocol for distributions #216

Conversation

blemale
Copy link
Contributor

@blemale blemale commented Feb 24, 2023

There is a new experimental Dogstatsd protocol (1.1) which supports sending multiple values in a single datagram.

This is supported since agent 7.25.0/6.25.0.

This evolution in the protocol allows clients to buffer histogram and distribution values and send them in fewer payload to the agent (providing a behavior close to client-side aggregation for those types).

This commit add support for this new protocol for distribution.

Fix #215

There is a new experimental Dogstatsd protocol (1.1) which supports sending
multiple values in a single datagram.

This is supported since agent 7.25.0/6.25.0.

This evolution in the protocol allows clients to buffer histogram and
distribution values and send them in fewer payload to the agent (providing a
behavior close to client-side aggregation for those types).

This commit add support for this new protocol for distribution.

Fix DataDog#215
@blemale
Copy link
Contributor Author

blemale commented Feb 24, 2023

@vickenty do you have an idea why CircleCI is complaining as I have not touch anything related to it?

CircleCI Pipeline — Could not find a usable config.yml, you may have revoked the CircleCI OAuth app.

@vickenty
Copy link
Contributor

Hi @blemale , thank you for opening the PR, but I don't think this is the right way to do this: multi-valued payloads are not a feature that we expose directly to the users, but rather than an optimization that can be toggled on and off (via "client side aggregation").

For example, please see how datadog-go implements this: clients provide one sample at a time, which are then aggregated internally and sent as a multi-valued message. Serialization code should also take care to split values over several messages to avoid too big payloads.

@blemale
Copy link
Contributor Author

blemale commented Feb 27, 2023

👋 Hi @vickenty ! Thanks for the insights :)

Sadly, on the code base i'm working on, i.e DD logs-backend, we are not using the aggregation layer, i.e .enableAggregation(false), as we use dropwizard-metrics that already does the aggregation and only reports point to dogstatsd through this library every 10 or 20s.

However every time we report a distribution we report 1 000 values, currently using recordDistributionValue, which create burst of points on the dogstatsd agent and so this proposal to let client code submit several points in one datagram.

I understand that this proposal goes a bit against the design of this library but dropwizard-metrics has been the go to library for metrics java for a long time and it already does the aggregation. So having a way to integrate efficiently dropwizard-metrics and java-dogstatsd-client would be really appreciated.

@blemale blemale closed this Feb 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support Dogstatsd protocol 1.1
2 participants