Add Exemplar support to Metrics proto #159

cnnradams · 2020-06-22T17:49:06Z

Adds support for OTEP#113. This should also handle duplicate labels in exemplars by bringing labels up a level and making exemplars only hold additional_labels.

Proto question (since I'm new to this): I defined a measurement_type enum for the RawValue type, but couldn't create a new enum with just INT64 and DOUBLE (because I can't share names with other enums). So right now I'm using Type, which means things other than INT64 and DOUBLE can be picked for measurement_type. What is the right solution for this?

cnnradams · 2020-07-09T18:07:38Z

Closing in favor of #162 (which implements exemplars)

jmacd · 2020-07-18T08:18:10Z

I reopened this and put it in draft state. This has been referred to heavily in #168, and after #168 merges this PR should be restored to roughly its original state.

cnnradams · 2020-07-20T19:09:59Z

updated to not make structual changes and to add raw_value_data_points. Still need to specify the type of RawValue in some way, which will be unblocked after #168

bogdandrutu · 2020-08-05T17:29:27Z

opentelemetry/proto/metrics/v1/metrics.proto

@@ -104,6 +133,7 @@ message Metric {
  repeated DoubleDataPoint double_data_points = 3;
  repeated HistogramDataPoint histogram_data_points = 4;
  repeated SummaryDataPoint summary_data_points = 5;
+  repeated RawValue raw_data_points = 6;


I think this should be removed for the moment, this is not exemplar, is supporting "RawMeasurements" which is out of scope for this PR.

opentelemetry/proto/metrics/v1/metrics.proto

bogdandrutu · 2020-08-05T17:32:16Z

The generated code was removed, please remove that as well. Rebase the PR please.

opentelemetry/proto/metrics/v1/metrics.proto

bogdandrutu · 2020-08-05T22:29:26Z

opentelemetry/proto/metrics/v1/metrics.proto

+message RawValue {
+  // The set of labels that were dropped by the aggregator, but recorded
+  // alongside the original measurement. Only labels that were dropped by the aggregator should be included
+  repeated opentelemetry.proto.common.v1.StringKeyValue labels = 1;


What is the relationship between these labels and the labels in the DataPoint:

Do we duplicate them?

Do we extract these labels and the actual set of labels is the combination of these + datapoint.lables?

For exemplars these labels are only the labels not included in the DataPoint's labels. I would change this field to be dropped_labels but if RawValue is used as a data point itself the labels would include all labels

@jmacd I know you tried to share the messages, can you help define the behavior here?

I think you recommended calling these "dropped_labels". This sounds good to me.

opentelemetry/proto/metrics/v1/metrics.proto

lubingfeng · 2020-08-06T12:18:09Z

@jmacd When can we get #159 and #171 closed, which are blockers for getting stats proposal finalized, based on discussion two weeks ago.

bogdandrutu · 2020-08-06T16:45:45Z

@lubingfeng

@jmacd When can we get #159 and #171 closed, which are blockers for getting stats proposal finalized, based on discussion two weeks ago.

Not sure I understand:

159 is this PR
171 is merged

opentelemetry/proto/metrics/v1/metrics.proto

aabmass · 2020-08-06T19:15:20Z

opentelemetry/proto/metrics/v1/metrics.proto

+
+  // (Optional) List of exemplars collected from 
+  // measurements that were used to form the data point
+  repeated RawValue exemplars = 7;


Would this just be a list of random samples from the whole window? Open question in the OTEP:

We don’t have a strong grasp on how the sketch aggregator works in terms of implementation - so we don’t have enough information to design how exemplars should work properly.

The proto does not define how the exemplars were sampled, not sure your question?

Nvm, I see now

At the recent OTLP discussion meeting, we agreed to remove the sample_count field from the current proposal. We also agreed to move the Exemplars into the DataPoints so that they can refer to dropped labels, not include full label sets in each point.

@bogdandrutu does this sound right to you, for now?

We also agreed to move the Exemplars into the DataPoints so that they can refer to dropped labels, not include full label sets in each point.

I think we agreed that you will evaluate what is better for performance/semantics:

Having a repeated RawValue exemplars in the Metric that applies to all data points (user may need to do another remapping to every data point) vs repeated RawValue exemplars in every point (if we go with every point then "dropped_labels" is better name for that).

My point is that I don't have a strong opinion between the both, and was trying to make you investigate and decide which way. Here are my thoughts:

Having repeated RawValue exemplars in the Metrics:

Pros:

Saves some memory in the internal representation (have extra 24 bytes per data point).

Same message may be able to be re-used with raw-measurements because labels don't represent dropped.

Cons:

Duplicate labels on the wire.

User needs to re-map every exemplar to the data point by doing the labels matching.

I feel cons are more "significant" than pros, so personally I would go with exemplars in every DataPoint as you suggested.

If we go with exemplars in every DataPoint I would say to rename the message to "Exemplar" :)

This discussion makes me want a way to intern label sets to avoid the "re-map every exemplar" problem.

I don't feel inclined to invest time in this now, so we should probably choose "repeated RawValue exemplars in every point".

This discussion makes me want a way to intern label sets to avoid the "re-map every exemplar" problem.

Even with an "intern label" you still need to map every exemplar to a point (that mapping may be faster if we have "intern label" but still needs some work)

jmacd · 2020-08-08T03:56:18Z

I left a lengthy remark on this topic here:
open-telemetry/opentelemetry-specification#617 (comment)

I am worried that my request for @cnnradams to explore and implement statistical sampling for exemplars has led to some confusion, and (with my apologies) I am willing to omit it, but as noted in the comment, there are many related questions and even if we take the statistical question out of it, we're left with tough questions.

jmacd · 2020-08-12T21:39:09Z

@cnnradams I know your internship is over, but let us know if you're willing to make these changes.

cnnradams · 2020-08-12T21:51:48Z

sure. from reading the discussion, it seems like the only things I need to change are RawValue -> Exemplar, labels -> dropped_labels, and remove sample_count? Since there is already an exemplar set per datapoint?

cnnradams · 2020-08-13T01:44:54Z

sure. from reading the discussion, it seems like the only things I need to change are RawValue -> Exemplar, labels -> dropped_labels, and remove sample_count? Since there is already an exemplar set per datapoint?

done all 3.

MrAlias

🚀

* Add exemplars to proto * handle just exemplars, nit fixes * comments * rawvalue -> exemplar, remove sample_count

* Add exemplars to proto * handle just exemplars, nit fixes * comments * rawvalue -> exemplar, remove sample_count Co-authored-by: Connor Adams <[email protected]>

cnnradams requested review from arminru, bogdandrutu, c24t, carlosalberto, iredelmeier, jmacd, reyang, SergeyKanzhelev, tedsuo, tigrannajaryan and yurishkuro as code owners June 22, 2020 17:49

jmacd mentioned this pull request Jun 22, 2020

Integrate Exemplars with Metrics SDK open-telemetry/oteps#113

Merged

jmacd mentioned this pull request Jul 1, 2020

OTLP Metrics update #162

Closed

cnnradams closed this Jul 9, 2020

jmacd pushed a commit to jmacd/opentelemetry-proto that referenced this pull request Jul 15, 2020

One more open-telemetry#159 TODO

39e7967

jmacd pushed a commit to jmacd/opentelemetry-proto that referenced this pull request Jul 15, 2020

One more open-telemetry#159 TODO

5e7fa7f

This was referenced Jul 15, 2020

Replace Temporality with Kind; Type with ValueType #168

Closed

Add the requirement for the probability sampler open-telemetry/opentelemetry-specification#570

Closed

jmacd reopened this Jul 18, 2020

jmacd marked this pull request as draft July 18, 2020 08:17

jmacd added the spec:metrics label Jul 18, 2020

jmacd mentioned this pull request Jul 20, 2020

Merge Int64DataPoint and DoubleDataPoint into ScalarDataPoint #172

Closed

cnnradams force-pushed the exemplars branch from 04e83de to abb01f6 Compare July 20, 2020 19:08

jmacd mentioned this pull request Jul 21, 2020

Metrics terminology: Grouping instruments as (opposed to Adding instruments) open-telemetry/opentelemetry-specification#625

Closed

bogdandrutu reviewed Aug 5, 2020

View reviewed changes

cnnradams force-pushed the exemplars branch from abb01f6 to 15e3576 Compare August 5, 2020 19:17

cnnradams marked this pull request as ready for review August 5, 2020 19:18

cnnradams requested review from a team August 5, 2020 19:18

bogdandrutu reviewed Aug 5, 2020

View reviewed changes

lubingfeng mentioned this pull request Aug 6, 2020

Replace percentile with quantile #171

Merged

cnnradams added 3 commits August 6, 2020 11:08

Add exemplars to proto

3bc17ef

handle just exemplars, nit fixes

746f137

comments

a85ff22

cnnradams force-pushed the exemplars branch from 15e3576 to a85ff22 Compare August 6, 2020 15:09

aabmass reviewed Aug 6, 2020

View reviewed changes

rawvalue -> exemplar, remove sample_count

e7fe7a2

bogdandrutu approved these changes Aug 13, 2020

View reviewed changes

jmacd approved these changes Aug 13, 2020

View reviewed changes

MrAlias approved these changes Aug 13, 2020

View reviewed changes

Merge branch 'master' into exemplars

eced061

bogdandrutu merged commit 7b33f20 into open-telemetry:master Aug 13, 2020

bogdandrutu pushed a commit to bogdandrutu/opentelemetry-proto that referenced this pull request Aug 13, 2020

Add Exemplar support to Metrics proto (open-telemetry#159)

8ff66ad

* Add exemplars to proto * handle just exemplars, nit fixes * comments * rawvalue -> exemplar, remove sample_count

bogdandrutu pushed a commit to bogdandrutu/opentelemetry-proto that referenced this pull request Aug 13, 2020

Add Exemplar support to Metrics proto (open-telemetry#159)

c5341db

* Add exemplars to proto * handle just exemplars, nit fixes * comments * rawvalue -> exemplar, remove sample_count

jmacd mentioned this pull request Aug 14, 2020

Add statsdreceiver skeleton open-telemetry/opentelemetry-collector-contrib#566

Merged

jmacd mentioned this pull request Sep 11, 2020

Adopting DDSketch as the default ValueRecorder aggregation open-telemetry/opentelemetry-specification#919

Closed

jmacd mentioned this pull request Oct 27, 2020

Document Metrics Processor APIs and SDK requirements open-telemetry/opentelemetry-specification#1116

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Exemplar support to Metrics proto #159

Add Exemplar support to Metrics proto #159

cnnradams commented Jun 22, 2020

cnnradams commented Jul 9, 2020

jmacd commented Jul 18, 2020

cnnradams commented Jul 20, 2020

bogdandrutu Aug 5, 2020

cnnradams Aug 5, 2020

bogdandrutu commented Aug 5, 2020

bogdandrutu Aug 5, 2020

cnnradams Aug 6, 2020

bogdandrutu Aug 6, 2020

jmacd Aug 12, 2020

lubingfeng commented Aug 6, 2020

bogdandrutu commented Aug 6, 2020

aabmass Aug 6, 2020

cnnradams Aug 7, 2020

aabmass Aug 7, 2020

jmacd Aug 12, 2020

jmacd Aug 12, 2020

bogdandrutu Aug 12, 2020

bogdandrutu Aug 12, 2020

jmacd Aug 12, 2020

bogdandrutu Aug 12, 2020

jmacd commented Aug 8, 2020

jmacd commented Aug 12, 2020

cnnradams commented Aug 12, 2020

cnnradams commented Aug 13, 2020

MrAlias left a comment

Add Exemplar support to Metrics proto #159

Add Exemplar support to Metrics proto #159

Conversation

cnnradams commented Jun 22, 2020

cnnradams commented Jul 9, 2020

jmacd commented Jul 18, 2020

cnnradams commented Jul 20, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bogdandrutu commented Aug 5, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lubingfeng commented Aug 6, 2020

bogdandrutu commented Aug 6, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jmacd commented Aug 8, 2020

jmacd commented Aug 12, 2020

cnnradams commented Aug 12, 2020

cnnradams commented Aug 13, 2020

MrAlias left a comment

Choose a reason for hiding this comment