Export Jaeger storage metrics as OTEL metrics

[pavolloffay]

Jaeger storage implementations export various metrics that should be exported as Jaeger OTEL collector metrics.

[pavolloffay]

OTEL collector uses internally OpenCensus metrics.

@yurishkuro do you know if there is OC implementation of jaeger-lib metrics interface?

[yurishkuro]

I've never heard of one. I think long(er) term we'd want to use OTel API directly.

[pavolloffay]

We should make a decision how to export our storage metrics in OTEL collector. We can forward them to OC API to be consistent and use OTEL once they switch to it.

[pavolloffay]

I have created an issue in OTEL to migrate from OC to OTEL API open-telemetry/opentelemetry-collector#816.

[pavolloffay]

The adapter for OTEL metrics is an easy part. The problem is the Prometheus registry in OTEL is not exported https://github.com/open-telemetry/opentelemetry-collector/blob/master/service/telemetry.go#L140

I am not sure whether exporting it makes sense since OTEL will migrate the internal observability to OTEL which supports multiple backends open-telemetry/opentelemetry-collector#816.

In the meantime, we could export the metrics on a different port e.g. 14269 and implement the OTEL metrics adapter in jaeger-lib. Or we could migrate all codebase to OTEL metrics in one go so the adapter won't be needed.

@yurishkuro any thoughts?

[yurishkuro]

Is the OTel API for metrics ready for use? I see a lot of back and forth on some tickets. Until OTel Collector converts to OTel API metrics, I don't think we should try to do it either. Exporting two endpoints for metrics is not ideal, but possible as a temporary solution (just in jaeger-otel-collector).

[pavolloffay]

Good points, I am not sure how stable it is. I have noticed they just recently added p8s exporter.

Then as an interim solution I will export metrics in two endpoints - one for OTEL metrics and one for Jaeger metrics.

[pavolloffay]

Closing as it is a "low level" issue related to V2 which might take a different approach.