Complete headers override for OpenTelemetry

[artkonr]

Description

As a user I may occasionally want to use quarkus.opentelemetry.tracer.exporter.otlp.headers to not only add headers, but also to override the existing ones set by the upstream.

For example, this will have no effect:

quarkus.opentelemetry.tracer.exporter.otlp.headers=Content-Type=application/x-protobuf

because the Content-Type header is (probably) set in the downstream by the lower-level GRPC handlers.

It would be great to have this property have the "last word" regarding which headers are sent and how.

Implementation ideas

Idk if it's possible, but I guess there could be a way to wire the headers on the final stages of formulating the request.

[quarkus-bot]

/cc @brunobat, @radcortez

[geoand]

Although I understand the reasoning for this, I personally think this it is conceptually wrong to have OpenTelemetry have the last say in what HTTP headers are going to be sent, so I am -1 on this.

[radcortez]

These headers are not related to gRPC user code, but with the OTLP, the OpenTelemetry Protocol used to export span data: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/exporter.md.

@artkonr Are you sure this is what you are looking for?

[artkonr]

@geoand @radcortez I guess you're right;

probably, best if I describe my scenario: I have a presumably OT-compliant ingestion service, but it looks like it does not support content type application/grpc, no idea why. I wanted to try and set the content type it supports, but I'm not sure if there's a way to wire it in.

maybe you could point me to an example, which does that? if not, will quarkus respect OTEL_EXPORTER_OTLP_PROTOCOL if set to, say, http/json?

[radcortez]

Right now, we only support gRPC. We have an issue to add http support as well, but we have been focusing on other things: #21535.

I believe that changing the content-type is not going to help, because we expressly set the exporter to the gRPC one:

quarkus/extensions/opentelemetry/opentelemetry-exporter-otlp/runtime/src/main/java/io/quarkus/opentelemetry/exporter/otlp/runtime/OtlpRecorder.java

Line 27 in c378629

OtlpGrpcSpanExporterBuilder otlpGrpcSpanExporterBuilder = OtlpGrpcSpanExporter.builder()

Maybe you can use the OTel Collector in the middle to receive Quarkus gRPC data and then export it with the otlphttp to your service? https://opentelemetry.io/docs/collector/configuration/#exporters

[artkonr]

very well @radcortez , thanks for the help 👍 I guess I'll close this ticket then