Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support otel.instrumentation.kafka.experimental-span-attributes in spring starter #11263

Merged
merged 5 commits into from
May 15, 2024
Merged

support otel.instrumentation.kafka.experimental-span-attributes in spring starter #11263

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
f1add51
support otel.instrumentation.kafka.experimental-span-attributes in sp…
zeitlinger Apr 30, 2024
54cddcc
use ConfigProperties instead of spring style injection, because it's …
zeitlinger May 3, 2024
8271414
add test for experimental kafka span attributes
zeitlinger May 14, 2024
00b4424
better bean name
zeitlinger May 15, 2024
06d9404
comment
zeitlinger May 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 56 additions & 19 deletions ...io/opentelemetry/instrumentation/spring/autoconfigure/OpenTelemetryAutoConfiguration.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,16 @@
import io.opentelemetry.sdk.autoconfigure.internal.AutoConfigureUtil;
import io.opentelemetry.sdk.autoconfigure.internal.ComponentLoader;
import io.opentelemetry.sdk.autoconfigure.internal.SpiHelper;
import io.opentelemetry.sdk.autoconfigure.spi.ConfigProperties;
import io.opentelemetry.sdk.autoconfigure.spi.ResourceProvider;
import io.opentelemetry.sdk.autoconfigure.spi.internal.DefaultConfigProperties;
import java.util.Collections;
import java.util.List;
import java.util.Optional;
import java.util.function.Function;
import java.util.function.Supplier;
import java.util.stream.Collectors;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.ConfigurationPropertiesBinding;
Expand Down Expand Up @@ -80,31 +86,47 @@ public ResourceProvider otelDistroVersionResourceProvider() {
}

@Bean
// If you change the bean name, also change it in the OpenTelemetryJdbcDriverAutoConfiguration
// class
public OpenTelemetry openTelemetry(
public AutoConfiguredOpenTelemetrySdk autoConfiguredOpenTelemetrySdk(
Environment env,
OtlpExporterProperties otlpExporterProperties,
OtelResourceProperties resourceProperties,
PropagationProperties propagationProperties,
OpenTelemetrySdkComponentLoader componentLoader) {

OpenTelemetry openTelemetry =
AutoConfigureUtil.setComponentLoader(
AutoConfigureUtil.setConfigPropertiesCustomizer(
AutoConfiguredOpenTelemetrySdk.builder(),
c ->
SpringConfigProperties.create(
env,
otlpExporterProperties,
resourceProperties,
propagationProperties,
c)),
componentLoader)
.build()
.getOpenTelemetrySdk();

return openTelemetry;
return AutoConfigureUtil.setComponentLoader(
AutoConfigureUtil.setConfigPropertiesCustomizer(
AutoConfiguredOpenTelemetrySdk.builder(),
c ->
SpringConfigProperties.create(
env,
otlpExporterProperties,
resourceProperties,
propagationProperties,
c)),
componentLoader)
.build();
}

@Bean
public OpenTelemetry openTelemetry(
AutoConfiguredOpenTelemetrySdk autoConfiguredOpenTelemetrySdk) {
return autoConfiguredOpenTelemetrySdk.getOpenTelemetrySdk();
}

/**
* Expose the {@link ConfigProperties} bean for use in other auto-configurations.
*
* <p>Why not use spring boot properties directly? <br>
* 1. issues with older spring boot versions <br>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldn't these issues affect our (broader) usage of spring boot properties to configure the SDK?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merging to unblock #11350, but i'd be in favor of removing this bullet (or explaining it better), since as written I think it creates more questions than it answers

Copy link
Member Author

@zeitlinger zeitlinger May 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's this style that silently returns false in boot 2

@Value("${otel.instrumentation.common.db-statement-sanitizer.enabled:true}")

https://github.com/open-telemetry/opentelemetry-java-instrumentation/pull/11350/files#diff-74d5efe463708e7c420646f0b1a8b490c76528e0abba9e8605f6f79bc73e1c40L33

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldn't these issues affect our (broader) usage of spring boot properties to configure the SDK?

no

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zeitlinger

i'd be in favor of removing this bullet (or explaining it better), since as written I think it creates more questions than it answers

Suggested change
* 1. issues with older spring boot versions <br>

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(i'll send PR)

* 2. support for {@link
* io.opentelemetry.sdk.autoconfigure.spi.AutoConfigurationCustomizer#addPropertiesCustomizer(Function)}
* and {@link
* io.opentelemetry.sdk.autoconfigure.spi.AutoConfigurationCustomizer#addPropertiesSupplier(Supplier)}
*/
@Bean
public ConfigProperties otelProperties(
zeitlinger marked this conversation as resolved.
Show resolved Hide resolved
AutoConfiguredOpenTelemetrySdk autoConfiguredOpenTelemetrySdk) {
return AutoConfigureUtil.getConfig(autoConfiguredOpenTelemetrySdk);
}
}

Expand All @@ -117,6 +139,21 @@ public static class DisabledOpenTelemetrySdkConfig {
public OpenTelemetry openTelemetry() {
return OpenTelemetry.noop();
}

@Bean
public ConfigProperties otelProperties() {
return DefaultConfigProperties.createFromMap(Collections.emptyMap());
}
}

@Configuration
@ConditionalOnBean(OpenTelemetry.class)
@ConditionalOnMissingBean({ConfigProperties.class})
zeitlinger marked this conversation as resolved.
Show resolved Hide resolved
public static class FallbackConfigProperties {
@Bean
public ConfigProperties otelProperties() {
return DefaultConfigProperties.create(Collections.emptyMap());
}
}

/**
Expand Down
13 changes: 11 additions & 2 deletions ...configure/instrumentation/kafka/ConcurrentKafkaListenerContainerFactoryPostProcessor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,21 @@

import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.instrumentation.spring.kafka.v2_7.SpringKafkaTelemetry;
import io.opentelemetry.sdk.autoconfigure.spi.ConfigProperties;
import org.springframework.beans.factory.ObjectProvider;
import org.springframework.beans.factory.config.BeanPostProcessor;
import org.springframework.kafka.config.ConcurrentKafkaListenerContainerFactory;

class ConcurrentKafkaListenerContainerFactoryPostProcessor implements BeanPostProcessor {

private final ObjectProvider<OpenTelemetry> openTelemetryProvider;
private final ObjectProvider<ConfigProperties> configPropertiesProvider;

ConcurrentKafkaListenerContainerFactoryPostProcessor(
ObjectProvider<OpenTelemetry> openTelemetryProvider) {
ObjectProvider<OpenTelemetry> openTelemetryProvider,
ObjectProvider<ConfigProperties> configPropertiesProvider) {
this.openTelemetryProvider = openTelemetryProvider;
this.configPropertiesProvider = configPropertiesProvider;
}

@Override
Expand All @@ -29,7 +33,12 @@ public Object postProcessAfterInitialization(Object bean, String beanName) {
ConcurrentKafkaListenerContainerFactory<?, ?> listenerContainerFactory =
(ConcurrentKafkaListenerContainerFactory<?, ?>) bean;
SpringKafkaTelemetry springKafkaTelemetry =
SpringKafkaTelemetry.create(openTelemetryProvider.getObject());
SpringKafkaTelemetry.builder(openTelemetryProvider.getObject())
.setCaptureExperimentalSpanAttributes(
configPropertiesProvider
.getObject()
.getBoolean("otel.instrumentation.kafka.experimental-span-attributes", false))
zeitlinger marked this conversation as resolved.
Show resolved Hide resolved
.build();
listenerContainerFactory.setBatchInterceptor(springKafkaTelemetry.createBatchInterceptor());
listenerContainerFactory.setRecordInterceptor(springKafkaTelemetry.createRecordInterceptor());

Expand Down
7 changes: 5 additions & 2 deletions ...ion/spring/autoconfigure/instrumentation/kafka/KafkaInstrumentationAutoConfiguration.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.instrumentation.kafkaclients.v2_6.KafkaTelemetry;
import io.opentelemetry.instrumentation.spring.autoconfigure.internal.ConditionalOnEnabledInstrumentation;
import io.opentelemetry.sdk.autoconfigure.spi.ConfigProperties;
import org.springframework.beans.factory.ObjectProvider;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.kafka.DefaultKafkaProducerFactoryCustomizer;
Expand All @@ -32,7 +33,9 @@ DefaultKafkaProducerFactoryCustomizer otelKafkaProducerFactoryCustomizer(
@Bean
static ConcurrentKafkaListenerContainerFactoryPostProcessor
otelKafkaListenerContainerFactoryBeanPostProcessor(
ObjectProvider<OpenTelemetry> openTelemetryProvider) {
return new ConcurrentKafkaListenerContainerFactoryPostProcessor(openTelemetryProvider);
ObjectProvider<OpenTelemetry> openTelemetryProvider,
ObjectProvider<ConfigProperties> configPropertiesProvider) {
return new ConcurrentKafkaListenerContainerFactoryPostProcessor(
openTelemetryProvider, configPropertiesProvider);
}
}
6 changes: 6 additions & 0 deletions ...t-autoconfigure/src/main/resources/META-INF/additional-spring-configuration-metadata.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -285,6 +285,12 @@
"description": "Enable the Kafka instrumentation.",
"defaultValue": true
},
{
"name": "otel.instrumentation.kafka.experimental-span-attributes",
"type": "java.lang.Boolean",
"description": "Enable the capture of experimental Kafka span attributes.",
"defaultValue": false
},
{
"name": "otel.instrumentation.log4j-appender.enabled",
"type": "java.lang.Boolean",
Expand Down
5 changes: 3 additions & 2 deletions ...pentelemetry/instrumentation/spring/autoconfigure/OpenTelemetryAutoConfigurationTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,8 @@ void customOpenTelemetry() {
context ->
assertThat(context)
.hasBean("customOpenTelemetry")
.doesNotHaveBean("openTelemetry"));
.doesNotHaveBean("openTelemetry")
.hasBean("otelProperties"));
}

@Test
Expand All @@ -57,7 +58,7 @@ void customOpenTelemetry() {
void initializeProvidersAndOpenTelemetry() {
this.contextRunner
.withConfiguration(AutoConfigurations.of(OpenTelemetryAutoConfiguration.class))
.run(context -> assertThat(context).hasBean("openTelemetry"));
.run(context -> assertThat(context).hasBean("openTelemetry").hasBean("otelProperties"));
}

@Test
Expand Down
1 change: 1 addition & 0 deletions ...c/main/java/io/opentelemetry/spring/smoketest/AbstractJvmKafkaSpringStarterSmokeTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ void setUpContext() {
KafkaInstrumentationAutoConfiguration.class,
KafkaConfig.class))
.withPropertyValues(
"otel.instrumentation.kafka.experimental-span-attributes=true",
"spring.kafka.bootstrap-servers=" + kafka.getBootstrapServers(),
"spring.kafka.consumer.auto-offset-reset=earliest",
"spring.kafka.consumer.linger-ms=10",
Expand Down
4 changes: 4 additions & 0 deletions .../src/main/java/io/opentelemetry/spring/smoketest/AbstractKafkaSpringStarterSmokeTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
import static io.opentelemetry.sdk.testing.assertj.OpenTelemetryAssertions.satisfies;

import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.api.common.AttributeKey;
import io.opentelemetry.api.trace.SpanKind;
import io.opentelemetry.semconv.incubating.MessagingIncubatingAttributes;
import org.apache.kafka.clients.admin.NewTopic;
Expand Down Expand Up @@ -97,6 +98,9 @@ void shouldInstrumentProducerAndConsumer() {
equalTo(
MessagingIncubatingAttributes.MESSAGING_KAFKA_CONSUMER_GROUP,
"testListener"),
satisfies(
AttributeKey.longKey("kafka.record.queue_time_ms"),
AbstractLongAssert::isNotNegative),
satisfies(
MessagingIncubatingAttributes.MESSAGING_CLIENT_ID,
stringAssert -> stringAssert.startsWith("consumer"))),
Expand Down
2 changes: 2 additions & 0 deletions smoke-tests-otel-starter/spring-boot-common/src/main/resources/application.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
otel:
instrumentation:
kafka:
experimental-span-attributes: true
logback-appender:
experimental:
capture-code-attributes: true
Expand Down
Loading