Skip to content

Commit

Permalink
Fix potential deadlock (#59)
Browse files Browse the repository at this point in the history
## Motivation

Some `Debug` implementations access a span's extensions, for example a
`color_eyre` `Report` when printing a Spantrace.
This can currently lead to a deadlock because `tracing-opentelemetry`'s
tracing `Layer` holds an `extensions_mut()` guard for its Span while
recording the event.

## Solution

Move the the `OtelData` out of the span to not hold the lock and later
write the extension back.
  • Loading branch information
AsmPrgmC3 authored Sep 27, 2023
1 parent 5223a67 commit 1c61ea6
Showing 1 changed file with 8 additions and 5 deletions.
13 changes: 8 additions & 5 deletions src/layer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -835,10 +835,9 @@ where
#[cfg(not(feature = "tracing-log"))]
let target = target.string(meta.target());

let mut extensions = span.extensions_mut();
let span_builder = extensions
.get_mut::<OtelData>()
.map(|data| &mut data.builder);
// Move out extension data to not hold the extensions lock across the event.record() call, which could result in a deadlock
let mut otel_data = span.extensions_mut().remove::<OtelData>();
let span_builder = otel_data.as_mut().map(|data| &mut data.builder);

let mut otel_event = otel::Event::new(
String::new(),
Expand All @@ -852,7 +851,9 @@ where
exception_config: self.exception_config,
});

if let Some(OtelData { builder, .. }) = extensions.get_mut::<OtelData>() {
if let Some(mut otel_data) = otel_data {
let builder = &mut otel_data.builder;

if builder.status == otel::Status::Unset
&& *meta.level() == tracing_core::Level::ERROR
{
Expand Down Expand Up @@ -895,6 +896,8 @@ where
} else {
builder.events = Some(vec![otel_event]);
}

span.extensions_mut().replace(otel_data);
}
};
}
Expand Down

0 comments on commit 1c61ea6

Please sign in to comment.