Compressed spans #432
Labels
Comments
This was referenced Apr 27, 2021
This was referenced Jun 7, 2021
beniwohli
added a commit
to beniwohli/apm-agent-python
that referenced
this issue
Sep 9, 2021
This was referenced Sep 20, 2021
2 tasks
marclop
added a commit
to elastic/apm-agent-go
that referenced
this issue
Oct 13, 2021
Implements the span compression algorithm described in elastic/apm#432. The implementation pretty close to what the spec describes, with a few modifications due to the differences in the Go agent implementation. The span compression is experimental and is disabled by default with the default thresholds for the different compression strategies set: - `ELASTIC_APM_SPAN_COMPRESSION_ENABLED=false` - `ELASTIC_APM_SPAN_COMPRESSION_EXACT_MATCH_MAX_DURATION=50ms` - `ELASTIC_APM_SPAN_COMPRESSION_SAME_KIND_MAX_DURATION=5ms` The implementation uses a new field (cache `*Span`) in `TransactionData` and `SpanData` which is used to cache compressed or compression eligible spans in that field. An additional `composite` field has also been added to the `SpanData` and `TransactionData`. The algorithm states that any exit span that is lower or equal duration than the set `ELASTIC_APM_SPAN_COMPRESSION_EXACT_MATCH_MAX_DURATION` or `ELASTIC_APM_SPAN_COMPRESSION_SAME_KIND_MAX_DURATION` with a destination service set is compressable into a composite span using the appropriate strategy. Compressable spans need to be "exit" and siblings which **have not** propagated their context downstream. Sibling spans are spans that share the same parent span (or transaction). Spans can be compressed with `same_kind` or `exact_match` as a strategy, and their sum is the aggregated duration of of all When a compressed span has been compressed into a composite using the `same_kind` strategy, its name is mutated to `Calls to <span.Type>`. Spans will be compressed into a composite only when the siblings are consecutive and any compression attempt that doesn't meet that, will cause the cache to be evicted. Additionally, two helper functions have been added under the `apmtest`: package: `WriteTraceTable` and `WriteTraceWaterfall`, which help with understanding why a test is failing if they are and debugging those. Signed-off-by: Marc Lopez Rubio <[email protected]>
beniwohli
added a commit
to elastic/apm-agent-python
that referenced
this issue
Oct 16, 2021
* Implemented span compression algorithm See elastic/apm#432 * disable span compression by default in tests * make _try_to_compress_composite and _try_to_compress_regular side effect free * fix issue with compression buffer not being reported * fix some type hinting issues/errors * move traceparent initialization into Transaction constructor This cleans up a wart where `traceparent` is initialized to None, and needs to be set by the callee right after, as there is code that assumes `traceparent` to be set. * fix breakdown metrics calculation * fix breakdown metrics test we no longer only call `child_ended` for breakdown metrics * update duration of the composite span based on compressed spans * add docs for compressed span config options * clean up merge Co-authored-by: Colton Myers <[email protected]>
2 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Implement the span compression algorithm in the agents to handle excessive database calls and huge amount of external spans.
The algorithm is described in the handling huge traces spec.
Spec issue
Agent issues
The text was updated successfully, but these errors were encountered: