Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[APM] Prefer span metrics over span events #141519

Merged
merged 10 commits into from
Oct 6, 2022

Conversation

cauemarcondes
Copy link
Contributor

@cauemarcondes cauemarcondes commented Sep 22, 2022

closes #139728

@cauemarcondes cauemarcondes requested a review from a team as a code owner September 22, 2022 20:05
@botelastic botelastic bot added the Team:APM All issues that need APM UI Team support label Sep 22, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/apm-ui (Team:APM)

@@ -52,6 +52,7 @@ export default function ApiTest({ getService }: FtrProviderContext) {
environment,
kuery,
dependencyName,
searchServiceDestinationMetrics: false,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we have two API tests that ensure we get the same response back regardless if searchServiceDestinationMetrics is true or false?

Copy link
Member

@sorenlouv sorenlouv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. Please manually double check that this does return the same data wtih and without service destination metrics.

@cauemarcondes
Copy link
Contributor Author

cauemarcondes commented Sep 23, 2022

@sqren I was adding the API test you mentioned, for span metrics, and I noticed that the values are not the same. Take a look.

This is the ES response for the query. On the left querying Span events (searchServiceDestinationMetrics: false) and on the left Span metrics (searchServiceDestinationMetrics: true):

Screen Shot 2022-09-23 at 3 08 33 PM

  • For the Span events you see doc_count: 20 and duration: 1000000
  • For the Span metrics you see doc_count: 1 and duration: 20000000

Thoughts?
cc: @dgieselaar

@dgieselaar
Copy link
Member

@cauemarcondes are you considering weight? These are metrics, so a single span metrics might represent multiple documents. Simply averaging the duration isn't accurate because no weight is applied. This automatically happens for histogram fields, but this is not a histogram.

@cauemarcondes
Copy link
Contributor Author

cauemarcondes commented Oct 4, 2022

@dgieselaar / @sqren I made some changes to calculate the latency manually when it's using span metrics. I sum the field span.destination.service.response_time.sum.us and divide it by the sum of the count field span.destination.service.response_time.count this results in the average latency for span metrics. Comparing the results with the Span events, the values still don't match, but I think they are very close:

Screen Shot 2022-10-04 at 2 53 54 PM

What do you think? FYI: I used apm-integration-test with 100% sample rate to test it.

@cauemarcondes cauemarcondes enabled auto-merge (squash) October 6, 2022 16:05
@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
apm 3.1MB 3.1MB +144.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@cauemarcondes cauemarcondes merged commit dc53a59 into elastic:main Oct 6, 2022
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 6, 2022
* [APM] Prefer span metrics over span events

* [APM] Prefer span metrics over span events

* using weighted_avg when span metrics

* fixing span metrics

* throughput fix

* adding api test

(cherry picked from commit dc53a59)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.5

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Oct 6, 2022
* [APM] Prefer span metrics over span events

* [APM] Prefer span metrics over span events

* using weighted_avg when span metrics

* fixing span metrics

* throughput fix

* adding api test

(cherry picked from commit dc53a59)

Co-authored-by: Cauê Marcondes <[email protected]>
WafaaNasr pushed a commit to WafaaNasr/kibana that referenced this pull request Oct 11, 2022
* [APM] Prefer span metrics over span events

* [APM] Prefer span metrics over span events

* using weighted_avg when span metrics

* fixing span metrics

* throughput fix

* adding api test
WafaaNasr pushed a commit to WafaaNasr/kibana that referenced this pull request Oct 14, 2022
* [APM] Prefer span metrics over span events

* [APM] Prefer span metrics over span events

* using weighted_avg when span metrics

* fixing span metrics

* throughput fix

* adding api test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:fix Team:APM All issues that need APM UI Team support v8.5.0 v8.6.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[APM] Prefer span metrics over span events
6 participants