Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[APM][Otel] Use fields instead of _source on APM queries #195242

Merged
merged 34 commits into from
Oct 15, 2024

Conversation

crespocarlos
Copy link
Contributor

@crespocarlos crespocarlos commented Oct 7, 2024

closes #192606

Summary

v2 based on the work done in this PR #192608 and the suggestion from Dario #194424

This PR replaces the _source usage in APM queries with fields to support Otel data. The idea is to get rid of existing UI errors we have and make sure that otel data is shown correctly in the UI.

One way to check it is using the e2e PoC.

@reakaleek
Copy link
Member

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@crespocarlos crespocarlos added apm apm:opentelemetry APM UI - OTEL Work Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team labels Oct 7, 2024
Copy link
Contributor

@rmyz rmyz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments about using _source instead of fields regarding errors, other than that LGTM, will be checking it locally today 🚀

@crespocarlos
Copy link
Contributor Author

/ci

@kibana-ci
Copy link
Collaborator

kibana-ci commented Oct 7, 2024

💔 Build Failed

Failed CI Steps

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Copy link
Contributor

@miloszmarcinkowski miloszmarcinkowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! It was a great idea to take advantage of APMEventClient returning response type.

@crespocarlos
Copy link
Contributor Author

@elasticmachine merge upstream

@crespocarlos
Copy link
Contributor Author

@elasticmachine merge upstream

elasticmachine and others added 3 commits October 14, 2024 11:28
…an-error-type-result-in-error-details-page-crashing

[APM][Otel] Errors without an error type result in error details page crashing
@elasticmachine
Copy link
Contributor

elasticmachine commented Oct 15, 2024

💚 Build Succeeded

  • Buildkite Build
  • Commit: db19165
  • Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-195242-db191655d6c9

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
apm 1844 1880 +36

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/apm-types 316 333 +17

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
apm 3.4MB 3.4MB +224.0B

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
apm 118 119 +1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
apm 36.4KB 38.0KB +1.5KB
Unknown metric groups

API count

id before after diff
@kbn/apm-types 317 334 +17

ESLint disabled line counts

id before after diff
@kbn/observability-utils 1 2 +1

Total ESLint disabled count

id before after diff
@kbn/observability-utils 1 2 +1

History

@crespocarlos crespocarlos changed the title [DO NOT MERGE][APM][Otel] Use fields instead of _source on APM queries [APM][Otel] Use fields instead of _source on APM queries Oct 15, 2024
@crespocarlos crespocarlos merged commit 7235ed0 into elastic:main Oct 15, 2024
51 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11343408262

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 15, 2024
…#195242)

closes elastic#192606

## Summary

v2 based on the work done in this PR
elastic#192608 and the suggestion from
Dario elastic#194424

This PR replaces the _source usage in APM queries with fields to support
Otel data. The idea is to get rid of existing UI errors we have and make
sure that otel data is shown correctly in the UI.

One way to check it is using the [e2e
PoC](https://github.com/elastic/otel-apm-e2e-poc/blob/main/README.md).

---------

Co-authored-by: kibanamachine <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
Co-authored-by: Jenny <[email protected]>
(cherry picked from commit 7235ed0)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

@crespocarlos crespocarlos deleted the 192606-move-source-to-fields branch October 15, 2024 10:16
kibanamachine added a commit that referenced this pull request Oct 15, 2024
…0; on APM queries (#195242) (#196265)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[APM][Otel] Use &#x60;fields&#x60; instead of &#x60;_source&#x60; on
APM queries (#195242)](#195242)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Carlos
Crespo","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-10-15T09:38:44Z","message":"[APM][Otel]
Use `fields` instead of `_source` on APM queries (#195242)\n\ncloses
https://github.com/elastic/kibana/issues/192606\r\n\r\n##
Summary\r\n\r\nv2 based on the work done in this
PR\r\nhttps://github.com//pull/192608 and the suggestion
from\r\nDario https://github.com/elastic/kibana/pull/194424\r\n\r\nThis
PR replaces the _source usage in APM queries with fields to
support\r\nOtel data. The idea is to get rid of existing UI errors we
have and make\r\nsure that otel data is shown correctly in the
UI.\r\n\r\nOne way to check it is using the
[e2e\r\nPoC](https://github.com/elastic/otel-apm-e2e-poc/blob/main/README.md).\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>\r\nCo-authored-by:
Elastic Machine
<[email protected]>\r\nCo-authored-by: Jenny
<[email protected]>","sha":"7235ed0425100bbf04ff157d0af7980875473c99","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","v9.0.0","apm","apm:opentelemetry","backport:prev-minor","ci:project-deploy-observability","Team:obs-ux-infra_services"],"title":"[APM][Otel]
Use `fields` instead of `_source` on APM
queries","number":195242,"url":"https://github.com/elastic/kibana/pull/195242","mergeCommit":{"message":"[APM][Otel]
Use `fields` instead of `_source` on APM queries (#195242)\n\ncloses
https://github.com/elastic/kibana/issues/192606\r\n\r\n##
Summary\r\n\r\nv2 based on the work done in this
PR\r\nhttps://github.com//pull/192608 and the suggestion
from\r\nDario https://github.com/elastic/kibana/pull/194424\r\n\r\nThis
PR replaces the _source usage in APM queries with fields to
support\r\nOtel data. The idea is to get rid of existing UI errors we
have and make\r\nsure that otel data is shown correctly in the
UI.\r\n\r\nOne way to check it is using the
[e2e\r\nPoC](https://github.com/elastic/otel-apm-e2e-poc/blob/main/README.md).\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>\r\nCo-authored-by:
Elastic Machine
<[email protected]>\r\nCo-authored-by: Jenny
<[email protected]>","sha":"7235ed0425100bbf04ff157d0af7980875473c99"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/195242","number":195242,"mergeCommit":{"message":"[APM][Otel]
Use `fields` instead of `_source` on APM queries (#195242)\n\ncloses
https://github.com/elastic/kibana/issues/192606\r\n\r\n##
Summary\r\n\r\nv2 based on the work done in this
PR\r\nhttps://github.com//pull/192608 and the suggestion
from\r\nDario https://github.com/elastic/kibana/pull/194424\r\n\r\nThis
PR replaces the _source usage in APM queries with fields to
support\r\nOtel data. The idea is to get rid of existing UI errors we
have and make\r\nsure that otel data is shown correctly in the
UI.\r\n\r\nOne way to check it is using the
[e2e\r\nPoC](https://github.com/elastic/otel-apm-e2e-poc/blob/main/README.md).\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>\r\nCo-authored-by:
Elastic Machine
<[email protected]>\r\nCo-authored-by: Jenny
<[email protected]>","sha":"7235ed0425100bbf04ff157d0af7980875473c99"}}]}]
BACKPORT-->

Co-authored-by: Carlos Crespo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
apm:opentelemetry APM UI - OTEL Work apm backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) ci:project-deploy-observability Create an Observability project release_note:fix Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team v8.16.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[APM][Otel] PoC Otel data with APM UI: Replacing _source with fields