Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(operator): User-guide for OTLP configuration #14620

Merged
merged 5 commits into from
Oct 29, 2024

Conversation

xperimental
Copy link
Collaborator

What this PR does / why we need it:

This PR provides a user-guide for setting up the OTLP configuration using the LokiStack resource provided by the Loki Operator to be displayed on the operator's website.

Which issue(s) this PR fixes:

Special notes for your reviewer:

Checklist

  • Reviewed the CONTRIBUTING.md guide (required)
  • Documentation added
  • Title matches the required conventional commits format, see here

@xperimental
Copy link
Collaborator Author

Marked as a draft, because I want to give it a second read as well. But please feel free to review already 🙂


Because OTLP is not specifically geared towards Loki but is a standard format, it needs additional configuration on Loki's side to map the OpenTelemetry data format to Loki's data model.

Specifically, OTLP has no concept of "stream labels" or "structured metadata". OTLP instead provides metadata about a log entry in _attributes_ that are grouped into three buckets (resource, scope and log), depending on the number of entries an attribute applies to.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the end of this phrase, I'm unsure what you mean: "depending on the number of entries an attribute applies to."

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had rewritten this paragraph on the second try. Can you check, if the new text makes this clearer?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes much better 👍

operator/docs/user-guides/open-telemetry.md Outdated Show resolved Hide resolved
operator/docs/user-guides/open-telemetry.md Outdated Show resolved Hide resolved
operator/docs/user-guides/open-telemetry.md Outdated Show resolved Hide resolved
operator/docs/user-guides/open-telemetry.md Outdated Show resolved Hide resolved

Once the `effectiveDate` has passed your `LokiStack` will be using the new schema configuration and is ready to store structured metadata.

## Attribute Mapping
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we briefly describe the difference between stream label & structured metadata? In this section or other section that you find relevant

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried getting around that by linking to the stream-label / structured-metadata documentation on the Loki site. Another idea I had, was for us to have a few "concepts" pages where we can describe things like stream-labels, structured-metadata, schema versions and so on.

I'll think about this again. Maybe it makes sense to make this distinction clear in this document as well in a short paragraph.

Copy link
Collaborator Author

@xperimental xperimental left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated with code-review comments and my second read from Friday.

I'll give it another go, but I'm also marking this as ready-to-review now.


Once the `effectiveDate` has passed your `LokiStack` will be using the new schema configuration and is ready to store structured metadata.

## Attribute Mapping
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried getting around that by linking to the stream-label / structured-metadata documentation on the Loki site. Another idea I had, was for us to have a few "concepts" pages where we can describe things like stream-labels, structured-metadata, schema versions and so on.

I'll think about this again. Maybe it makes sense to make this distinction clear in this document as well in a short paragraph.


Because OTLP is not specifically geared towards Loki but is a standard format, it needs additional configuration on Loki's side to map the OpenTelemetry data format to Loki's data model.

Specifically, OTLP has no concept of "stream labels" or "structured metadata". OTLP instead provides metadata about a log entry in _attributes_ that are grouped into three buckets (resource, scope and log), depending on the number of entries an attribute applies to.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had rewritten this paragraph on the second try. Can you check, if the new text makes this clearer?

@xperimental xperimental marked this pull request as ready for review October 28, 2024 10:54
@xperimental xperimental requested review from periklis and a team as code owners October 28, 2024 10:54
Copy link
Collaborator

@JoaoBraveCoding JoaoBraveCoding left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my side lgtm 👍

@periklis periklis merged commit 27b4071 into grafana:main Oct 29, 2024
68 checks passed
@xperimental xperimental deleted the otlp-docs branch October 29, 2024 11:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants