Clarifying the quickstart documentation for application instrumentation

[mpetazzoni]

Hi all!

I wanted to start a conversation on making the documentation for the "common path" more visible and easier to understand. As OpenTelemetry gains in popularity, adopted by more and more application developers to instrument their applications and services, the primary audience that will be looking at the documentation is not an library maintainer instrumenting their library, but an application developer instrumenting their application.

In that context, the "common path" for instrumentation is usually to attach the Java agent to their application, and then to add more detail to their traces by adding instrumentation to their application's business logic using the OpenTelemetry API.

Going through the README.md in this repository, we're very early on presented with the instructions for attaching the Java agent to the JVM, which is great. After this, it becomes a bit more of a challenge to find the right information, as the README goes over a long list of Java agent options, supported instrumentations, etc. Eventually, you can find a section about manual instrumentation and a link to the QUICKSTART.md in https://github.com/open-telemetry/opentelemetry-java-instrumentation#configure-the-opentelemetry-gettracer.

Going through the QUICKSTART, the most confusing part for an application developer is to understand how to use OpenTelemetry.getGlobalTracer(...); in particular, it's easy to think that the parameters to this function need to match some kind of pre-defined library name.

I would like to propose the following changes:

• Adjust the structure/ordering of sections in the README to bring documentation about manual instrumentation closer to the top, ideally just after the instructions for attaching the Java agent;
• Include an example of how to get a Tracer directly in the README, within those manual instrumentation instructions;
• Provide more explicit information about the parameters of getGlobalTracer() that are easier to understand for an application developer, for example OpenTelemetry.getGlobalTracer("my-app-name", "my-app-version").

If there is consencus on making those changes, I would be happy to send a PR!

Cheers,
/Max

[jkwatson]

I've been pondering this, and I think the easiest, path-of-least resistance if you're using the auto-instrumentation agent is two-fold:

1. You just want to add attributes/events to the current span, whatever is being provided by the auto-instrumentation:

  Span span = Span.current();
  span.setAttribute(...);

2. You want to create a new span to trace some specific bits in your code:

  @WithSpan("important_method")
  void myMethod() {
    Span span = Span.current();
    span.setAttribute(...);
    ...
    //my method stuff is here
    span.addEvent(...);
  }

Does that seem right?

[mpetazzoni]

Yes, I think those are great examples to show for manual instrumentation in a Quickstart Guide.

What do you think of my other suggestions on getting the Tracer instance, and how to find that information in the README?

[tobias-]

this should work, though needs a span.end():

DOH! Thanks! I've actually look at several examples and missed that in all those places. I know this is a question for the the otel-java, but what exactly is a Scope and a Context? I (incorrectly) thought Scope was just a convenience wrapper for making sure the span was always ended via AutoCloseable. Span is described in the Glossary, but I haven't found any similar description for those two.

It actually DID work, but unlike what I assumed, it seems that the traces exporting isn't "flushed" on JVM end, so nothing was logged and I thought it didn't work. An extra Thread.sleep(6000) did it.

make sure you are using the same version of OpenTelemetry API and OpenTelemetry Java agent.

That isn't really possible right now. I'm using otel-java 0.14.0-SNAPSHOT from yesterday and 0.13.1 of the otel-javaagent, because of a "circular dependency" snafu that was fixed since the last release. Forced versions with BOM and now using 0.13.1 of both. Had problems with the "-alpha" subproject and it's dependency on old versions of the same project.

Also that @WithSpan is only needed if you actually need a span corresponding to the main method, otherwise six would be the root span if you didn't have it.

Good to know! I assumed that some sort of magic was needed because I used the agent and it required extra hoops.

[jkwatson]

The Context is a general context object that by default lives on a thread-local. It is used to store trace-related information, including the current span and baggage. I recommend reading (carefully) the javadoc on the Context class if you want to understand this more fully.

The Scope is a hook that is used to remove the current span from the Context when it the Scope is closed. (note...since the Context is immutable, what this actually does is to replace the thread local (or whatever the storage might be) with the previously attached Context).

[anuraaga]

@tobias- I think you already know the concepts Context and Scope well :) When comparing to Armeria, Context == RequestContext and Scope == SafeCloseable. Just as closing SafeCloseable doesn't end the Armeria request, closing Scope doesn't either, they only modify the threadlocal. This is because context may need to be mounted to threadlocal several times during the processing of a Span == HttpRequest when using asynchronous processing.

[tobias-]

@jkwatson, @anuraaga Thanks for the explanation, it confirmed my guesses from when I wrote the question. @anuraaga I don't think I've ever actually used anything SafeCloseable 😊 .

[ecourreges-orange]

To follow up on this, a GlobalOpenTelemetry.getTracer("...") example is missing from the doc: https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/manual-instrumentation.md#creating-spans-manually-with-a-tracer
And without knowing that GlobalOpenTelemetry exists, it is fairly hard to find.

I am going to try the code example provided here and if it still works fine, I may post a Documentation PR for this.
Maybe I will add a sentence to go with @jkwatson's idea that creating a span manually should be a last recourse, as there are other ways of splitting your code and using @WithSpan instead.

[HarryEMartland]

It is still early days for my understanding of using OpenTelemetry and this post resonates with me.
For me, I am mainly trying to instrument spring boot APIs and I am a bit confused if I need the auto-instrument or the libraries for spring boot or both.

Would some interactive tutorials using KataKoda be welcome? As I am learning I could put something together?