Update Getting Started guide with improved Docker and K8s setup details

[ericmustin]

Hello friends,

I'd like to kindly ask about whether there are plans to update the Getting Started guides for docker and kubernetes on the current roadmap? At present, they leave a lot to be desired

• For example, Docker points to a local repo instead of the appropriate image.

• And, the Kubernetes setup guide points a k8s manifest which doesn't include any references to the k8s_processor, which is going to be required for the very large majority of OTEL users deploying the collector in a production otel environment.

While it's great to see so many contributors from so many Commercial Open Source Software vendors, it seems like most of the organisations involves have preferred to contribute to their own proprietary documentation that exists in their own repositories or on their forks of the opentelemetry collector, and these documentation improvements have not made it back upstream . Some examples, (not meant in a negative way, just showing off some strong documentation, diagrams, or explanations of key concepts which live outside the OTEL docs).

• https://github.com/signalfx/splunk-otel-collector
• https://docs.signalfx.com/en/latest/apm/apm-getting-started/apm-opentelemetry-collector.html
• https://www.elastic.co/guide/en/apm/get-started/current/open-telemetry-elastic.html
• https://help.sumologic.com/Traces/Getting_Started_with_Transaction_Tracing/Set_up_traces_collection_for_Kubernetes_environments
• https://opentelemetry.lightstep.com/the-collector-and-exporters/

Not having the best in class docs live within the opentelemetry.io documentation itself makes it difficult for users to onboard onto OpenTelemetry in a way that is generally agreed to be portable+vendor agnostic. In the long run it may hurt adoption as users struggle to onboard or don't enable the optimal processors within their collector pipelines. It would be wonderful to see some better documentation and getting started examples , especially for containerised and k8s environments

[austinlparker]

FWIW I'm very supportive of the idea of improving these (opentelemetry.io) docs, I've just haven't had the time to commit to improving the upstream docs in a while. Would absolutely welcome even small PRs to improve them, though!

[bboreham]

My 2 cents as someone with a bit of experience in metrics and tracing: when I was trying to get started with OpenTelemetry in Go a couple of months back I found it very frustrating.

My recommendation would be to port this page to Otel: https://opentracing.io/docs/getting-started/
The similar page here has improved since I needed it, but the example dumps traces to stdout. That is not going to show newcomers the power of distributed tracing.

[SergeyKanzhelev]

consider referencing posts from @jpkrohling on the topic: https://medium.com/@jpkroehling

[flands]

@ericmustin have a look at #380. The getting started documentation was ported directly from the core repository and the core repository had pre-existing examples. I went ahead and added the Docker Hub information especially since it comes with a default configuration now this really is easier than the example -- thanks for pointing this out!

Regarding k8s -- the examples are built on the core repository and the k8s_processor is in contrib. Unfortunately, the k8s_processor is lacking documentation which includes RBAC rules. Once this is added, it would be easy enough to update the k8s example to leverage this processor.

There is a bigger question around core versus contrib docs -- for getting started, current thought process is that core should be the priority, Suggestions welcome here.

Regarding vendor documentation -- I am familiar with the first two links :) what do you think is missing from the getting started documentation here? Fully agree with Austin that the goal should be to provide the best docs on this site.

[jpkrohling]

consider referencing posts from @jpkrohling on the topic

Feel free to use/reuse the blog posts as you see fit!

[ericmustin]

@flands Sorry for the delayed response, I have been a little underwater.

I appreciate all the work you've put in here. I think it shows, and certainly makes the docs clearer. The docker addition in #380 along with the additional details around Linux packaging is definitely helpful.

I do happen to think that a well documented k8s_processor will be quite necessary for most k8s use cases, so i'm glad to hear that's being considered to be added to onboarding docs. Perhaps I am mistaken but afaik, in an "otel-agent running as daemonset + otel-collector running as gateway service" scenario, which is what the example k8s manifest shows, any sort of accurate correlation of traces or metrics with appropriate host would not be possible, unless the application sdk or service emitting the telemetry data to the otel-agents were adding that metadata.

That being said it's entirely possible I am mistaking the limitations/concerns/bias of my own employer with that of the community (that seems to be a common habit for folks these days 😅 ), and a k8s_processor is not nearly as important as I've indicated. In which case, disregard!

Broadly speaking, while considering whether to focus on core vs contrib, I would encourage the docs to highlight relevant contrib components where applicable in the future. For example, highlighting the docker stats receiver for dockerized setups, and so on.

Lastly (hats off if you're still reading), regarding your followup Q, Two areas I think some vendor docs excel, especially LS,

• they do a 1st class job of walking through the concepts with visual aids, and in general I think they provide a clear explanation of some otherwise confusing topics. Their explanation of context propagation is a good example of making few assumptions about what a user might already know about Tracing, starting broadly, and working toward specific implementation details for OpenTelemetry. I'm not sure this type of bottoms up approach to key concepts exists as thoughtfully in the Otel docs.

• They also do a good job of suggesting useful semantic conventions to use for things like resource attributes. Something like service.version and service.name, which are more or less "key" resource attributes, are specified clearly in the LS docs but not so much in the OpenTelemetry.io docs as far as I can tell. The OTEL SDK Env vars would also be a nice thing to highlight since their use will be a much easier onboarding path than attempting to have users modify application code with, in some cases, potentially dynamic values. Lastly it's just sort've like, if we went through all the trouble of bikeshedding for years over semantic conventions and attribute names and so on, we should totally promote their use now that there's some stability and consensus.

Anyway @flands and @austinlparker and friends, I hope that helps, I am totally fine if you want to close this issue, or if you'd prefer to leave it open for further comment from the community. ty for coming to my ted talk.

[austinlparker]

I'm fine leaving it open for more discussion for now. If we can identify specific workable items (such as visual aids for concepts, context prop, etc.) then let's split those out into separate issues.

[bboreham]

My recommendation would be to port this page to Otel: https://opentracing.io/docs/getting-started/

I had a go at this: https://github.com/bboreham/opentracing.io/blob/otel/content/guides/golang/quick-start.md

[cartermp]

Probably more to improve, but the collector getting started docs have improved a bit since late last year: https://opentelemetry.io/docs/collector/getting-started/

Some other things mentioned here are also in docs now:

• The spec is pulled into the site itself, so anything spec-related is now searchable in the site
• There are now conceptual docs for common env vars
• There are (thankfully) now more visual aids in the docs
• Some of the vendor-owned docs (lightstep and honeycomb) have been pulled in - these are language-agnostic conceptual docs
• Things like service.name/service.version are present in the getting started and manual instrumentation guides for several languages now (intent is that we show best practices for all languages)
• Semantic conventions do get a call-out in JS docs, and should definitely be added in to every other language's docs just so that there's awareness about them no matter which language you're coming from.

I filed #1538 to track the specific issue of recommending the k8sattributesprocessor by default when getting started with k8s.

I also filed #1539 to track adding semantic attributes to each language's docs pages since.

Given that, I'll close this in favor of the two specific issues we've had. Thanks a bunch of the feedback (that I'm now reading like 1.5 yrs later, heh). And please do comment on if there's more you think should be added. We can break them out into specific issues.