v1.0 requirements

[samspills]

Hello there 👋🏻

I don't see any other issues or discussions about the subject so I thought I'd open one, I hope that's okay!

What are the concrete requirements for otel4s to make a v1.0 release?

The api seems consistent, and java otel is >1.0, so those pieces could be broken out and a stable release made while the scala sdk stays pre-1.0 ? We'd like to rely on otel4s at $work but we're quite shy about relying on a pre-1.0 library due to the possible toil across teams with potential breaking changes. If there's a plan with steps towards v1.0, we'd be happy to help out!!

[iRevive]

Hey, thanks for raising this question. I have a feeling that v1.0 is long overdue.

Current status

otel4s-core and otel4s-oteljava are fairly consistent for the past (6?) releases.
Even though there were breaking changes, the user-facing API remained almost unchanged since 0.5.0.

otel4s-sdk looks relatively stable. Specification-wise, the implementation is in a good spot:

• There are both HTTP and gRPC OTLP exporters
• Both push-based (OTLP) and pull-based (Prometheus) metrics exporters have been implemented and tested
• The implementation is flexible enough and can be extended

Remaining items to explore before 1.0

These items fall into the 'nice-to-have' category but are not mandatory.

1. Seamless propagation of context for oteljava

Here is the prototype: #214.

Waiting for: typelevel/cats-effect#3636.

2. CE runtime metrics (WSTP, Starvation)

Metrics would be implemented in the new otel4s-instrumentation package and available for both sdk and oteljava.

There is an experimental implementation on top of MBeans: https://github.com/typelevel/otel4s-experimental?tab=readme-ov-file#1-iometrics---cats-effect-runtime-metrics, https://github.com/typelevel/otel4s-experimental/blob/main/modules/metrics/jvm/src/main/scala/org/typelevel/otel4s/experimental/metrics/IOMetrics.scala.

Ideally, I would implement them on top of:

• Add IORuntimeMetrics and CpuStarvationMetrics cats-effect#4106
• Implement IORuntimeMetrics cats-effect#3317

That way, the CE runtime metrics will be available on all platforms.

3. Instrumentation of the fs2 Stream

#848

I never needed it, so I never explored this issue.

So what's the plan

Even though some things would be nice to have, I assume most users don't need them right now.
The v1.0 would finally make otel4s viable for various libraries.

1. Revisit current implementation (core, oteljava, SDK)

1. Remove unused traits, type params, etc
2. Make sure internals are private or package-private
3. Check for any opinionated behavior, for example core-trace: use ExitCase to build attributes in Histogram#recordDuration #761

2. Decide whether we want to keep otel4s-sdk outside the 1.0 and otel4s main repository

That means we should move the sdk code to another repo, right?

Pros:

• independent release cycles
• lightweight otel4s repository
• break (potential) future cycling dependencies: (http4s<->otel4s-sdk-exporter). Here is the discussion Proposal: otel4s-sdk repo #199

Cons:

• independent release cycles
• maintenance burden
• separate documentation
• cannot run tests against two implementations (SDK and oteljava) simultaneously

It's important to mention that I don't know the actual adoption of the otel4s-sdk. Perhaps we can get numbers from Sonatype.
I'm certain that https://github.com/lichess-org/lila-search is using otel4s-sdk, but I have no idea about other projects.

While I like the idea of the Scala-first backend that can be used on all platforms, does it really have any use?
The more or less relevant use case is integration with https://github.com/typelevel/feral.
In addition, otel4s-oteljava and otel4s-sdk are low-key confusing names. At least for me, otel4s-sdk resonates as a 'default backend' and otel4s-oteljava as an experimental integration, while it's the opposite.

When I started writing a response, I firmly believed that sdk should remain in the main repository. Now, I have doubts, and I tend to think that moving it out is a good thing 😅

[iRevive]

Apart from the numbers, we should also discuss potential cyclic dependencies: #199.
If I'm not mistaken, there was a plan to use otel4s directly in http4s. If the plan is still relevant, we cannot keep otel4s-sdk in the main repo.

[bpholt]

While I like the idea of the Scala-first backend that can be used on all platforms, does it really have any use?

FWIW, one of the reasons I'm very excited about otel4s-sdk is because the last time I checked, the OTel Java SDK brings in some dependencies that we've had bad bincompat experiences with in the past. The Typelevel community does such a good job maintaining bincompat and correctly encoding those expectations using semver, and I think it's hard to overstate how much pain that helps its users avoid. IMO otel4s-sdk will definitely be preferred to otel4s-oteljava if you're not also using another library that is already instrumented with oteljava.

• cannot run tests against two implementations (SDK and oteljava) simultaneously

Maybe otel4s-oteljava could also be broken into a separate repo to put the two implementations on equal footing?

[iRevive]

Maybe otel4s-oteljava could also be broken into a separate repo to put the two implementations on equal footing?

Currently we have several shared test suite, for example BaseMeterSuite and BaseTracerSuite.

Each implementation (SDK / oteljava) plugs in their backend and runs tests. That way, we ensure both implementations behave identically.

If we move SDK and oteljava to standalone repos, we either need to copy-paste the suite to each repo or publish it as an artifact.

---

A similar story with the documentation. We will need to pull SDK and oteljava artifacts to the docs project.
And that will create a cyclic dependency, which is not a big deal but could be annoying between major releases.

[samspills]

there was a plan to use otel4s directly in http4s. If the plan is still relevant, we cannot keep otel4s-sdk in the main repo.

Yup, as far as I understand it that plan is still relevant (I don't feel I have the experience to really offer an opinion about whether otel4s-oteljava should move as well, just chiming in about the http4s plans)

[NthPortal]

incidentally for point 1 of items to explore (the oteljava ContextStorageProvider), it will be its own module so that shouldn't impede releasing 1.0 terribly. it could even be in its own repo if needed (though I personally don't think it would need that)

[iRevive]

@rossabaker what's your opinion regarding the project structure?

So far, we've discussed the following options:

1. Move otel4s-sdk to a standalone repo, keep otel4s-core and otel4s-oteljava in the main repository
2. Move otel4s-sdk and otel4s-oteljava to standalone repos, keep otel4s-core in the main repository
3. Do nothing. Keep all modules in the main repository