[ENH] OTel tracing throughout the codebase

[beggers]

Description of changes

This PR adds OpenTelemetry tracing to ~all major methods throughout our codebase. It also adds configuration to specify where these traces should be sent. I focused more on laying the groundwork for tracing than on collecting all the data we need everywhere.

Default behavior is unchanged: no tracing, no printing.

Summarize the changes made by this PR.

• New functionality
  • OpenTelemetryClient with relevant config.
  • Wrap most of our code in tracing.

The only major design decision I made was to fully separate OpenTelemetry stuff and Posthog (product telemetry) stuff.

Justification:

It's tempting to combine OTel and product telemetry behind a single internal interface. I don't think this coupling is worth it. Product telemetry cares about a small and relatively static set of uses, whereas tracing by nature should be very deep in our codebase. I see two ways to couple them and problems with each:

• Have a unified Telemetry interface only for the events our product telemetry cares about and use raw OTel for the rest. In other words, use this telemetry interface only for collection.add()s, collection.delete()s, etc. This seems weird to me: tracing code would be implicit in some cases but explicit in others, making the codebase less easily comprehensible. Also if an engineer later decides to add product telemetry to a codepath that already has tracing, they need to know to remove existing tracing. This increases the cognitive overhead required to work on Chroma, reducing the readability and maintainability of our codebase.
• Have a unified Telemetry interface which does everything. In this case, it has the above behavior but also wraps all other OTel behavior we want. This seems weird to me because we're basically writing a wrapper around the complete set of OpenTelemetry behavior which only modifies a small part of it. This increases our maintenance burden for very little value.

Instead we have two well-encapsulated telemetry modules which we can modify and use without worrying about the other telemetry. The OTel module provides some lightweight helpers to make OTel a little easier to use, but we can put raw OTel code throughout our codebase and it'll play nicely.

Test plan

How are these changes tested?

• Tests pass locally with pytest for python, yarn test for js

Manual testing:

• Set environment variables to export traces to Honeycomb at
  various granularities.
• Went through various basic Chroma flows and checked that traces show up in Honeycomb as expected.

Documentation Changes

Are all docstrings for user-facing APIs updated if required? Do we need to make documentation changes in the docs repository?

Docs PR to land before this does.

[github-actions]

Reviewer Checklist

Please leverage this checklist to ensure your code review is thorough before approving

Testing, Bugs, Errors, Logs, Documentation

• Can you think of any use case in which the code does not behave as intended? Have they been tested?
• Can you think of any inputs or external events that could break the code? Is user input validated and safe? Have they been tested?
• If appropriate, are there adequate property based tests?
• If appropriate, are there adequate unit tests?
• Should any logging, debugging, tracing information be added or removed?
• Are error messages user-friendly?
• Have all documentation changes needed been made?
• Have all non-obvious changes been commented?

System Compatibility

• Are there any potential impacts on other parts of the system or backward compatibility?
• Does this change intersect with any items on our roadmap, and if so, is there a plan for fitting them together?

Quality

• Is this code of a unexpectedly high quality (Readbility, Modularity, Intuitiveness)

[HammadB]

discussed offline - make this strongly typed and put onus of conversion on caller, error internally if not expected types

[beggers]

Document params