Direct integration with prometheus instead of libp2p-metrics

[thomaseizinger]

Currently, gathering metrics from libp2p is done by integrating the libp2p-metrics crate which requires the user to do the wiring themselves, i.e. in the event-loop where the poll the Swarm, they need to also pass each event to the metrics instance to record all sorts of data.

Pros

• No impact of version bumps in prometheus-client to libp2p crates other than libp2p-metrics
• No conditional code in most libp2p crates

Cons

• Every metric we want to record needs to publicly accessible through a crate's API. For example, in time to establish connection #3134 we added a API element just to record this metric.
• No metrics about Swarm internals like size of buffers or queues

---

If we make prometheus-client lean enough (i.e. by default don't expose heavy macro dependencies, etc) we can just add it as a dependency unconditionally and thus avoid cfgs throughout our code.

Similarly, if we spend some cycles on future-proofing the API to avoid more breaking changes, having it in the public API of our crates is not too big of a deal.

Long term, I think it would make sense to integrate prometheus metrics directly into the Swarm and all other protocols. This would allow us to provide the user with detailed metrics out of the box without any additional wiring boilerplate.

Opinions welcome.

[mxinden]

If we make prometheus-client lean enough (i.e. by default don't expose heavy macro dependencies, etc) we can just add it as a dependency unconditionally and thus avoid cfgs throughout our code.

Note that we would want to make use of prometheus-client macros in the corresponding libp2p-xxx crates, e.g. to generate a LabelSet. Implementing these by hand would be very verbose.

[thomaseizinger]

Good point, we just need to make sure it is a reasonably lean derive then.

[thomaseizinger]

Alternatively to prometheus, we could also integrate with OTEL. That is more capable than prometheus, i.e. it can also to spans etc and it can be converted into prometheus metrics.

However, I actually think that once we have #1533, it would be quite easy to extract several metrics by just looking at the spans. For example, we can attach a span to the task for upgrading a connection which will give us the exact duration how long it took!

Closing this for now.