support cross-namespace instrumentation references in annotations

[tKe]

Is your feature request related to a problem? Please describe.
Currently, it is required to define an Instrumentation object in every namespace in which you wish to use the operator to auto-instrument pods. These many Instrumentations may represent duplicate configurations.

Describe the solution you'd like
Being able to define an Instrumentation in one namespace and reference it in the annotations of pods in other namespaces would allow for better centralisation of Instrumentation configuration when required.

I.e.

instrumentation.opentelemetry.io/inject-java: "tracing-components/my-instrumentation"

Describe alternatives you've considered
Alternatives to this would be using a helm chart or similar to approach to replicate a single configuration across multiple namespaces, or using another operator to replicate Instrumentation resources in a similar way to the many secret/configmap synchronisation solutions.

[ringerc]

It might make sense to do this how Istio does it for its destination rules and access policies, using an exportTo rule in the Instrumentation specification, rather than by pointing to the Instrumentation namespace in the annotations. That way the opentelemetry operator is "pushing out" its configuration to the interested parties. Combined with namespace-level instrumentation injection annotations, this makes it a lot easier to manage varying workloads.

See https://istio.io/latest/docs/reference/config/networking/destination-rule/ exportTo for example.

A list of namespaces to which this destination rule is exported. The resolution of a destination rule to apply to a service occurs in the context of a hierarchy of namespaces. Exporting a destination rule allows it to be included in the resolution hierarchy for services in other namespaces. This feature provides a mechanism for service owners and mesh administrators to control the visibility of destination rules across namespace boundaries.

Certainly the current state is difficult to manage. I land up having to untangle circular dependencies between your components and the OpenTelemetry deployment, and I have to manage a Kustomize Component with the Instrumentation definitions that lives separately to the collector so it can be referenced by all my other kustomizations.