[receiver/k8scluster] Remove usage of 'component.Host.GetExporters'

[djaglowski]

Component(s)

receiver/k8scluster

Describe the issue you're reporting

See open-telemetry/opentelemetry-collector#7370 for additional context.

This receiver depends on an experimental function that may be removed in a future version. It uses this function to get a handle to a "metadata exporter". It then generates a special kind of data and sends it to this exporter.

The metadata is reported in an event driven style, where only changes are reported from this receiver to the metadata exporter.

I propose that this is not an appropriate way to pass non-standard data between components. Instead, an extension should be developed that can facilitate a publisher/subscriber model (or similar). The exporter, which currently must implement a special interface, can instead check for this extension and listen for updates. The receiver, which currently asks for a handle to the exporter and pushes updates, can instead check for the extension and push there instead.

Todo

We will execute according to design document here: https://docs.google.com/document/d/1Tg18sIck3Nakxtd3TFFcIjrmRO_0GLMdHXylVqBQmJA/edit

• Add data structures to model entity events as log records. #23565
• Implement conversion from Metadata to the new model. #24394
• Begin emitting entity events (as logs) from k8scluster receiver #24400
• [receiver/k8sclusterreceiver] Report entity state periodically to be immune to event losses and reordering #24413
• [receiver/k8sclusterreceiver] Add timestamp field to entity events #24428
• Watch for entity events in signalfx exporter, compare to direct metadata and surface any differences. Make configurable which source to use for the events: the MetadataUpdate or entity events. #27890
• Test to ensure no differences are found and there is no measurable performance degradation.
• Switch to using entity events only via user config option.
• Delete code that emits metadata in k8scluster and listens for it signalfx exporter.
• Deprecate user config option and make entity events via logs the only way.

[github-actions]

Pinging code owners:

• receiver/k8scluster: @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[dmitryax]

I propose that this is not an appropriate way to pass non-standard data between components. Instead, an extension should be developed that can facilitate a publisher/subscriber model (or similar). The exporter, which currently must implement a special interface, can instead check for this extension and listen for updates. The receiver, which currently asks for a handle to the exporter and pushes updates, can instead check for the extension and push there instead.

Thanks, @djaglowski. The suggestion makes sense to me. This functionality is important for Splunk. I'll try to prioritize it in my team.

[dmitryax]

I discussed this with @tigrannajaryan offline, and he reminded me about a document he wrote specifically for this use case. https://docs.google.com/document/d/1Tg18sIck3Nakxtd3TFFcIjrmRO_0GLMdHXylVqBQmJA/edit#heading=h.hxs8bzpumhfw I think we should migrate metadata updates to the entity data model and add logs pipeline support to k8s cluster receiver.

[djaglowski]

Looks like a well thought out solution. I don't have much of an opinion on which approach we take, aside from suggesting we should remove GetExporters.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• receiver/k8scluster: @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[tigrannajaryan]

I updated the todo list. Anyone planning to work on any item please convert it to an issue and assign to yourself.