[APM+InfraUI] Show cross-links between infra/apm

[roncohen]

After we unfortunately had to backtrack on the cross-links we should discuss how to progress on this. I've created this issue to discuss how we can make this work.

Here's the initial description of the "host" links: #23315

The rest of this issue assumes ECS alignment.

For the links to be useful, the logs, metrics and APM data needs to have the appropriate fields in their documents, for example container.id or kubernetes.pod.id.

We're working to make it easier to set things up such that fields are added and the links are useful.
For example, with elastic/apm#22 container ID will automatically be added if the agent runs in a container. With elastic/apm#21 APM agent data will have kubernetes fields as well.

However, there are plenty of situations where the links will not be useful anyway. It's not a great user experience to always have the links shown, but always have them lead to a blank-ish page.

The links in question on the APM side are:

Link	Description
"Show pod logs"	link to logs UI filtered for kubeternetes.pod.name
"Show container logs"	link to logs UI filtered for container.id
"Show host logs"	link to logs UI filtered for host.name
"Show pod metrics"	links to infra UI filtered for kubeternetes.pod.name
"Show container metrics"	links to infra UI filtered for container.id
"Show host metrics"	link to infra UI filtered for host.name

Since the "host" view in the infra UI is based on container name, rather than container id, we might need to translate the container.id to container.name either on the APM side or the infra UI side.

On the Infra/Logs side, the links are:

Link	Description
"Show pod APM traces"	link to APM UI filtered for kubeternetes.pod.name
"Show container APM traces"	link to APM UI filtered for container.id
"Show host APM traces"	link to APM UI filtered for host.name

There's also the outstanding question on whether we should link to APM errors from infra/logs ui. Might be useful.

[elasticmachine]

Pinging @elastic/apm-ui

[elasticmachine]

Pinging @elastic/infrastructure-ui

[makwarth]

However, there are plenty of situations where the links will not be useful anyway. It's not a great user experience to always have the links shown, but always have them lead to a blank-ish page.

It's definitely not ideal, but as we want this out as soon as possible, I propose updating the respective "no results"-pages with links to FAQ documentation in which we can elaborate on how the integrations work / can be configured. This is something we need to update across solutions.

On the Infra/Logs side, the links are:

In the logging UI, we should make certain configurable fields linkable as well. We'd need to add trace.id as a configurable field. #20428 (comment). Clicking the trace id would link to the APM UI filtered for trace.id

[exekias]

a note on kubeternetes.pod.name, this may not be globally unique, as you may have Pods sharing the same name on different namespaces. Would it be possible to use kubernetes.namespace + kubernetes.pod.name?

We recently added Pod UID (it is a UUID) as an optional field in metadata, we could enable it by default and use it.

[roncohen]

@exekias OK. I guess that technically goes for container.name as well?

[roncohen]

Should we consider switching to a model where we link/filter using pod.uid/container.id and then the UI looks up the corresponding names to show them, instead of using names to link/filter?

[exekias]

That sounds good to me, there is no need to hide this logic from the user, but names should appear in the target views

[weltenwort]

On a related note, a user correctly points out that the container/pod name ambiguity can also lead to problems within the waffle map itself: #25939

[roncohen]

@weltenwort OK. Sounds like a plan. Will you coordinate the switch across logs/waffle/hostview? Do we need a separate issue with checkboxes?

[weltenwort]

A separate issue would be helpful, since I'm a bit swamped with other planning and implementation tasks right now.

[roncohen]

Filed here: #26359

[makwarth]

APM implementation issue: #26574
Infra UI implementation issue: #26620

Feedback welcome on the respective issues