[APM] InfraOps cannot display hosts linked from APM UI

[formgeist]

Created a new ticket, since the PR was closed.

When clicking the InfraOps links in the ActionsMenu, you're taken to an error state in InfraOps, saying that it cannot locate the host.

I don't know if it's related, but the links are using context.system.host.name rather than host.name which is added to the transaction document. host.name uses the same ID as beat.hostname which I assume would show metrics.

cc @makwarth

[elasticmachine]

Pinging @elastic/apm-ui

[roncohen]

For context, context.system.hostname is the hostname of the host where the agent is running. host.name is the hostname of where the APM Server is running. We want to link to the metrics for where the agent is running.

[makwarth]

Clicking the InfraOps links from the ActionsMenu aren't working.

Can you elaborate?

…

On Wed, 24 Oct 2018 at 12.11 PM, Ron Cohen ***@***.***> wrote: For context, context.system.hostname is the hostname of the host where the agent is running. host.name is the hostname of where the APM Server is running. We want to link to the metrics for where the agent is running. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#24481 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAzYVfC4nwT0-HSTRTp0YM2c1tsa8XNuks5uoDzJgaJpZM4X3kjd> .

[jasonrhodes]

Do we expect Opbeans test data to work with the infra metrics and logs links? When I tried to test them yesterday, the links ended up in the infra UI but didn't find anything, and I assumed it was because of the test data.

[sorenlouv]

For InfraOps to work you need to setup Logstash or filebeat to send the logs to ES.

[formgeist]

@makwarth Updated the description with a screenshot from InfraOps.

@jasonrhodes I'm running the setup locally with metricbeat and filebeat, but not sure if we can count on the agent test data to carry the appropiate host names. Never seen it work besides when browsing InfraOps metrics and logs where it does show data.

[weltenwort]

Would it be possible to get a sample of the links that are being generated in this case?

[jasonrhodes]

@weltenwort sure:

Metrics link: http://localhost:5601/app/infra#/link-to/host-detail/2bf5b40123ef?from=1540346055945&to=1540346655945&_g=()&_a=

Logs link: http://localhost:5601/app/infra#/link-to/host-logs/2bf5b40123ef?time=1540346355945&_g=()&_a=

[weltenwort]

Based on these links, the InfraOps and logging UI filter the data such that only documents with beats.hostname === 2bf5b40123ef (to be changed to host.name after the migration to ECS) are taken into account.

The hostname looks like a hostname inside a docker container. If the corresponding metricbeat and filebeat processes do not run in the same container they will annotate their documents with a different beat.hostname (i.e. the host/container they are running in).

[roncohen]

I also took a look and I agree with what @weltenwort said: When APM agents run inside containers, their hostname becomes the first part of the container id. That's the only hostname we have at the moment. Filebeat, on the other hand, will report the container id of the container it's running in as it's hostname. For these links to actually work, both would need to instead supply the hostname of the parent host. AFAICT getting the hostname of the parent host from inside a container is not simple.

If this was a situation with no docker involved, this would probably work out of the box. With things running in Docker, it doesn't work and as it is currently, it's not a great user experience.

We could either:

1. remove the link for this version and work on something better
2. disable the button if there is no matching documents
3. leave it as is

I think i would vote for (1) now and (2) in the next iteration.

[roncohen]

@weltenwort @simianhacker what do you think about changing the Infraui GraphQL error message to something more user friendly?

[weltenwort]

what do you think about changing the Infraui GraphQL error message to something more user friendly?

That certainly can't hurt 👍 -> #24493

[alvarolobato]

@weltenwort , @simianhacker the links in the other direction are also not working.
When clicking "View APM Traces" in InfraOps the services list is always empty.

• If you are in a host view it will generate a link to beat.hostname : "apm-ui-dev" but it should be context.system.hostname : "apm-ui-dev"
• If you are in a container, the link generated filtering on docker.container.name : "localtesting_7.0.0-alpha1_opbeans-node" but it should be context.system.hostname : "94f3902390c3" where 94f3902390c3

Is it possible to change these links?, the second one would probably be temporary until we land https://github.com/elastic/apm-dev/issues/263

[formgeist]

Due to limited resources and we're pushing hard for other issues to be fixed in time for release, we're opting to remove the InfraOps links from APM and pushing it to next release. I've made a ticket in APM UI which describes the change #24562

It also looks like the opposite way (going from InfraOps -> APM) link doesn't work either when APM runs in containers, which means the services list will eventually be empty and it's not helpful for users. We're suggesting that the link from Metrics (in the context menu for hosts and containers) should be removed until next release. By then we should also have our routing in place so you can link to Traces, and not Services, as the link in the UI describes.

Example of a link from Host metrics -> APM services from a host;

Thoughts?

[formgeist]

@alvarolobato and I had the same idea at the same time. Mine is a little more visual though... 😉

[roncohen]

We decided to remove all the cross links at this time and instead shoot for a great UX in the next iteration: #24562