The elastic-agent status command should show unit states by default

[cmacknz]

The output of the elastic-agent status command from 8.6 onward only shows component states, which can be confusing because the overall agent status takes the unit states into account. We should include the unit states in the elastic-agent status command to allow easily debugging the state of an agent from the command line.

Example output where all components are shown as healthy, but the overall agent state is DEGRADED because of a failing unit whose state is hidden:

> elastic-agent status
State: DEGRADED
Message: 1 or more components/units in a failed state
Components:
* endpoint (HEALTHY)
Healthy: communicating with endpoint service
* log (HEALTHY)
Healthy: communicating with pid '4088'
* winlog (HEALTHY)
Healthy: communicating with pid '28196'
* system/metrics (HEALTHY)
Healthy: communicating with pid '22852'
* filestream (HEALTHY)
Healthy: communicating with pid '14908'
* beat/metrics (HEALTHY)
Healthy: communicating with pid '13832'
* http/metrics (HEALTHY)
Healthy: communicating with pid '20348'

We should additionally update the official documentation to explain what this output means if we are going to use the terms "component" and "unit" directly in the output: https://www.elastic.co/guide/en/fleet/current/elastic-agent-cmd-options.html#elastic-agent-status-command

Source for the docs is in https://github.com/elastic/observability-docs/tree/main/docs/en/ingest-management

Note: Running elastic-agent status --output=json or elastic-agent status --output=yaml will show the unit states. We should not hide this information by default because it is impossible to debug problems without it.

[Liblor]

What does it mean to have a failed unit? Or in other words what's the difference between a unit and a component? Because I have plenty of agents report as unhealthy in fleet, but all componenets are shown as healthy, as described in this issue.

[cmacknz]

We also need a documentation update to go with this change to explain this. The internals of the agent changed significantly in 8.6.

A component is a process, like Filebeat or Endpoint security. A unit is like a configurable plugin that process can run. For Beats this maps to Filebeat inputs or Metricbeat modules for example. For Beats the output (like Elasticsearch) also gets its own unit with a separate status.

Generally these "units" map directly back to the individual inputs in an agent policy, and can then be associated back to specific integrations.

Right now we are only surfacing the component or process state.

[cmacknz]

We should also show the agent ID, version, and commit in the default elastic-agent status output. This is included in the yaml and json outputs and we occasionally get questions about how to obtain this information (particularly the agent ID), so let's just make this information obvious instead of hiding it in the other output formats.

sudo elastic-agent status --output yaml
info:
  id: caf01044-0531-4fe8-b3ef-d8218fc48624
  version: 8.7.0
  commit: dab5a58adeb0d22670fe996ff266eb2a9f96ce28
  build_time: 2023-01-23 14:41:09 +0000 UTC
  snapshot: true

We should show the id, version, and commit in the elastic-agent status output. We could show all of it, but I'm not sure how valuable the build_time and snapshot attributes if it is easier but this will likely just clutter the output as they aren't that valuable.

[blakerouse]

I would like to see when this is fixed that we output a nice clean table for the the status command output.

https://github.com/jedib0t/go-pretty or something similar

[belimawr]

It would also be great to have the status command as well as the JSON/YAML outputs sorted.

I don't know if it's just me, but I often watch the status command while developing/testing (something like watch -n 0.5 elastic-agent status) and having some lines moving around gets very confusing some times.

[cmacknz]

Related and should also be addressed as part of these changes: #2829

[cmacknz]

Reopening this until we update the docs. The status command output appears in a few places we need to update: https://github.com/search?q=repo%3Aelastic%2Fingest-docs%20elastic-agent%20status&type=code

[leehinman]

Reopening this until we update the docs. The status command output appears in a few places we need to update: https://github.com/search?q=repo%3Aelastic%2Fingest-docs%20elastic-agent%20status&type=code

elastic/ingest-docs#265 is PR for doc changes.

[jlind23]

Closing this as elastic/ingest-docs#265 has been merged.