Metricbeat logstash module should honor monitoring.cluster_uuid field in Logstash API responses

[ycombinator]

Describe the enhancement:

The Metricbeat logstash module's metricsets (x-pack data path only) must be enhanced to look for an optional monitoring.cluster_uuid field in Logstash HTTP API responses. If set, it should use this field to set the value of the cluster_uuid field in the events it generates.

Describe a specific use case for the enhancement or feature:

Sometimes a user may setup a Logstash pipeline that is not talking to Elasticsearch. When they use Metricbeat to monitoring the Logstash node running this pipeline, Metricbeat will collect the necessary monitoring data and ship it to the configured monitoring cluster (as defined by output.elasticsearch.hosts in the Metricbeat configuration).

However, this data will not contain the cluster_uuid field in it, causing the Logstash node and pipeline to appear under a "Standalone Cluster" in the Stack Monitoring UI. If the user wants the data to appear under a specific production Elasticsearch cluster in the UI, they must also set xpack.monitoring.cluster_uuid in their Logstash node's configuration (see elastic/logstash#11106). Once this is set, the Logstash HTTP APIs will start exposing a monitoring.cluster_uuid field in their responses, which the Metricbeat logstash module needs to start honoring as described above.

[elasticmachine]

Pinging @elastic/stack-monitoring (Stack monitoring)