Monitoring Filebeat fails to send data to ES

[belimawr]

Under V2 the monitoring Filebeat is failing with Failed to publish event: key not found.

{"log.level":"error","@timestamp":"2022-12-21T13:37:56.206+0100","message":"Failed to publish event: key not found","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"filestream-monitoring","type":"filestream"},"log.logger":"publisher","log.origin":{"file.line":102,"file.name":"pipeline/client.go"},"service.name":"filebeat","ecs.version":"1.6.0","ecs.version":"1.6.0"}

[cmacknz]

Blake has tracked the root cause of this problem to elastic/beats@6c13c67#diff-1469f81c814512893df63d886bc7940ed7f8d941f198193691647c7694281ecaR232

The order of the processors in Beats has changed after that commit causing this problem.

[cmacknz]

Blake is actively working to fix this, updating the assignee to reflect that.

[cmacknz]

This is a duplicate of elastic/kibana#146656, we can rerun that test once this issue is resolved.

[blakerouse]

This is not an issue with Elastic Agent, but an issue caused by a change in beats. elastic/beats#34093

Closing this issue as beats needs to be fixed, not the Elastic Agent.