[Stack Monitoring] Kibana log showing field [json] doesn't exist in error.message field

[crespocarlos]

Summary

A change was introduced to Kibana log ingest pipeline, to fix a problem in which the pipeline was not considering fields included in the module configuration.

After this change, if a user sets json.keys_under_root is true in the module configuration, it will make the pipeline fail.

 - module: kibana
    log:
      enabled: true
      var.paths:
        - PATH_TO/kibana-json*.log
      input:
        json.keys_under_root: true -> breaks it

Details

This happens because:

• If json.keys_under_root: true is present in the module configuration, it will move all attributes present in message to the object root
• By the time filebeat tries to put all attributes from message into an object named json, message will already be empty
• When filebeat makes the decision of which pipeline to run, json is empty. so it will default to pipeline-7 (causing ECS formatted logs to fail)

Side note: even with the error.message the error mentioned, Logs UI will look OK, because the log ingested is ECS formatted.

Acceptance criteria

• The pipeline should work, even with json.keys_under_root: true in the module configuration.

[fludo]

Do you have any progress regarding this issue ?
We are waiting for it on a case in elastic cloud: https://support.elastic.co/cases/5008X00002LWDXZQA5

Regards

[crespocarlos]

@fludo a PR has been opened to address this issue.