[FEATURE REQUEST] Enable multiline support for logs statements in Filebeat

[tim-gates]

Is your feature request related to a problem? Please describe.
For DE services we would like to be able to have multiline ingestion of log statements into Elasticsearch 'out of the box'.
Currently, the Docker Filebeat configuration is not set up to specify a multiline properties. We've worked around this by
manually setting the following configuration in the filebeat.yml for each K8s node.

Describe the solution you'd like
For each K8s node we did the following.

SSH into the node (See the Environments page for instructions and IP addresses)

Edit the Filebeat configuration file to add the appropriate filebeat configuration:
sudo vim /etc/filebeat/filebeat.yml
Add the following bold lines in the location shown below. Be careful of indentation (this is a YAML file)

- type: docker
   enabled: true
   containers.ids: "*"
   multiline.pattern: '([12]\d{3}[- \/.](0[1-9]|1[0-2])[- \/.](0[1-9]|[12]\d|3[01]))|(0[1-9]|1[012])[- \/.](0[1-9]|[12][0-9]|3[01])[- \/.]\d\d'
   multiline.negate: true
   multiline.match: after
   processors:

Restart filebeat:
sudo service filebeat restart

Describe alternatives you've considered
Nothing to add.

Additional context
Nothing to add.

[erzetpe]

Removed link to internal company documentation.

[erzetpe]

@tim-gates: What do you think about making such options configurable? Then you would specify your own options in settings according to filebeat documentation? https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html

We can also setup such options for other components.

[gj25]

@erzetpe I think that is OK, as long as it can be done during the Epiphany install/config.

[erzetpe]

Yes, this is our intent. Thank you for the information.

[przemyslavic]

There is a bug to be fixed: [BUG] Filebeat multiline support - truncated single quotes in the regular expression pattern

[to-bar]

Test OK after fixing #1669.