Cherry-pick #9856 to 6.x: Add docker event metricset (#9962)

* Add docker `event` metricset (#9856)

* Add docker `event` metricset

This metricset will retrieve events coming from the Docker events
API [0].

Example output:

```
"docker": {
  "event": {
    "id": "8c229155b039c2adcb4fab1f987f35a0d1f913dfaa95f3113ed6e4f91eb5398c",
    "from": "busybox",
    "type": "container",
    "action": "die",
    "actor": {
      "id": "8c229155b039c2adcb4fab1f987f35a0d1f913dfaa95f3113ed6e4f91eb5398c",
      "attributes": {
        "image": "busybox",
        "name": "distracted_lichterman",
        "exitCode": "0"
      }
    },
    "time": "2019-01-02T22:41:02.000Z",
    "status": "die"
  }
}
```

Actor attributes will container labels in the case of container
events, the module will perform dedotting (if enabled) on them.

[0] https://docs.docker.com/engine/api/v1.37/#operation/SystemEvents

* Add tests

* Update changelog

* Add missing header

* Add missing file

* Adapt to ECS

* Use a cheaper event for tests

* Close docker client on module shutdown

* Update data

* clean created container

* fix image pull

(cherry picked from commit eef102d)

* Disable migration in docker event fields (#9947)

* Move back docker event fields to it's namespace (#10073)

* Move back docker event fields to it's namespace

CHANGELOG.next.asciidoc

@@ -165,6 +165,7 @@ https://github.com/elastic/beats/compare/v6.6.0...6.x[Check the HEAD diff]
 - Remove experimental tag from ceph metricsets. {pull}9708[9708]
 - Add `key` metricset to the Redis module. {issue}9582[9582] {pull}9657[9657]
 - Add DeDot for kubernetes labels and annotations. {issue}9860[9860] {pull}9939[9939]
+- Add docker `event` metricset. {pull}9856[9856]
 - Release Ceph module as GA. {pull}10202[10202]
 - Release windows Metricbeat module as GA. {pull}10163[10163]
 - Release traefik Metricbeat module as GA. {pull}10166[10166]

metricbeat/docs/fields.asciidoc

@@ -2996,6 +2996,90 @@ type: scaled_float
 Number of reads and writes per second
 
 
+--
+
+[float]
+== event fields
+
+Docker event
+
+
+
+*`docker.event.status`*::
++
+--
+type: keyword
+
+Event status
+
+
+--
+
+*`docker.event.id`*::
++
+--
+type: keyword
+
+Event id when available
+
+
+--
+
+*`docker.event.from`*::
++
+--
+type: keyword
+
+Event source
+
+
+--
+
+*`docker.event.type`*::
++
+--
+type: keyword
+
+The type of object emitting the event
+
+
+--
+
+*`docker.event.action`*::
++
+--
+type: keyword
+
+The type of event
+
+
+--
+
+[float]
+== actor fields
+
+Actor
+
+
+
+*`docker.event.actor.id`*::
++
+--
+type: keyword
+
+The ID of the object emitting the event
+
+
+--
+
+*`docker.event.actor.attributes`*::
++
+--
+type: object
+
+Various key/value attributes of the object, depending on its type
+
+
 --
 
 [float]

metricbeat/docs/modules/docker.asciidoc

@@ -38,6 +38,7 @@ metricbeat.modules:
     - "container"
     - "cpu"
     - "diskio"
+    - "event"
     - "healthcheck"
     - "info"
     #- "image"
@@ -71,6 +72,8 @@ The following metricsets are available:
 
 * <<metricbeat-metricset-docker-diskio,diskio>>
 
+* <<metricbeat-metricset-docker-event,event>>
+
 * <<metricbeat-metricset-docker-healthcheck,healthcheck>>
 
 * <<metricbeat-metricset-docker-image,image>>
@@ -87,6 +90,8 @@ include::docker/cpu.asciidoc[]
 
 include::docker/diskio.asciidoc[]
 
+include::docker/event.asciidoc[]
+
 include::docker/healthcheck.asciidoc[]
 
 include::docker/image.asciidoc[]

metricbeat/docs/modules/docker/event.asciidoc

@@ -0,0 +1,21 @@
+////
+This file is generated! See scripts/docs_collector.py
+////
+
+[[metricbeat-metricset-docker-event]]
+=== Docker event metricset
+
+include::../../../module/docker/event/_meta/docs.asciidoc[]
+
+
+==== Fields
+
+For a description of each field in the metricset, see the
+<<exported-fields-docker,exported fields>> section.
+
+Here is an example document generated by this metricset:
+
+[source,json]
+----
+include::../../../module/docker/event/_meta/data.json[]
+----

metricbeat/docs/modules_list.asciidoc

@@ -22,9 +22,10 @@ This file is generated! See scripts/docs_collector.py
 |<<metricbeat-metricset-couchbase-cluster,cluster>>   
 |<<metricbeat-metricset-couchbase-node,node>>   
 |<<metricbeat-module-docker,Docker>>     |image:./images/icon-yes.png[Prebuilt dashboards are available]    |  
-.8+| .8+|  |<<metricbeat-metricset-docker-container,container>>   
+.9+| .9+|  |<<metricbeat-metricset-docker-container,container>>   
 |<<metricbeat-metricset-docker-cpu,cpu>>   
 |<<metricbeat-metricset-docker-diskio,diskio>>   
+|<<metricbeat-metricset-docker-event,event>>   
 |<<metricbeat-metricset-docker-healthcheck,healthcheck>>   
 |<<metricbeat-metricset-docker-image,image>>   
 |<<metricbeat-metricset-docker-info,info>>   

metricbeat/metricbeat.reference.yml

@@ -167,6 +167,7 @@ metricbeat.modules:
     - "container"
     - "cpu"
     - "diskio"
+    - "event"
     - "healthcheck"
     - "info"
     #- "image"

metricbeat/module/docker/_meta/config.reference.yml

@@ -3,6 +3,7 @@
     - "container"
     - "cpu"
     - "diskio"
+    - "event"
     - "healthcheck"
     - "info"
     #- "image"

metricbeat/module/docker/_meta/config.yml

@@ -3,6 +3,7 @@
   #  - container
   #  - cpu
   #  - diskio
+  #  - event
   #  - healthcheck
   #  - info
   #  - memory

metricbeat/module/docker/event/_meta/data.json

@@ -0,0 +1,29 @@
+{
+    "@timestamp": "2017-10-12T08:05:34.853Z",
+    "agent": {
+        "hostname": "host.example.com",
+        "name": "host.example.com"
+    },
+    "docker": {
+        "event": {
+            "action": "pull",
+            "actor": {
+                "attributes": {
+                    "name": "busybox"
+                },
+                "id": "busybox:latest"
+            },
+            "from": "",
+            "id": "busybox:latest",
+            "status": "pull",
+            "type": "image"
+        }
+    },
+    "event": {
+        "dataset": "event",
+        "module": "docker"
+    },
+    "service": {
+        "type": "docker"
+    }
+}

metricbeat/module/docker/event/_meta/docs.asciidoc

@@ -0,0 +1 @@
+This is the event metricset of the module docker.

metricbeat/module/docker/event/_meta/fields.yml

@@ -0,0 +1,40 @@
+- name: event
+  type: group
+  description: >
+    Docker event
+  release: ga
+  fields:
+    - name: status
+      type: keyword
+      description: >
+        Event status
+    - name: id
+      type: keyword
+      description: >
+        Event id when available
+    - name: from
+      type: keyword
+      description: >
+        Event source
+    - name: type
+      type: keyword
+      description: >
+        The type of object emitting the event
+    - name: action
+      type: keyword
+      description: >
+        The type of event
+    - name: actor
+      type: group
+      description: >
+        Actor
+      fields:
+        - name: id
+          type: keyword
+          description: >
+            The ID of the object emitting the event
+        - name: attributes
+          type: object
+          object_type: keyword
+          description: >
+            Various key/value attributes of the object, depending on its type