Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docker event metricset #9856

Merged
merged 11 commits into from
Jan 4, 2019
Merged

Add docker event metricset #9856

merged 11 commits into from
Jan 4, 2019

Conversation

exekias
Copy link
Contributor

@exekias exekias commented Jan 2, 2019

This metricset will retrieve events coming from the Docker events
API [0].

Example output:

"docker": {
  "event": {
    "id": "8c229155b039c2adcb4fab1f987f35a0d1f913dfaa95f3113ed6e4f91eb5398c",
    "from": "busybox",
    "type": "container",
    "action": "die",
    "actor": {
      "id": "8c229155b039c2adcb4fab1f987f35a0d1f913dfaa95f3113ed6e4f91eb5398c",
      "attributes": {
        "image": "busybox",
        "name": "distracted_lichterman",
        "exitCode": "0"
      }
    },
    "time": "2019-01-02T22:41:02.000Z",
    "status": "die"
  }
}

Actor attributes will contain labels in the case of container
events, the module will perform dedotting (if enabled) on them.

[0] https://docs.docker.com/engine/api/v1.37/#operation/SystemEvents

closes #6799

Carlos Pérez-Aradros Herce added 2 commits January 2, 2019 23:44
This metricset will retrieve events coming from the Docker events
API [0].

Example output:

```
"docker": {
  "event": {
    "id": "8c229155b039c2adcb4fab1f987f35a0d1f913dfaa95f3113ed6e4f91eb5398c",
    "from": "busybox",
    "type": "container",
    "action": "die",
    "actor": {
      "id": "8c229155b039c2adcb4fab1f987f35a0d1f913dfaa95f3113ed6e4f91eb5398c",
      "attributes": {
        "image": "busybox",
        "name": "distracted_lichterman",
        "exitCode": "0"
      }
    },
    "time": "2019-01-02T22:41:02.000Z",
    "status": "die"
  }
}
```

Actor attributes will container labels in the case of container
events, the module will perform dedotting (if enabled) on them.

[0] https://docs.docker.com/engine/api/v1.37/#operation/SystemEvents
@exekias exekias added enhancement review Metricbeat Metricbeat containers Related to containers use case Team:Integrations Label for the Integrations team labels Jan 2, 2019
@exekias exekias requested a review from a team as a code owner January 2, 2019 23:30
@urso urso removed the request for review from a team January 3, 2019 00:12
Copy link
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the schema here setup to follow the format of the existing metricset’s in this module? Based only on a brief look I think there are some ECS fields that could be populated either in addition to or as an alternative.

func (m *MetricSet) reportEvent(reporter mb.PushReporterV2, event events.Message) {
time := time.Unix(event.Time, 0)

attributes := common.MapStr{}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could allocate a map based on len(event.Actor.Attributes).

Copy link
Member

@jsoriano jsoriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good, just a couple of comments.

metricbeat/module/docker/event/event.go Show resolved Hide resolved
metricbeat/module/docker/event/event_integration_test.go Outdated Show resolved Hide resolved
metricbeat/module/docker/event/event.go Show resolved Hide resolved
@exekias exekias requested review from a team as code owners January 3, 2019 13:49
@exekias exekias added needs_backport PR is waiting to be backported to other branches. v6.7.0 labels Jan 3, 2019
@exekias exekias merged commit eef102d into elastic:master Jan 4, 2019
migration: true
description: >
The type of object emitting the event
- name: action
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You set migration: true here. Is this an alias we only want to be enabled if migration is enabled or have it always? Same for the other aliases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want it to stay, I'm guessing I should not have put migration there 😇 ? I can open a PR to remove them 👍

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, otherwise they will disappear.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@exekias BTW this is a more general discussion if we want to have fields in ECS only or also the "original" name around.

@exekias exekias self-assigned this Jan 7, 2019
@exekias exekias removed the needs_backport PR is waiting to be backported to other branches. label Jan 9, 2019
exekias added a commit to exekias/beats that referenced this pull request Jan 18, 2019
* Add docker `event` metricset

This metricset will retrieve events coming from the Docker events
API [0].

Example output:

```
"docker": {
  "event": {
    "id": "8c229155b039c2adcb4fab1f987f35a0d1f913dfaa95f3113ed6e4f91eb5398c",
    "from": "busybox",
    "type": "container",
    "action": "die",
    "actor": {
      "id": "8c229155b039c2adcb4fab1f987f35a0d1f913dfaa95f3113ed6e4f91eb5398c",
      "attributes": {
        "image": "busybox",
        "name": "distracted_lichterman",
        "exitCode": "0"
      }
    },
    "time": "2019-01-02T22:41:02.000Z",
    "status": "die"
  }
}
```

Actor attributes will container labels in the case of container
events, the module will perform dedotting (if enabled) on them.

[0] https://docs.docker.com/engine/api/v1.37/#operation/SystemEvents

* Add tests

* Update changelog

* Add missing header

* Add missing file

* Adapt to ECS

* Use a cheaper event for tests

* Close docker client on module shutdown

* Update data

* clean created container

* fix image pull

(cherry picked from commit eef102d)
exekias added a commit that referenced this pull request Jan 22, 2019
* Add docker `event` metricset (#9856)

* Add docker `event` metricset

This metricset will retrieve events coming from the Docker events
API [0].

Example output:

```
"docker": {
  "event": {
    "id": "8c229155b039c2adcb4fab1f987f35a0d1f913dfaa95f3113ed6e4f91eb5398c",
    "from": "busybox",
    "type": "container",
    "action": "die",
    "actor": {
      "id": "8c229155b039c2adcb4fab1f987f35a0d1f913dfaa95f3113ed6e4f91eb5398c",
      "attributes": {
        "image": "busybox",
        "name": "distracted_lichterman",
        "exitCode": "0"
      }
    },
    "time": "2019-01-02T22:41:02.000Z",
    "status": "die"
  }
}
```

Actor attributes will container labels in the case of container
events, the module will perform dedotting (if enabled) on them.

[0] https://docs.docker.com/engine/api/v1.37/#operation/SystemEvents

* Add tests

* Update changelog

* Add missing header

* Add missing file

* Adapt to ECS

* Use a cheaper event for tests

* Close docker client on module shutdown

* Update data

* clean created container

* fix image pull

(cherry picked from commit eef102d)

* Disable migration in docker event fields (#9947)

* Move back docker event fields to it's namespace (#10073)

* Move back docker event fields to it's namespace
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
containers Related to containers use case enhancement Metricbeat Metricbeat review Team:Integrations Label for the Integrations team v6.7.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Auditbeat] Add support for reporting Docker events
4 participants