[Meta] SIEM Alerts view

[tsg]

We'd like to add an easy way for people to see 3rd party Alerts in the SIEM app. Common source of alerts are:

• Elastic Endpoint protection (the Endgame platform in Phase 0)
• Suricata
• Zeek
• Firewall devices, other IDS

This is currently partially possible by adding event.kind: alert in the Filter bar and seeing the Events table in Hosts overview page:

But we want to make this more straight forward. The proposal is:

• Add a new Alerts tab on the Hosts overview, Host details and Network Overview pages. Very likely, once we create an Overview page, we should also have alerts showing up there.
• This tab will have a date histogram by event.module (this will give zeek vs suricata vs endgame
• The table will have columns for @timestamp,event.module, event.dataset, event.category, event.severity, observer.name, host.name, message.
• The table should have configurable columns. We can use the Timeline rendering, defaulting the row renderers to off.
• On the Hosts and Host details pages, the table and the histogram should be filtered by _exists_: host.name. On the Network Overview page, the filter should be on _exists_: source or _exists_: destination. On the Overview page, there should be no filter.

Stretch goal

Probably not available in the first implementation, but we'd like to also have a way to link to the 3rd party system where the Alert initiated. This is especially important for the Endpoint product since we want the experience to feel perfectly integrated. The reason this is a stretch goal is that we don't currently have an established mechanism to link back to the Endgame platform, and there are no standard ECS fields for that.

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)

[angorayc]

Created a similar table on discover, and found some questions about the fields we ned:
https://siem-dev-kibana.app.elstc.co/goto/ff0bb3d076c3db4a56f160541426ff6b

1. Is there any where I can find the possible for fields?
2. event.severity seems to be a number, what's the range of it? would we want to map it?
3. Cannot find observer.name
4. Would we want to group by event.module when displaying it?
5. Which fields are sortable?

[angorayc]

Had a chat with @tsg, and summary some discussion here:

Created a similar table on discover, and found some questions about the fields we ned:

https://siem-dev-kibana.app.elstc.co/goto/ff0bb3d076c3db4a56f160541426ff6b

1. Is there any where I can find the possible for fields?

https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html

2. event.severity seems to be a number, what's the range of it? would we want to map it?

Range: 0-7, no mapping at the moment.

3. Cannot find observer.name

This field is not available yet.

4. Would we want to group by event.module when displaying it?

No, no grouping.

5. Which fields are sortable?

All the field beside message.

[angorayc]

Query for histogram:

{
  "aggs": {
    "4": {
      "date_histogram": {
        "field": "@timestamp",
        "fixed_interval": "3h",
        "time_zone": "Europe/London"
      },
      "aggs": {
        "5": {
          "terms": {
            "field": "event.module",
            "order": {
              "_count": "desc"
            },
            "size": 11
          }
        }
      }
    }
  },
  "size": 0,
  "stored_fields": [
    "*"
  ],
  "script_fields": {},
  "_source": {
    "excludes": []
  },
  "query": {
    "bool": {
      "must": [],
      "filter": [
        {
          "bool": {
            "should": [
              {
                "match": {
                  "event.kind": "alert"
                }
              }
            ],
            "minimum_should_match": 1
          }
        },
        {
          "range": {
            "@timestamp": {
              "gte": "2019-12-01T00:00:00.000Z",
              "lte": "2019-12-07T23:59:59.999Z",
              "format": "strict_date_optional_time"
            }
          }
        }
      ],
      "should": [],
      "must_not": []
    }
  }
}