From f2c231996d6810a56fbac847af002857ff2e84b5 Mon Sep 17 00:00:00 2001 From: Eric Beahan Date: Wed, 4 Aug 2021 12:12:49 -0500 Subject: [PATCH] New event categorization values to support threat intel use cases (#1510) (#1553) * introduce event.kind:enrichment, event.category:threat, and event.type:indicator * update docs and artifacts * Drop mention of "cybersecurity threats" Co-authored-by: Andrew Pease <7442091+peasead@users.noreply.github.com> * artifacts * alphabetize Co-authored-by: Andrew Pease <7442091+peasead@users.noreply.github.com> Co-authored-by: Andrew Pease <7442091+peasead@users.noreply.github.com> --- docs/field-details.asciidoc | 6 ++-- docs/field-values.asciidoc | 35 +++++++++++++++++++++++ experimental/generated/ecs/ecs_flat.yml | 17 +++++++++++ experimental/generated/ecs/ecs_nested.yml | 17 +++++++++++ generated/ecs/ecs_flat.yml | 17 +++++++++++ generated/ecs/ecs_nested.yml | 17 +++++++++++ schemas/event.yml | 22 +++++++++++++- 7 files changed, 127 insertions(+), 4 deletions(-) diff --git a/docs/field-details.asciidoc b/docs/field-details.asciidoc index 9702b8fe77..90c2ae7bb3 100644 --- a/docs/field-details.asciidoc +++ b/docs/field-details.asciidoc @@ -2584,7 +2584,7 @@ Note: this field should contain an array of values. *Important*: The field value must be one of the following: -authentication, configuration, database, driver, file, host, iam, intrusion_detection, malware, network, package, process, registry, session, web +authentication, configuration, database, driver, file, host, iam, intrusion_detection, malware, network, package, process, registry, session, threat, web To learn more about when to use which value, visit the page <> @@ -2757,7 +2757,7 @@ type: keyword *Important*: The field value must be one of the following: -alert, event, metric, state, pipeline_error, signal +alert, enrichment, event, metric, state, pipeline_error, signal To learn more about when to use which value, visit the page <> @@ -3014,7 +3014,7 @@ Note: this field should contain an array of values. *Important*: The field value must be one of the following: -access, admin, allowed, change, connection, creation, deletion, denied, end, error, group, info, installation, protocol, start, user +access, admin, allowed, change, connection, creation, deletion, denied, end, error, group, indicator, info, installation, protocol, start, user To learn more about when to use which value, visit the page <> diff --git a/docs/field-values.asciidoc b/docs/field-values.asciidoc index e655ba6c26..883b3b1ec0 100644 --- a/docs/field-values.asciidoc +++ b/docs/field-values.asciidoc @@ -41,6 +41,7 @@ The value of this field can be used to inform how these kinds of events should b *Allowed Values* * <> +* <> * <> * <> * <> @@ -59,6 +60,16 @@ This value is not used by Elastic solutions for alert documents that are created +[float] +[[ecs-event-kind-enrichment]] +==== enrichment + +The `enrichment` value indicates an event collected to provide additional context, often to other events. + +An example is collecting indicators of compromise (IOCs) from a threat intelligence provider with the intent to use those values to enrich other events. The IOC events from the intelligence provider should be categorized as `event.kind:enrichment`. + + + [float] [[ecs-event-kind-event]] ==== event @@ -136,6 +147,7 @@ This field is an array. This will allow proper categorization of some events tha * <> * <> * <> +* <> * <> [float] @@ -314,6 +326,18 @@ The session category is applied to events and metrics regarding logical persiste start, end, info +[float] +[[ecs-event-category-threat]] +==== threat + +Use this category to visualize and analyze events describing threat actors' targets, motives, or behaviors. + + +*Expected event types for category threat:* + +indicator + + [float] [[ecs-event-category-web]] ==== web @@ -348,6 +372,7 @@ This field is an array. This will allow proper categorization of some events tha * <> * <> * <> +* <> * <> * <> * <> @@ -442,6 +467,16 @@ The group event type is used for the subset of events within a category that are +[float] +[[ecs-event-type-indicator]] +==== indicator + +The indicator event type is used for the subset of events within a category that contain details about indicators of compromise (IOCs). + +A common example is `event.category:threat AND event.type:indicator`. + + + [float] [[ecs-event-type-info]] ==== info diff --git a/experimental/generated/ecs/ecs_flat.yml b/experimental/generated/ecs/ecs_flat.yml index e8b38881ef..bd9f39aafa 100644 --- a/experimental/generated/ecs/ecs_flat.yml +++ b/experimental/generated/ecs/ecs_flat.yml @@ -2495,6 +2495,11 @@ event.category: - end - info name: session + - description: Use this category to visualize and analyze events describing threat + actors' targets, motives, or behaviors. + expected_event_types: + - indicator + name: threat - description: 'Relating to web server access. Use this category to create a dashboard of web server/proxy activity from apache, IIS, nginx web servers, etc. Note: events from network observers such as Zeek http log may also be included in @@ -2655,6 +2660,13 @@ event.kind: This value is not used by Elastic solutions for alert documents that are created by rules executing within the Kibana alerting framework.' name: alert + - description: 'The `enrichment` value indicates an event collected to provide additional + context, often to other events. + + An example is collecting indicators of compromise (IOCs) from a threat intelligence + provider with the intent to use those values to enrich other events. The IOC + events from the intelligence provider should be categorized as `event.kind:enrichment`.' + name: enrichment - description: This value is the most general and most common value for this field. It is used to represent events that indicate that something happened. name: event @@ -3004,6 +3016,11 @@ event.type: AND event.type:group`. You can further distinguish group operations using the ECS `event.action` field.' name: group + - description: 'The indicator event type is used for the subset of events within + a category that contain details about indicators of compromise (IOCs). + + A common example is `event.category:threat AND event.type:indicator`.' + name: indicator - description: The info event type is used for the subset of events within a category that indicate that they are purely informational, and don't report a state change, or any type of action. For example, an initial run of a file integrity monitoring diff --git a/experimental/generated/ecs/ecs_nested.yml b/experimental/generated/ecs/ecs_nested.yml index ee036b694a..2b1f8e5e7c 100644 --- a/experimental/generated/ecs/ecs_nested.yml +++ b/experimental/generated/ecs/ecs_nested.yml @@ -3275,6 +3275,11 @@ event: - end - info name: session + - description: Use this category to visualize and analyze events describing + threat actors' targets, motives, or behaviors. + expected_event_types: + - indicator + name: threat - description: 'Relating to web server access. Use this category to create a dashboard of web server/proxy activity from apache, IIS, nginx web servers, etc. Note: events from network observers such as Zeek http log may also @@ -3438,6 +3443,13 @@ event: This value is not used by Elastic solutions for alert documents that are created by rules executing within the Kibana alerting framework.' name: alert + - description: 'The `enrichment` value indicates an event collected to provide + additional context, often to other events. + + An example is collecting indicators of compromise (IOCs) from a threat intelligence + provider with the intent to use those values to enrich other events. The + IOC events from the intelligence provider should be categorized as `event.kind:enrichment`.' + name: enrichment - description: This value is the most general and most common value for this field. It is used to represent events that indicate that something happened. name: event @@ -3796,6 +3808,11 @@ event: AND event.type:creation AND event.type:group`. You can further distinguish group operations using the ECS `event.action` field.' name: group + - description: 'The indicator event type is used for the subset of events within + a category that contain details about indicators of compromise (IOCs). + + A common example is `event.category:threat AND event.type:indicator`.' + name: indicator - description: The info event type is used for the subset of events within a category that indicate that they are purely informational, and don't report a state change, or any type of action. For example, an initial run of a diff --git a/generated/ecs/ecs_flat.yml b/generated/ecs/ecs_flat.yml index b55fe89829..285cd8e811 100644 --- a/generated/ecs/ecs_flat.yml +++ b/generated/ecs/ecs_flat.yml @@ -2062,6 +2062,11 @@ event.category: - end - info name: session + - description: Use this category to visualize and analyze events describing threat + actors' targets, motives, or behaviors. + expected_event_types: + - indicator + name: threat - description: 'Relating to web server access. Use this category to create a dashboard of web server/proxy activity from apache, IIS, nginx web servers, etc. Note: events from network observers such as Zeek http log may also be included in @@ -2222,6 +2227,13 @@ event.kind: This value is not used by Elastic solutions for alert documents that are created by rules executing within the Kibana alerting framework.' name: alert + - description: 'The `enrichment` value indicates an event collected to provide additional + context, often to other events. + + An example is collecting indicators of compromise (IOCs) from a threat intelligence + provider with the intent to use those values to enrich other events. The IOC + events from the intelligence provider should be categorized as `event.kind:enrichment`.' + name: enrichment - description: This value is the most general and most common value for this field. It is used to represent events that indicate that something happened. name: event @@ -2571,6 +2583,11 @@ event.type: AND event.type:group`. You can further distinguish group operations using the ECS `event.action` field.' name: group + - description: 'The indicator event type is used for the subset of events within + a category that contain details about indicators of compromise (IOCs). + + A common example is `event.category:threat AND event.type:indicator`.' + name: indicator - description: The info event type is used for the subset of events within a category that indicate that they are purely informational, and don't report a state change, or any type of action. For example, an initial run of a file integrity monitoring diff --git a/generated/ecs/ecs_nested.yml b/generated/ecs/ecs_nested.yml index c518a8702e..b2454083d2 100644 --- a/generated/ecs/ecs_nested.yml +++ b/generated/ecs/ecs_nested.yml @@ -2841,6 +2841,11 @@ event: - end - info name: session + - description: Use this category to visualize and analyze events describing + threat actors' targets, motives, or behaviors. + expected_event_types: + - indicator + name: threat - description: 'Relating to web server access. Use this category to create a dashboard of web server/proxy activity from apache, IIS, nginx web servers, etc. Note: events from network observers such as Zeek http log may also @@ -3004,6 +3009,13 @@ event: This value is not used by Elastic solutions for alert documents that are created by rules executing within the Kibana alerting framework.' name: alert + - description: 'The `enrichment` value indicates an event collected to provide + additional context, often to other events. + + An example is collecting indicators of compromise (IOCs) from a threat intelligence + provider with the intent to use those values to enrich other events. The + IOC events from the intelligence provider should be categorized as `event.kind:enrichment`.' + name: enrichment - description: This value is the most general and most common value for this field. It is used to represent events that indicate that something happened. name: event @@ -3362,6 +3374,11 @@ event: AND event.type:creation AND event.type:group`. You can further distinguish group operations using the ECS `event.action` field.' name: group + - description: 'The indicator event type is used for the subset of events within + a category that contain details about indicators of compromise (IOCs). + + A common example is `event.category:threat AND event.type:indicator`.' + name: indicator - description: The info event type is used for the subset of events within a category that indicate that they are purely informational, and don't report a state change, or any type of action. For example, an initial run of a diff --git a/schemas/event.yml b/schemas/event.yml index ad937ef349..ed7ec19a3a 100644 --- a/schemas/event.yml +++ b/schemas/event.yml @@ -64,9 +64,18 @@ `event.kind:alert` is often populated for events coming from firewalls, intrusion detection systems, endpoint detection and response systems, and so on. - + This value is not used by Elastic solutions for alert documents that are created by rules executing within the Kibana alerting framework. + - name: enrichment + description: > + The `enrichment` value indicates an event collected to provide additional + context, often to other events. + + An example is collecting indicators of compromise (IOCs) from a threat + intelligence provider with the intent to use those values to enrich other + events. The IOC events from the intelligence provider should be categorized + as `event.kind:enrichment`. - name: event description: > This value is the most general and most common value for this field. @@ -296,6 +305,11 @@ - start - end - info + - name: threat + description: > + Use this category to visualize and analyze events describing threat actors' targets, motives, or behaviors. + expected_event_types: + - indicator - name: web description: > Relating to web server access. Use this category to create a dashboard of @@ -475,6 +489,12 @@ Common example: `event.category:iam AND event.type:creation AND event.type:group`. You can further distinguish group operations using the ECS `event.action` field. + - name: indicator + description: > + The indicator event type is used for the subset of events within a category + that contain details about indicators of compromise (IOCs). + + A common example is `event.category:threat AND event.type:indicator`. - name: info description: > The info event type is used for the subset of events within a category