New event categorization values to support threat intel use cases (el…

…astic#1510)

* introduce event.kind:enrichment, event.category:threat, and event.type:indicator

* update docs and artifacts

* Drop mention of "cybersecurity threats"

Co-authored-by: Andrew Pease <[email protected]>

* artifacts

* alphabetize

Co-authored-by: Andrew Pease <[email protected]>

docs/field-details.asciidoc

@@ -2576,7 +2576,7 @@ Note: this field should contain an array of values.
 
 *Important*: The field value must be one of the following:
 
-authentication, configuration, database, driver, file, host, iam, intrusion_detection, malware, network, package, process, registry, session, web
+authentication, configuration, database, driver, file, host, iam, intrusion_detection, malware, network, package, process, registry, session, threat, web
 
 To learn more about when to use which value, visit the page
 <<ecs-allowed-values-event-category,allowed values for event.category>>
@@ -2749,7 +2749,7 @@ type: keyword
 
 *Important*: The field value must be one of the following:
 
-alert, event, metric, state, pipeline_error, signal
+alert, enrichment, event, metric, state, pipeline_error, signal
 
 To learn more about when to use which value, visit the page
 <<ecs-allowed-values-event-kind,allowed values for event.kind>>
@@ -3006,7 +3006,7 @@ Note: this field should contain an array of values.
 
 *Important*: The field value must be one of the following:
 
-access, admin, allowed, change, connection, creation, deletion, denied, end, error, group, info, installation, protocol, start, user
+access, admin, allowed, change, connection, creation, deletion, denied, end, error, group, indicator, info, installation, protocol, start, user
 
 To learn more about when to use which value, visit the page
 <<ecs-allowed-values-event-type,allowed values for event.type>>

docs/field-values.asciidoc

@@ -41,6 +41,7 @@ The value of this field can be used to inform how these kinds of events should b
 *Allowed Values*
 
 * <<ecs-event-kind-alert,alert>>
+* <<ecs-event-kind-enrichment,enrichment>>
 * <<ecs-event-kind-event,event>>
 * <<ecs-event-kind-metric,metric>>
 * <<ecs-event-kind-state,state>>
@@ -59,6 +60,16 @@ This value is not used by Elastic solutions for alert documents that are created
 
 
 
+[float]
+[[ecs-event-kind-enrichment]]
+==== enrichment
+
+The `enrichment` value indicates an event collected to provide additional context, often to other events.
+
+An example is collecting indicators of compromise (IOCs) from a threat intelligence provider with the intent to use those values to enrich other events. The IOC events from the intelligence provider should be categorized as `event.kind:enrichment`.
+
+
+
 [float]
 [[ecs-event-kind-event]]
 ==== event
@@ -136,6 +147,7 @@ This field is an array. This will allow proper categorization of some events tha
 * <<ecs-event-category-process,process>>
 * <<ecs-event-category-registry,registry>>
 * <<ecs-event-category-session,session>>
+* <<ecs-event-category-threat,threat>>
 * <<ecs-event-category-web,web>>
 
 [float]
@@ -314,6 +326,18 @@ The session category is applied to events and metrics regarding logical persiste
 start, end, info
 
 
+[float]
+[[ecs-event-category-threat]]
+==== threat
+
+Use this category to visualize and analyze events describing threat actors' targets, motives, or behaviors.
+
+
+*Expected event types for category threat:*
+
+indicator
+
+
 [float]
 [[ecs-event-category-web]]
 ==== web
@@ -348,6 +372,7 @@ This field is an array. This will allow proper categorization of some events tha
 * <<ecs-event-type-end,end>>
 * <<ecs-event-type-error,error>>
 * <<ecs-event-type-group,group>>
+* <<ecs-event-type-indicator,indicator>>
 * <<ecs-event-type-info,info>>
 * <<ecs-event-type-installation,installation>>
 * <<ecs-event-type-protocol,protocol>>
@@ -442,6 +467,16 @@ The group event type is used for the subset of events within a category that are
 
 
 
+[float]
+[[ecs-event-type-indicator]]
+==== indicator
+
+The indicator event type is used for the subset of events within a category that contain details about indicators of compromise (IOCs).
+
+A common example is `event.category:threat AND event.type:indicator`.
+
+
+
 [float]
 [[ecs-event-type-info]]
 ==== info

experimental/generated/ecs/ecs_flat.yml

@@ -2407,6 +2407,11 @@ event.category:
     - end
     - info
     name: session
+  - description: Use this category to visualize and analyze events describing threat
+      actors' targets, motives, or behaviors.
+    expected_event_types:
+    - indicator
+    name: threat
   - description: 'Relating to web server access. Use this category to create a dashboard
       of web server/proxy activity from apache, IIS, nginx web servers, etc. Note:
       events from network observers such as Zeek http log may also be included in
@@ -2567,6 +2572,13 @@ event.kind:
       This value is not used by Elastic solutions for alert documents that are created
       by rules executing within the Kibana alerting framework.'
     name: alert
+  - description: 'The `enrichment` value indicates an event collected to provide additional
+      context, often to other events.
+
+      An example is collecting indicators of compromise (IOCs) from a threat intelligence
+      provider with the intent to use those values to enrich other events. The IOC
+      events from the intelligence provider should be categorized as `event.kind:enrichment`.'
+    name: enrichment
   - description: This value is the most general and most common value for this field.
       It is used to represent events that indicate that something happened.
     name: event
@@ -2916,6 +2928,11 @@ event.type:
       AND event.type:group`. You can further distinguish group operations using the
       ECS `event.action` field.'
     name: group
+  - description: 'The indicator event type is used for the subset of events within
+      a category that contain details about indicators of compromise (IOCs).
+
+      A common example is `event.category:threat AND event.type:indicator`.'
+    name: indicator
   - description: The info event type is used for the subset of events within a category
       that indicate that they are purely informational, and don't report a state change,
       or any type of action. For example, an initial run of a file integrity monitoring

experimental/generated/ecs/ecs_nested.yml

@@ -3185,6 +3185,11 @@ event:
         - end
         - info
         name: session
+      - description: Use this category to visualize and analyze events describing
+          threat actors' targets, motives, or behaviors.
+        expected_event_types:
+        - indicator
+        name: threat
       - description: 'Relating to web server access. Use this category to create a
           dashboard of web server/proxy activity from apache, IIS, nginx web servers,
           etc. Note: events from network observers such as Zeek http log may also
@@ -3348,6 +3353,13 @@ event:
           This value is not used by Elastic solutions for alert documents that are
           created by rules executing within the Kibana alerting framework.'
         name: alert
+      - description: 'The `enrichment` value indicates an event collected to provide
+          additional context, often to other events.
+
+          An example is collecting indicators of compromise (IOCs) from a threat intelligence
+          provider with the intent to use those values to enrich other events. The
+          IOC events from the intelligence provider should be categorized as `event.kind:enrichment`.'
+        name: enrichment
       - description: This value is the most general and most common value for this
           field. It is used to represent events that indicate that something happened.
         name: event
@@ -3706,6 +3718,11 @@ event:
           AND event.type:creation AND event.type:group`. You can further distinguish
           group operations using the ECS `event.action` field.'
         name: group
+      - description: 'The indicator event type is used for the subset of events within
+          a category that contain details about indicators of compromise (IOCs).
+
+          A common example is `event.category:threat AND event.type:indicator`.'
+        name: indicator
       - description: The info event type is used for the subset of events within a
           category that indicate that they are purely informational, and don't report
           a state change, or any type of action. For example, an initial run of a

generated/ecs/ecs_flat.yml

@@ -2057,6 +2057,11 @@ event.category:
     - end
     - info
     name: session
+  - description: Use this category to visualize and analyze events describing threat
+      actors' targets, motives, or behaviors.
+    expected_event_types:
+    - indicator
+    name: threat
   - description: 'Relating to web server access. Use this category to create a dashboard
       of web server/proxy activity from apache, IIS, nginx web servers, etc. Note:
       events from network observers such as Zeek http log may also be included in
@@ -2217,6 +2222,13 @@ event.kind:
       This value is not used by Elastic solutions for alert documents that are created
       by rules executing within the Kibana alerting framework.'
     name: alert
+  - description: 'The `enrichment` value indicates an event collected to provide additional
+      context, often to other events.
+
+      An example is collecting indicators of compromise (IOCs) from a threat intelligence
+      provider with the intent to use those values to enrich other events. The IOC
+      events from the intelligence provider should be categorized as `event.kind:enrichment`.'
+    name: enrichment
   - description: This value is the most general and most common value for this field.
       It is used to represent events that indicate that something happened.
     name: event
@@ -2566,6 +2578,11 @@ event.type:
       AND event.type:group`. You can further distinguish group operations using the
       ECS `event.action` field.'
     name: group
+  - description: 'The indicator event type is used for the subset of events within
+      a category that contain details about indicators of compromise (IOCs).
+
+      A common example is `event.category:threat AND event.type:indicator`.'
+    name: indicator
   - description: The info event type is used for the subset of events within a category
       that indicate that they are purely informational, and don't report a state change,
       or any type of action. For example, an initial run of a file integrity monitoring

generated/ecs/ecs_nested.yml

@@ -2835,6 +2835,11 @@ event:
         - end
         - info
         name: session
+      - description: Use this category to visualize and analyze events describing
+          threat actors' targets, motives, or behaviors.
+        expected_event_types:
+        - indicator
+        name: threat
       - description: 'Relating to web server access. Use this category to create a
           dashboard of web server/proxy activity from apache, IIS, nginx web servers,
           etc. Note: events from network observers such as Zeek http log may also
@@ -2998,6 +3003,13 @@ event:
           This value is not used by Elastic solutions for alert documents that are
           created by rules executing within the Kibana alerting framework.'
         name: alert
+      - description: 'The `enrichment` value indicates an event collected to provide
+          additional context, often to other events.
+
+          An example is collecting indicators of compromise (IOCs) from a threat intelligence
+          provider with the intent to use those values to enrich other events. The
+          IOC events from the intelligence provider should be categorized as `event.kind:enrichment`.'
+        name: enrichment
       - description: This value is the most general and most common value for this
           field. It is used to represent events that indicate that something happened.
         name: event
@@ -3356,6 +3368,11 @@ event:
           AND event.type:creation AND event.type:group`. You can further distinguish
           group operations using the ECS `event.action` field.'
         name: group
+      - description: 'The indicator event type is used for the subset of events within
+          a category that contain details about indicators of compromise (IOCs).
+
+          A common example is `event.category:threat AND event.type:indicator`.'
+        name: indicator
       - description: The info event type is used for the subset of events within a
           category that indicate that they are purely informational, and don't report
           a state change, or any type of action. For example, an initial run of a

schemas/event.yml

@@ -64,9 +64,18 @@
 
             `event.kind:alert` is often populated for events coming from firewalls,
             intrusion detection systems, endpoint detection and response systems, and so on.
-
+            
             This value is not used by Elastic solutions for alert documents
             that are created by rules executing within the Kibana alerting framework.
+        - name: enrichment
+          description: >
+            The `enrichment` value indicates an event collected to provide additional
+            context, often to other events.
+
+            An example is collecting indicators of compromise (IOCs) from a threat
+            intelligence provider with the intent to use those values to enrich other
+            events. The IOC events from the intelligence provider should be categorized
+            as `event.kind:enrichment`.
         - name: event
           description: >
             This value is the most general and most common value for this field.
@@ -296,6 +305,11 @@
             - start
             - end
             - info
+        - name: threat
+          description: >
+            Use this category to visualize and analyze events describing threat actors' targets, motives, or behaviors.
+          expected_event_types:
+            - indicator
         - name: web
           description: >
             Relating to web server access. Use this category to create a dashboard of
@@ -475,6 +489,12 @@
             Common example: `event.category:iam AND event.type:creation AND event.type:group`.
             You can further distinguish group operations using the ECS
             `event.action` field.
+        - name: indicator
+          description: >
+            The indicator event type is used for the subset of events within a category
+            that contain details about indicators of compromise (IOCs).
+
+            A common example is `event.category:threat AND event.type:indicator`.
         - name: info
           description: >
             The info event type is used for the subset of events within a category