[DOCS] Rules updates (7.12)

docs/detections/detection-engine-intro.asciidoc

@@ -68,16 +68,26 @@ To make sure you can access Detections and manage rules, see
 Cold tier is a {ref}/data-tiers.html[data tier] that holds time series data that is accessed only occasionally. In {stack} version >=7.11.0, {es-sec} supports cold tier data for the following {es} indices:
 
 * Index patterns specified in `securitySolution:defaultIndex`
-* Index patterns specified in the definitions of detection rules
-* Index patterns specified in the data sources selector on various {security-app} pages
+* Index patterns specified in the definitions of detection rules, except for indicator match rules
+* Index patterns specified in the data sources selector on various {es-sec-app} pages
 
-{es-sec} does NOT support cold tier data for the following {es} indices:
+{es-sec} does *NOT* support cold tier data for the following {es} indices:
 
 * Index patterns controlled by {es-sec}, including signals and list indices
-* Index patterns specified in indicator match rules as indicator index patterns
+Index patterns specified in indicator match rules
 
 Using cold tier data for unsupported indices may result in detection rule timeouts and overall performance degradation.
 
+[float]
+[[support-indicator-rules]]
+== Limited support for indicator match rules
+
+Indicator match rules provide a powerful capability to search your security data; however, their queries can consume significant deployment resources. As such, the following support restrictions are in place:
+
+* {es-sec} does not support the use of cold tier data with indicator match rules.
+* The use of Cross Cluster Search (CCS) with indicator match rules is not supported.
+* Indicator match rules with an additional look-back time value greater than 24 hours are not supported.
+
 [float]
 [[det-engine-terminology]]
 == Terminology

docs/detections/prebuilt-rules/tune-rule-signals.asciidoc

@@ -1,5 +1,5 @@
 [[tuning-detection-signals]]
-== Tuning prebuilt detection rules
+== Tune detection rules
 
 In the {es-sec-app}, prebuilt detection rules can be tuned to produce the best
 possible set of actionable alerts. To reduce the noise level, you can:
@@ -16,11 +16,12 @@ actionable alerts.
 * Clone and modify detection rule risk scores, and use branching logic to map
 higher risk scores to higher priority workflows.
 
-For details about tuning prebuilt rules for specific categories, see:
+For details about tuning rules for specific categories, see:
 
 * <<tune-authorized-processes>>
 * <<tune-windows-rules>>
 * <<tune-network-rules>>
+* <<tune-indicator-rules>>
 
 [float]
 [[filter-rule-process]]
@@ -33,7 +34,7 @@ add an exception for the required application.
 For example, to prevent the <<unusual-process-execution-temp>> rule from
 producing alerts for an in-house application named `myautomatedbuild`:
 
-. Go to *Security* -> *Detections* -> *Manage detection rules*.
+. Go to *Detections* -> *Manage detection rules*.
 . Search for and then click on the `Unusual Process Execution - Temp` rule.
 +
 The *Unusual Process Execution - Temp* rule details page is displayed.
@@ -72,8 +73,8 @@ To reduce noise for authorized activity, you can do any of these:
 the relevant host names, agent names, or other common identifiers.
 For example, `host.name is <server-name>`.
 * Add an exception to the rules that <<filter-rule-process, exclude specific
-processes>>. 
-For example, `process.name is <process-name>`. 
+processes>>.
+For example, `process.name is <process-name>`.
 * Add an exception to the rules that exclude a common user.
 For example, `user.name is <security-tester>`.
 
@@ -146,6 +147,17 @@ exception with the port number
 exception with the port number
 (`destination.ip is 172.16.0.0/12` and `destination.port is 445`).
 
+[float]
+[[tune-indicator-rules]]
+=== Tune indicator match rules
+
+Take the following steps to tune indicator match rules:
+
+* Specify a detailed query as part of the indicator index query. Every result returned from the indicator index query will be used by the detection engine to subsequently query the index patterns defined in your rule definition. Using no query or the wildcard `*.*` query will result in your rule executing potentially very large queries.
+* Limit your rule's additional look-back time to as short a duration as possible, and no more than 24 hours.
+
+NOTE: {es-sec} provides limited support for indicator match rules. See <<support-indicator-rules, support limitations>> for more information.
+
 [float]
 ==== Noise from common network traffic
 

docs/detections/rules-ui-create.asciidoc

@@ -1,6 +1,6 @@
 [[rules-ui-create]]
 [role="xpack"]
-== Creating detection rules
+== Create a detection rule
 
 Rules run periodically and search for source events or {ml} job anomaly scores
 that meet their criteria. When a rule's criteria are met, a detection alert is
@@ -134,22 +134,12 @@ and then:
 .. Define which {es} indices the rule analyzes for alerts.
 .. Use the filter and query fields to create the criteria used for detecting
 alerts.
-.. Use the `Field` and `Threshold` fields to determine which source event field
-is used as a threshold and the threshold's value.
+.. Use the `Group by` and `Threshold` fields to determine which source event field is used as a threshold and the threshold's value.
+.. Use the `Count` field to limit alerts by cardinality of a certain field.
 +
-For example, if the `Field` is `source.ip` and its `Threshold` is `10`, an
-alert is generated for every source IP address that appears in at least 10 of
-the rule's search results.
-+
-You can also leave the `Field` undefined. The rule then creates an alert when
-the number of search results is equal to or greater than the `Threshold` value.
+For example, if `Group by` is `source.ip`, `destination.ip` and its `Threshold` is `10`, an alert is generated for every pair of source and destination IP addresses that appear in at least 10 of the rule's search results.
 +
-If you want an alert for 10 or more failed login attempts to a specific host
-per rule execution, enter the following rule parameters:
-
-** *Custom query*: `host.name : liv-win-19 and event.category : "authentication" and event.outcome : "failure"`
-** *Field*: Leave blank
-** *Threshold*: `10`
+You can also leave the `Group by` field undefined. The rule then creates an alert when the number of search results is equal to or greater than the threshold value. If you set `Count` to limit the results by `process.name` >= 2, an alert will only be generated for source/destination IP pairs that appear with at least 2 unique process names across all events.
 
 [discrete]
 [[create-eql-rule]]
@@ -191,12 +181,14 @@ NOTE: For sequence events, the {security-app} generates a single alert when all
 [discrete]
 [[create-indicator-rule]]
 ==== Create an indicator match rule
+
+NOTE: {es-sec} provides limited support for indicator match rules. See <<support-indicator-rules>> for more information.
+
 * To create an indicator match rule that searches the specified indicator index patterns for
 field values, select *Indicator Match* and then fill in the following fields:
 .. *Index patterns*: The {es-sec} event indices on which the rules runs.
 .. *Custom query*: The query and filters used to retrieve the required results from
-the {es-sec} event indices. For example, if you only need to check
-`destination.ip` event values, add `destination.ip : *`.
+the {es-sec} event indices. For example, if you want to match documents that only contain a `destination.ip` address field, add `destination.ip : *`.
 +
 TIP: If you want the rule to check every field in the indices, use this
 wildcard expression: `*:*`.
@@ -222,9 +214,6 @@ index, enter the rule parameters seen in the following image:
 +
 [role="screenshot"]
 image::images/indicator-rule-example.png[]
-TIP: When an indicator match rule's conditions are met, the resulting detection alert does not contain explicit information about which event field(s) match which indicator field(s). As such, when you <<rule-ui-basic-params, configure basic rule settings>>, it is recommended that you include a reference to the field(s) to be matched in the rule `Name` and rule `Description`, and ensure that the Timeline template associated with the rule includes pre-defined column(s) for these fields. For example, if you create an indicator match rule that looks for matches between the `file.extension` field in file events and the `threat.file.extension` field in an indicator index, you might name your rule `"file.extension matches ransomware file extension"`, so that when an analyst investigates the detection alerts, they will see the rule name and know to further investigate the `file.extension` field value.
-. Select the Timeline template used when you investigate an alert created by
-the rule in Timeline (optional).
 +
 TIP: Before you create rules, create <<timelines-ui, Timeline templates>> so
 they can be selected here. When alerts generated by the rule are investigated

docs/detections/rules-ui-monitor.asciidoc

@@ -1,6 +1,6 @@
 [[alerts-ui-monitor]]
 [role="xpack"]
-== Monitoring and troubleshooting rule executions
+== Monitor and troubleshoot rule executions
 
 To view a summary of all rule executions, such as failures and last execution
 times, select the *Rule Monitoring* tab in the *All rules* table (*Security* ->
@@ -24,6 +24,8 @@ If you see `Gaps` in the All rules table or on the Rule details page
 for a small number of rules, you can increase those rules'
 `Additional look-back time` (*Detection rules* page -> the rule's *All actions* button (*...*) -> *Edit rule settings* -> *Schedule* -> _Additional look-back time_).
 
+NOTE: If the rule that experiences gaps is an indicator match rule, see <<tune-indicator-rules, how to tune indicator match rules>>. Also please note that {es-sec} provides <<support-indicator-rules, limited support for indicator match rules>>.
+
 If you see gaps for numerous rules:
 
 * If you restarted {kib} when many rules were activated, try deactivating them

docs/es-overview.asciidoc

@@ -88,13 +88,13 @@ and maps.
 Cold tier is a {ref}/data-tiers.html[data tier] that holds time series data that is accessed only occasionally. In {stack} version >=7.11.0, {es-sec} supports cold tier data for the following {es} indices:
 
 * Index patterns specified in `securitySolution:defaultIndex`
-* Index patterns specified in the definitions of detection rules
-* Index patterns specified in the data sources selector on various {security-app} pages
+* Index patterns specified in the definitions of detection rules, except for indicator match rules
+* Index patterns specified in the data sources selector on various {es-sec-app} pages
 
 {es-sec} does NOT support cold tier data for the following {es} indices:
 
 * Index patterns controlled by {es-sec}, including signals and list indices
-* Index patterns specified in indicator match rules as indicator index patterns
+Index patterns specified in indicator match rules
 
 Using cold tier data for unsupported indices may result in detection rule timeouts and overall performance degradation.