Merge branch 'master' into rebalanceXpackGroups

docs/management/watcher-ui/index.asciidoc

@@ -2,13 +2,13 @@
 [[watcher-ui]]
 == Watcher
 
-Watcher is an {es} feature that you can use to create actions based on 
-conditions, which are periodically evaluated using queries on your data. 
-Watches are helpful for analyzing mission-critical and business-critical 
-streaming data. For example, you might watch application logs for performance 
+Watcher is an {es} feature that you can use to create actions based on
+conditions, which are periodically evaluated using queries on your data.
+Watches are helpful for analyzing mission-critical and business-critical
+streaming data. For example, you might watch application logs for performance
 outages or audit access logs for security threats.
 
-To get started with the Watcher UI, go to *Management > Elasticsearch > Watcher*. 
+To get started with the Watcher UI, go to *Management > Elasticsearch > Watcher*.
 With this UI, you can:
 
 * <<watcher-create-threshold-alert, Create a simple threshold watch>>
@@ -20,10 +20,10 @@ With this UI, you can:
 image:management/watcher-ui/images/watches.png["Watcher list"]
 
 {ref}/xpack-alerting.html[Alerting on cluster and index events]
-is a good source for detailed 
-information on how watches work. If you are using the UI to create a 
-threshold watch, take a look at the different watcher actions. If you are 
-creating an advanced watch, you should be familiar with the parts of a 
+is a good source for detailed
+information on how watches work. If you are using the UI to create a
+threshold watch, take a look at the different watcher actions. If you are
+creating an advanced watch, you should be familiar with the parts of a
 watch&#8212;input, schedule, condition, and actions.
 
 [float]
@@ -40,41 +40,40 @@ and either of these watcher roles:
 * `watcher_admin`. You can perform all Watcher actions, including create and edit watches.
 * `watcher_user`. You can view watches, but not create or edit them.
 
-You can manage roles in *Management > Security > Roles*, or use the 
-<<role-management-api, Kibana Role Management API>>. Watches are shared between 
-all users with the same role.  
+You can manage roles in *Management > Security > Roles*, or use the
+<<role-management-api, Kibana Role Management API>>. Watches are shared between
+all users with the same role.
 
-NOTE: If you are creating a threshold watch, you must also have index management 
-privileges.  See 
+NOTE: If you are creating a threshold watch, you must also have the `view_index_metadata` index privilege. See
 <<managing-indices, Managing Indices>> for detailed information.
 
 [float]
 [[watcher-create-threshold-alert]]
 === Create a threshold alert
 
-A threshold alert is one of the most common types of watches that you can create.  
-This alert periodically checks when your data is above, below, equals, 
+A threshold alert is one of the most common types of watches that you can create.
+This alert periodically checks when your data is above, below, equals,
 or is in between a certain threshold within a given time interval.
 
-The following example walks you through creating a threshold alert. The alert 
-is triggered when the maximum total CPU usage on a machine goes above a 
-certain percentage. The example uses https://www.elastic.co/products/beats/metricbeat[Metricbeat] 
-to collect metrics from your systems and services. 
-{metricbeat-ref}/metricbeat-installation.html[Learn more] on how to install 
+The following example walks you through creating a threshold alert. The alert
+is triggered when the maximum total CPU usage on a machine goes above a
+certain percentage. The example uses https://www.elastic.co/products/beats/metricbeat[Metricbeat]
+to collect metrics from your systems and services.
+{metricbeat-ref}/metricbeat-installation.html[Learn more] on how to install
 and get started with Metricbeat.
 
 [float]
 ==== Define the watch input and schedule
 
-. Click *Create* and then select *Create threshold alert*.  
+. Click *Create* and then select *Create threshold alert*.
 +
 You're navigated to a page where you're asked to define the watch name, the data that you want to evaluate, and
 how often you want to trigger the watch.
 
 . Enter a name that you want to call the alert, for example, `cpu_threshold_alert`.
 
-. In the *Indices to query* field, enter `metricbeat-*` and select `@timestamp` 
-as the time field. 
+. In the *Indices to query* field, enter `metricbeat-*` and select `@timestamp`
+as the time field.
 
 . Use the default schedule to run the watch every 1 minute.
 +
@@ -84,22 +83,22 @@ image:management/watcher-ui/images/threshold-alert/create-threshold-alert-create
 [float]
 ==== Add a condition
 
-You should now see a panel with default conditions and a visualization of the 
-data based on those conditions. The condition evaluates the data you’ve loaded 
+You should now see a panel with default conditions and a visualization of the
+data based on those conditions. The condition evaluates the data you’ve loaded
 into the watch and determines if any action is required.
 
-. Click the `WHEN` expression and change the value to `max()`. 
+. Click the `WHEN` expression and change the value to `max()`.
 +
-The `OF` expression now appears.  
+The `OF` expression now appears.
 
-. Search for `system.process.cpu.total.norm.pct` and select it from the list. 
+. Search for `system.process.cpu.total.norm.pct` and select it from the list.
 
 
-. Select the `IS ABOVE` expression and change the value to `.25` to trigger 
+. Select the `IS ABOVE` expression and change the value to `.25` to trigger
 an alert whenever the CPU is above 25%.
 +
-As you change the condition, the visualization is automatically updated. The black 
-line represents the threshold (25%), while the green fluctuating line 
+As you change the condition, the visualization is automatically updated. The black
+line represents the threshold (25%), while the green fluctuating line
 represents the change in CPU over the set time period.
 +
 [role="screenshot"]
@@ -108,46 +107,46 @@ image:management/watcher-ui/images/threshold-alert/threshold-alert-condition.png
 [float]
 ==== Add an action
 
-Now that the condition is set, you must add an action. The action triggers 
-when the watch condition is met. For a complete list of actions and how to configure them, see 
+Now that the condition is set, you must add an action. The action triggers
+when the watch condition is met. For a complete list of actions and how to configure them, see
 {ref}/action-conditions.html[Adding conditions to actions].
 
 In this example, you’ll configure an email action. You must have an {ref}/actions-email.html#configuring-email[email account configured]
-in {es} for this example to work. 
+in {es} for this example to work.
 
 . Click *Add action* and select *Email*.
 
-. In the *To email address* field, enter one or more email addresses to whom 
-you want to send the message when the condition is met. 
+. In the *To email address* field, enter one or more email addresses to whom
+you want to send the message when the condition is met.
 
 . Enter a subject and body for the email.
 +
 [role="screenshot"]
 image:management/watcher-ui/images/threshold-alert/threshold-alert-action.png["Action for threshold alert"]
 
-. To test the action before saving the watch, click *Send test email*. 
+. To test the action before saving the watch, click *Send test email*.
 +
 A sample email is sent using the configuration you set up.
 
-. Click *Create alert*.  
+. Click *Create alert*.
 +
-The alert appears on the Watcher overview page, where you can drill down into 
+The alert appears on the Watcher overview page, where you can drill down into
 the watch history and status.
 
 [float]
 ==== Delete the alert
 
-In this example, you set the threshold to 25% so you can see the watch fire. In 
-a  real-world scenario, this threshold is likely too low because the alerts are 
-so frequent. Once you are done experimenting, you should delete the alert. 
+In this example, you set the threshold to 25% so you can see the watch fire. In
+a  real-world scenario, this threshold is likely too low because the alerts are
+so frequent. Once you are done experimenting, you should delete the alert.
 Find the alert on the Watcher overview page and click the trash icon in the *Actions* column.
 
 [float]
 ==== Edit the alert
 
-Alternatively, you can keep the alert and adjust the threshold value. To edit 
-an alert, find the alert on the Watcher overview page and click the pencil icon 
-in the *Actions* column. 
+Alternatively, you can keep the alert and adjust the threshold value. To edit
+an alert, find the alert on the Watcher overview page and click the pencil icon
+in the *Actions* column.
 
 [float]
 [[watcher-getting-started]]
@@ -161,24 +160,24 @@ last fired, and last triggered.  A watch has one of four states:
 * *Disabled.* The watch will not fire under any circumstances.
 
 From this page you can drill down into a watch to investigate its history
-and status.  
+and status.
 
 [float]
 ==== View watch history
 
-The *Execution history* tab shows each time the watch is triggered and the 
-results of the query, whether the condition was met, and what actions were taken. 
+The *Execution history* tab shows each time the watch is triggered and the
+results of the query, whether the condition was met, and what actions were taken.
 
 [role="screenshot"]
 image:management/watcher-ui/images/execution-history.png["Execution history tab"]
 
 [float]
 ==== Acknowledge action status
 
-The *Action statuses* tab lists all actions associated with the watch and 
-the state of each action. If the action is firing, you can acknowledge the 
-watch to prevent too many executions of the same action for the same watch. 
-See {ref}/actions.html#actions-ack-throttle[Acknowledgement and throttling] for details. 
+The *Action statuses* tab lists all actions associated with the watch and
+the state of each action. If the action is firing, you can acknowledge the
+watch to prevent too many executions of the same action for the same watch.
+See {ref}/actions.html#actions-ack-throttle[Acknowledgement and throttling] for details.
 
 [role="screenshot"]
 image:management/watcher-ui/images/alerts-status.png["Action status tab"]
@@ -189,44 +188,44 @@ image:management/watcher-ui/images/alerts-status.png["Action status tab"]
 
 Actions for deactivating and deleting a watch are on each watch detail page:
 
-* *Deactivate a watch* if you know a situation is planned that will 
-cause a false alarm. You can reactivate the watch when the situation is resolved. 
-* *Delete a watch* to permanently remove it from the system. You can delete 
-the watch you are currently viewing, or go to the Watcher overview, and 
-delete watches in bulk. 
+* *Deactivate a watch* if you know a situation is planned that will
+cause a false alarm. You can reactivate the watch when the situation is resolved.
+* *Delete a watch* to permanently remove it from the system. You can delete
+the watch you are currently viewing, or go to the Watcher overview, and
+delete watches in bulk.
 
 [float]
 [[watcher-create-advanced-watch]]
 === Create an advanced watch
 
-Advanced watches are for users who are more familiar with {es} query syntax and 
-the Watcher framework. The UI is aligned with using the REST APIs. 
+Advanced watches are for users who are more familiar with {es} query syntax and
+the Watcher framework. The UI is aligned with using the REST APIs.
 For more information, see {ref}/query-dsl.html[Query DSL].
 
 [float]
 ==== Create the watch
 
-On the Watch overview page, click *Create* and choose *Create advanced watch*.  
-An advanced watch requires a name and ID.  Name is a user-friendly way to 
-identify the watch, and ID refers to the identifier used by {es}.  Refer to 
-{ref}/how-watcher-works.html#watch-definition[Watch definition] for how 
-to input the watch JSON.  
+On the Watch overview page, click *Create* and choose *Create advanced watch*.
+An advanced watch requires a name and ID.  Name is a user-friendly way to
+identify the watch, and ID refers to the identifier used by {es}.  Refer to
+{ref}/how-watcher-works.html#watch-definition[Watch definition] for how
+to input the watch JSON.
 
 [role="screenshot"]
 image:management/watcher-ui/images/advanced-watch/advanced-watch-create.png["Create advanced watch"]
 
 [float]
 ==== Simulate the watch
 
-The *Simulate* tab allows you to override parts of the watch, and then run a 
+The *Simulate* tab allows you to override parts of the watch, and then run a
 simulation. Be aware of these implementation details on overrides:
 
 * Trigger overrides use {ref}/common-options.html#date-math[date math].
 * Input overrides accepts a JSON blob.
 * Condition overrides indicates if you want to force the condition to always be `true`.
 * Action overrides support {ref}/watcher-api-execute-watch.html#watcher-api-execute-watch-action-mode[multiple options].
 
-After starting the simulation, you’ll see a results screen. For more information 
+After starting the simulation, you’ll see a results screen. For more information
 on the fields in the response, see the {ref}/watcher-api-execute-watch.html[Execute watch API].
 
 [role="screenshot"]
@@ -235,7 +234,7 @@ image:management/watcher-ui/images/advanced-watch/advanced-watch-simulate.png["C
 [float]
 ==== Examples of advanced watches
 
-Refer to these examples for creating an advanced watch: 
+Refer to these examples for creating an advanced watch:
 
 * {ref}/watch-cluster-status.html[Watch the status of an {es} cluster]
 * {ref}/watching-meetup-data.html[Watch event data]

package.json

@@ -203,7 +203,7 @@
     "leaflet-responsive-popup": "0.6.4",
     "leaflet-vega": "^0.8.6",
     "leaflet.heat": "0.2.0",
-    "less": "^3.0.2",
+    "less": "^2.7.3",
     "less-loader": "5.0.0",
     "lodash": "npm:@elastic/[email protected]",
     "lodash.clonedeep": "^4.5.0",

x-pack/legacy/plugins/siem/server/lib/detection_engine/routes/__mocks__/request_responses.ts

@@ -59,6 +59,7 @@ export const mockPrepackagedRule = (): PrepackagedRules => ({
   version: 1,
   false_positives: [],
   max_signals: 100,
+  note: '',
   timeline_id: 'timeline-id',
   timeline_title: 'timeline-title',
 });
@@ -392,6 +393,7 @@ export const getResult = (): RuleAlertType => ({
       },
     ],
     references: ['http://www.example.com', 'https://ww.example.com'],
+    note: '# Investigative notes',
     version: 1,
   },
   createdAt: new Date('2019-12-13T16:40:33.400Z'),

x-pack/legacy/plugins/siem/server/lib/detection_engine/routes/index/signals_mapping.json

@@ -133,6 +133,9 @@
                   }
                 }
               },
+              "note": {
+                "type": "text"
+              },
               "type": {
                 "type": "keyword"
               },

x-pack/legacy/plugins/siem/server/lib/detection_engine/routes/rules/create_rules_bulk_route.ts

@@ -78,6 +78,7 @@ export const createRulesBulkRoute = (router: IRouter) => {
               to,
               type,
               references,
+              note,
               timeline_id: timelineId,
               timeline_title: timelineTitle,
               version,
@@ -131,6 +132,7 @@ export const createRulesBulkRoute = (router: IRouter) => {
                 type,
                 threat,
                 references,
+                note,
                 version,
               });
               return transformValidateBulkError(ruleIdOrUuid, createdRule);

x-pack/legacy/plugins/siem/server/lib/detection_engine/routes/rules/create_rules_route.ts

@@ -55,6 +55,7 @@ export const createRulesRoute = (router: IRouter): void => {
         to,
         type,
         references,
+        note,
       } = request.body;
       const siemResponse = buildSiemResponse(response);
 
@@ -117,6 +118,7 @@ export const createRulesRoute = (router: IRouter): void => {
           type,
           threat,
           references,
+          note,
           version: 1,
         });
         const ruleStatuses = await savedObjectsClient.find<

x-pack/legacy/plugins/siem/server/lib/detection_engine/routes/rules/import_rules_route.ts

@@ -134,6 +134,7 @@ export const importRulesRoute = (router: IRouter, config: LegacyServices['config
                     to,
                     type,
                     references,
+                    note,
                     timeline_id: timelineId,
                     timeline_title: timelineTitle,
                     version,
@@ -183,6 +184,7 @@ export const importRulesRoute = (router: IRouter, config: LegacyServices['config
                         type,
                         threat,
                         references,
+                        note,
                         version,
                       });
                       resolve({ rule_id: ruleId, status_code: 200 });
@@ -217,6 +219,7 @@ export const importRulesRoute = (router: IRouter, config: LegacyServices['config
                         type,
                         threat,
                         references,
+                        note,
                         version,
                       });
                       resolve({ rule_id: ruleId, status_code: 200 });

x-pack/legacy/plugins/siem/server/lib/detection_engine/routes/rules/patch_rules_bulk_route.ts

@@ -71,6 +71,7 @@ export const patchRulesBulkRoute = (router: IRouter) => {
             type,
             threat,
             references,
+            note,
             version,
           } = payloadRule;
           const idOrRuleIdOrUnknown = id ?? ruleId ?? '(unknown id)';
@@ -104,6 +105,7 @@ export const patchRulesBulkRoute = (router: IRouter) => {
               type,
               threat,
               references,
+              note,
               version,
             });
             if (rule != null) {