[ML] Adds ML tasks to the kibana audit log (elastic#195120)

Adds a new `MlAuditLogger` service for logging calls to elasticsearch in
kibana's audit log.
Not all calls are logged, only ones which make changes to ML jobs or
trained models, e.g. creating, deleting, starting, stopping etc.

Calls to the es client are wrapped in a logging function so successes
and failures can be caught and logged.

the audit log can be enabed by adding this to the kibana yml or dev.yml
file
`xpack.security.audit.enabled: true`

An example log entry (NDJSON formatted to make it readable):
```
{
  "event": {
    "action": "ml_start_ad_datafeed",
    "type": [
      "change"
    ],
    "category": [
      "database"
    ],
    "outcome": "success"
  },
  "labels": {
    "application": "elastic/ml"
  },
  "user": {
    "id": "u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0",
    "name": "elastic",
    "roles": [
      "superuser"
    ]
  },
  "kibana": {
    "space_id": "default",
    "session_id": "U6HQCDkk+fAEUCXs7i4qM2/MZITPxE02pp8o7h09P68="
  },
  "trace": {
    "id": "4f1b616b-8535-43e1-8516-32ea9fe76d19"
  },
  "client": {
    "ip": "127.0.0.1"
  },
  "http": {
    "request": {
      "headers": {
        "x-forwarded-for": "127.0.0.1"
      }
    }
  },
  "service": {
    "node": {
      "roles": [
        "background_tasks",
        "ui"
      ]
    }
  },
  "ecs": {
    "version": "8.11.0"
  },
  "@timestamp": "2024-10-11T09:07:47.933+01:00",
  "message": "Starting anomaly detection datafeed datafeed-11aaaa",
  "log": {
    "level": "INFO",
    "logger": "plugins.security.audit.ecs"
  },
  "process": {
    "pid": 58305,
    "uptime": 100.982390291
  },
  "transaction": {
    "id": "77c14aadc6901324"
  }
}
```

---------

Co-authored-by: kibanamachine <[email protected]>
(cherry picked from commit 923c450)

docs/user/security/audit-logging.asciidoc

@@ -17,20 +17,20 @@ by cluster-wide privileges. For more information on enabling audit logging in
 Audit logs are **disabled** by default. To enable this functionality, you must
 set `xpack.security.audit.enabled` to `true` in `kibana.yml`.
 
-You can optionally configure audit logs location, file/rolling file appenders and 
+You can optionally configure audit logs location, file/rolling file appenders and
 ignore filters using <<audit-logging-settings>>.
 ============================================================================
 
 [[xpack-security-ecs-audit-logging]]
 ==== Audit events
 
-Refer to the table of events that can be logged for auditing purposes. 
+Refer to the table of events that can be logged for auditing purposes.
 
 Each event is broken down into <<field-event-category, category>>, <<field-event-type, type>>, <<field-event-action, action>> and
 <<field-event-outcome, outcome>> fields to make it easy to filter, query and aggregate the resulting logs. The <<field-trace-id, trace.id>>
 field can be used to correlate multiple events that originate from the same request.
 
-Refer to <<xpack-security-ecs-audit-schema>> for a table of fields that get logged with audit event. 
+Refer to <<xpack-security-ecs-audit-schema>> for a table of fields that get logged with audit event.
 
 [NOTE]
 ============================================================================
@@ -116,6 +116,38 @@ Refer to the corresponding {es} logs for potential write errors.
 .1+| `case_user_action_create_case`
 | `success` | User has created a case.
 
+.2+| `ml_put_ad_job`
+| `success` | Creating anomaly detection job.
+| `failure` | Failed to create anomaly detection job.
+
+.2+| `ml_put_ad_datafeed`
+| `success` | Creating anomaly detection datafeed.
+| `failure` | Failed to create anomaly detection datafeed.
+
+.2+| `ml_put_calendar`
+| `success` | Creating calendar.
+| `failure` | Failed to create calendar.
+
+.2+| `ml_post_calendar_events`
+| `success` | Adding events to calendar.
+| `failure` | Failed to add events to calendar.
+
+.2+| `ml_forecast`
+| `success` | Creating anomaly detection forecast.
+| `failure` | Failed to create anomaly detection forecast.
+
+.2+| `ml_put_filter`
+| `success` | Creating filter.
+| `failure` | Failed to create filter.
+
+.2+| `ml_put_dfa_job`
+| `success` | Creating data frame analytics job.
+| `failure` | Failed to create data frame analytics job.
+
+.2+| `ml_put_trained_model`
+| `success` | Creating trained model.
+| `failure` | Failed to create trained model.
+
 3+a|
 ====== Type: change
 
@@ -234,6 +266,74 @@ Refer to the corresponding {es} logs for potential write errors.
 .1+| `case_user_action_update_case_title`
 | `success` | User has updated the case title.
 
+.2+| `ml_open_ad_job`
+| `success` | Opening anomaly detection job.
+| `failure` | Failed to open anomaly detection job.
+
+.2+| `ml_close_ad_job`
+| `success` | Closing anomaly detection job.
+| `failure` | Failed to close anomaly detection job.
+
+.2+| `ml_start_ad_datafeed`
+| `success` | Starting anomaly detection datafeed.
+| `failure` | Failed to start anomaly detection datafeed.
+
+.2+| `ml_stop_ad_datafeed`
+| `success` | Stopping anomaly detection datafeed.
+| `failure` | Failed to stop anomaly detection datafeed.
+
+.2+| `ml_update_ad_job`
+| `success` | Updating anomaly detection job.
+| `failure` | Failed to update anomaly detection job.
+
+.2+| `ml_reset_ad_job`
+| `success` | Resetting anomaly detection job.
+| `failure` | Failed to reset anomaly detection job.
+
+.2+| `ml_revert_ad_snapshot`
+| `success` | Reverting anomaly detection snapshot.
+| `failure` | Failed to revert anomaly detection snapshot.
+
+.2+| `ml_update_ad_datafeed`
+| `success` | Updating anomaly detection datafeed.
+| `failure` | Failed to update anomaly detection datafeed.
+
+.2+| `ml_put_calendar_job`
+| `success` | Adding job to calendar.
+| `failure` | Failed to add job to calendar.
+
+.2+| `ml_delete_calendar_job`
+| `success` | Removing job from calendar.
+| `failure` | Failed to remove job from calendar.
+
+.2+| `ml_update_filter`
+| `success` | Updating filter.
+| `failure` | Failed to update filter.
+
+.2+| `ml_start_dfa_job`
+| `success` | Starting data frame analytics job.
+| `failure` | Failed to start data frame analytics job.
+
+.2+| `ml_stop_dfa_job`
+| `success` | Stopping data frame analytics job.
+| `failure` | Failed to stop data frame analytics job.
+
+.2+| `ml_update_dfa_job`
+| `success` | Updating data frame analytics job.
+| `failure` | Failed to update data frame analytics job.
+
+.2+| `ml_start_trained_model_deployment`
+| `success` | Starting trained model deployment.
+| `failure` | Failed to start trained model deployment.
+
+.2+| `ml_stop_trained_model_deployment`
+| `success` | Stopping trained model deployment.
+| `failure` | Failed to stop trained model deployment.
+
+.2+| `ml_update_trained_model_deployment`
+| `success` | Updating trained model deployment.
+| `failure` | Failed to update trained model deployment.
+
 3+a|
 ====== Type: deletion
 
@@ -289,6 +389,42 @@ Refer to the corresponding {es} logs for potential write errors.
 .1+| `case_user_action_delete_case_tags`
 | `success` | User has removed tags from a case.
 
+.2+| `ml_delete_ad_job`
+| `success` | Deleting anomaly detection job.
+| `failure` | Failed to delete anomaly detection job.
+
+.2+| `ml_delete_model_snapshot`
+| `success` | Deleting model snapshot.
+| `failure` | Failed to delete model snapshot.
+
+.2+| `ml_delete_ad_datafeed`
+| `success` | Deleting anomaly detection datafeed.
+| `failure` | Failed to delete anomaly detection datafeed.
+
+.2+| `ml_delete_calendar`
+| `success` | Deleting calendar.
+| `failure` | Failed to delete calendar.
+
+.2+| `ml_delete_calendar_event`
+| `success` | Deleting calendar event.
+| `failure` | Failed to delete calendar event.
+
+.2+| `ml_delete_filter`
+| `success` | Deleting filter.
+| `failure` | Failed to delete filter.
+
+.2+| `ml_delete_forecast`
+| `success` | Deleting forecast.
+| `failure` | Failed to delete forecast.
+
+.2+| `ml_delete_dfa_job`
+| `success` | Deleting data frame analytics job.
+| `failure` | Failed to delete data frame analytics job.
+
+.2+| `ml_delete_trained_model`
+| `success` | Deleting trained model.
+| `failure` | Failed to delete trained model.
+
 3+a|
 ====== Type: access
 
@@ -448,6 +584,10 @@ Refer to the corresponding {es} logs for potential write errors.
 | `success` | User has accessed the connectors of a case.
 | `failure` | User is not authorized to access the connectors of a case.
 
+.2+| `ml_infer_trained_model`
+| `success` | Inferring using trained model.
+| `failure` | Failed to infer using trained model.
+
 3+a|
 ===== Category: web
 
@@ -474,12 +614,12 @@ Audit logs are written in JSON using https://www.elastic.co/guide/en/ecs/1.6/ind
 | *Description*
 
 | `@timestamp`
-| Time when the event was generated. 
+| Time when the event was generated.
 
 Example: `2016-05-23T08:05:34.853Z`
 
 | `message`
-| Human readable description of the event. 
+| Human readable description of the event.
 
 2+a| ===== Event Fields
 
@@ -489,7 +629,7 @@ Example: `2016-05-23T08:05:34.853Z`
 | [[field-event-action]] `event.action`
 | The action captured by the event.
 
-Refer to <<xpack-security-ecs-audit-logging>> for a table of possible actions. 
+Refer to <<xpack-security-ecs-audit-logging>> for a table of possible actions.
 
 | [[field-event-category]] `event.category`
 | High level category associated with the event.
@@ -513,7 +653,7 @@ Possible values:
 `deletion`
 
 | [[field-event-outcome]] `event.outcome`
-a| Denotes whether the event represents a success or failure: 
+a| Denotes whether the event represents a success or failure:
 
 * Any actions that the user is not authorized to perform are logged with outcome:  `failure`
 * Authorized read operations are only logged after successfully fetching the data from {es} with outcome: `success`
@@ -553,7 +693,7 @@ Example: `[kibana_admin, reporting_user]`
 Example: `default`
 
 | `kibana.session_id`
-| ID of the user session associated with the event. 
+| ID of the user session associated with the event.
 
 Each login attempt results in a unique session id.
 
@@ -604,7 +744,7 @@ Example: `[marketing]`
 | Error code describing the error.
 
 | `error.message`
-| Error message. 
+| Error message.
 
 2+a| ===== HTTP and URL Fields
 

x-pack/plugins/ml/server/lib/ml_client/index.ts

@@ -8,3 +8,4 @@
 export { getMlClient } from './ml_client';
 export { MLJobNotFound, MLModelNotFound } from './errors';
 export type { MlClient } from './types';
+export { MlAuditLogger } from './ml_audit_logger';