[ML] Adds ML tasks to the kibana audit log #195120
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jgowdyelastic
commented
Oct 9, 2024
| @@ -25,28 +25,46 @@ | |||
| }, | |||
| // add references to other TypeScript projects the plugin depends on | |||
| "@kbn/actions-plugin", | |||
| "@kbn/aiops-change-point-detection", | |||
There was a problem hiding this comment.
sorting these alphabetically
jgowdyelastic
added
:ml
v9.0.0
release_note:feature
|
Pinging @elastic/ml-ui (:ml) |
peteharverson
approved these changes
Oct 11, 2024
darnautov
reviewed
Oct 11, 2024
| @@ -160,28 +162,44 @@ export function getMlClient( | |||
| return { | |||
| async closeJob(...p: Parameters<MlClient['closeJob']>) { | |||
| await jobIdsCheck('anomaly-detector', p); | |||
| return mlClient.closeJob(...p); | |||
| return auditLogger.wrapTask(() => mlClient.closeJob(...p), 'ml_close_ad_job', p); | |||
There was a problem hiding this comment.
Did you consider using Proxy instead of manually wrapping every call?
There was a problem hiding this comment.
I had not, it would probably be just as much code, but could be neater and mean we don't have changes in the ml_client file.
I'm not sure how we'd be able to intercept the calling of the function to log the error if it fails.
darnautov
approved these changes
Oct 11, 2024
There was a problem hiding this comment.
LGTM, but I reckon we should consider refactoring using Proxy or decorators for easier maintenance in the future.
peteharverson
changed the title
jgowdyelastic
added
release_note:enhancement
and removed
release_note:feature
💚 Build Succeeded
Metrics [docs]
History
|
|
Starting backport for target branches: 8.x https://github.com/elastic/kibana/actions/runs/11325840648 |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Oct 14, 2024
Adds a new `MlAuditLogger` service for logging calls to elasticsearch in
kibana's audit log.
Not all calls are logged, only ones which make changes to ML jobs or
trained models, e.g. creating, deleting, starting, stopping etc.
Calls to the es client are wrapped in a logging function so successes
and failures can be caught and logged.
the audit log can be enabed by adding this to the kibana yml or dev.yml
file
`xpack.security.audit.enabled: true`
An example log entry (NDJSON formatted to make it readable):
```
{
"event": {
"action": "ml_start_ad_datafeed",
"type": [
"change"
],
"category": [
"database"
],
"outcome": "success"
},
"labels": {
"application": "elastic/ml"
},
"user": {
"id": "u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0",
"name": "elastic",
"roles": [
"superuser"
]
},
"kibana": {
"space_id": "default",
"session_id": "U6HQCDkk+fAEUCXs7i4qM2/MZITPxE02pp8o7h09P68="
},
"trace": {
"id": "4f1b616b-8535-43e1-8516-32ea9fe76d19"
},
"client": {
"ip": "127.0.0.1"
},
"http": {
"request": {
"headers": {
"x-forwarded-for": "127.0.0.1"
}
}
},
"service": {
"node": {
"roles": [
"background_tasks",
"ui"
]
}
},
"ecs": {
"version": "8.11.0"
},
"@timestamp": "2024-10-11T09:07:47.933+01:00",
"message": "Starting anomaly detection datafeed datafeed-11aaaa",
"log": {
"level": "INFO",
"logger": "plugins.security.audit.ecs"
},
"process": {
"pid": 58305,
"uptime": 100.982390291
},
"transaction": {
"id": "77c14aadc6901324"
}
}
```
---------
Co-authored-by: kibanamachine <[email protected]>
(cherry picked from commit 923c450)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Oct 14, 2024
# Backport This will backport the following commits from `main` to `8.x`: - [[ML] Adds ML tasks to the kibana audit log (#195120)](#195120) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"James Gowdy","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-10-14T10:37:56Z","message":"[ML] Adds ML tasks to the kibana audit log (#195120)\n\nAdds a new `MlAuditLogger` service for logging calls to elasticsearch in\r\nkibana's audit log.\r\nNot all calls are logged, only ones which make changes to ML jobs or\r\ntrained models, e.g. creating, deleting, starting, stopping etc.\r\n\r\nCalls to the es client are wrapped in a logging function so successes\r\nand failures can be caught and logged.\r\n\r\nthe audit log can be enabed by adding this to the kibana yml or dev.yml\r\nfile\r\n`xpack.security.audit.enabled: true`\r\n\r\nAn example log entry (NDJSON formatted to make it readable):\r\n```\r\n{\r\n \"event\": {\r\n \"action\": \"ml_start_ad_datafeed\",\r\n \"type\": [\r\n \"change\"\r\n ],\r\n \"category\": [\r\n \"database\"\r\n ],\r\n \"outcome\": \"success\"\r\n },\r\n \"labels\": {\r\n \"application\": \"elastic/ml\"\r\n },\r\n \"user\": {\r\n \"id\": \"u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0\",\r\n \"name\": \"elastic\",\r\n \"roles\": [\r\n \"superuser\"\r\n ]\r\n },\r\n \"kibana\": {\r\n \"space_id\": \"default\",\r\n \"session_id\": \"U6HQCDkk+fAEUCXs7i4qM2/MZITPxE02pp8o7h09P68=\"\r\n },\r\n \"trace\": {\r\n \"id\": \"4f1b616b-8535-43e1-8516-32ea9fe76d19\"\r\n },\r\n \"client\": {\r\n \"ip\": \"127.0.0.1\"\r\n },\r\n \"http\": {\r\n \"request\": {\r\n \"headers\": {\r\n \"x-forwarded-for\": \"127.0.0.1\"\r\n }\r\n }\r\n },\r\n \"service\": {\r\n \"node\": {\r\n \"roles\": [\r\n \"background_tasks\",\r\n \"ui\"\r\n ]\r\n }\r\n },\r\n \"ecs\": {\r\n \"version\": \"8.11.0\"\r\n },\r\n \"@timestamp\": \"2024-10-11T09:07:47.933+01:00\",\r\n \"message\": \"Starting anomaly detection datafeed datafeed-11aaaa\",\r\n \"log\": {\r\n \"level\": \"INFO\",\r\n \"logger\": \"plugins.security.audit.ecs\"\r\n },\r\n \"process\": {\r\n \"pid\": 58305,\r\n \"uptime\": 100.982390291\r\n },\r\n \"transaction\": {\r\n \"id\": \"77c14aadc6901324\"\r\n }\r\n}\r\n```\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"923c450c1b044a12dd938c0c5ea380a895eeaf88","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement",":ml","v9.0.0","v8.16.0","backport:version"],"title":"[ML] Adds ML tasks to the kibana audit log","number":195120,"url":"https://github.com/elastic/kibana/pull/195120","mergeCommit":{"message":"[ML] Adds ML tasks to the kibana audit log (#195120)\n\nAdds a new `MlAuditLogger` service for logging calls to elasticsearch in\r\nkibana's audit log.\r\nNot all calls are logged, only ones which make changes to ML jobs or\r\ntrained models, e.g. creating, deleting, starting, stopping etc.\r\n\r\nCalls to the es client are wrapped in a logging function so successes\r\nand failures can be caught and logged.\r\n\r\nthe audit log can be enabed by adding this to the kibana yml or dev.yml\r\nfile\r\n`xpack.security.audit.enabled: true`\r\n\r\nAn example log entry (NDJSON formatted to make it readable):\r\n```\r\n{\r\n \"event\": {\r\n \"action\": \"ml_start_ad_datafeed\",\r\n \"type\": [\r\n \"change\"\r\n ],\r\n \"category\": [\r\n \"database\"\r\n ],\r\n \"outcome\": \"success\"\r\n },\r\n \"labels\": {\r\n \"application\": \"elastic/ml\"\r\n },\r\n \"user\": {\r\n \"id\": \"u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0\",\r\n \"name\": \"elastic\",\r\n \"roles\": [\r\n \"superuser\"\r\n ]\r\n },\r\n \"kibana\": {\r\n \"space_id\": \"default\",\r\n \"session_id\": \"U6HQCDkk+fAEUCXs7i4qM2/MZITPxE02pp8o7h09P68=\"\r\n },\r\n \"trace\": {\r\n \"id\": \"4f1b616b-8535-43e1-8516-32ea9fe76d19\"\r\n },\r\n \"client\": {\r\n \"ip\": \"127.0.0.1\"\r\n },\r\n \"http\": {\r\n \"request\": {\r\n \"headers\": {\r\n \"x-forwarded-for\": \"127.0.0.1\"\r\n }\r\n }\r\n },\r\n \"service\": {\r\n \"node\": {\r\n \"roles\": [\r\n \"background_tasks\",\r\n \"ui\"\r\n ]\r\n }\r\n },\r\n \"ecs\": {\r\n \"version\": \"8.11.0\"\r\n },\r\n \"@timestamp\": \"2024-10-11T09:07:47.933+01:00\",\r\n \"message\": \"Starting anomaly detection datafeed datafeed-11aaaa\",\r\n \"log\": {\r\n \"level\": \"INFO\",\r\n \"logger\": \"plugins.security.audit.ecs\"\r\n },\r\n \"process\": {\r\n \"pid\": 58305,\r\n \"uptime\": 100.982390291\r\n },\r\n \"transaction\": {\r\n \"id\": \"77c14aadc6901324\"\r\n }\r\n}\r\n```\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"923c450c1b044a12dd938c0c5ea380a895eeaf88"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/195120","number":195120,"mergeCommit":{"message":"[ML] Adds ML tasks to the kibana audit log (#195120)\n\nAdds a new `MlAuditLogger` service for logging calls to elasticsearch in\r\nkibana's audit log.\r\nNot all calls are logged, only ones which make changes to ML jobs or\r\ntrained models, e.g. creating, deleting, starting, stopping etc.\r\n\r\nCalls to the es client are wrapped in a logging function so successes\r\nand failures can be caught and logged.\r\n\r\nthe audit log can be enabed by adding this to the kibana yml or dev.yml\r\nfile\r\n`xpack.security.audit.enabled: true`\r\n\r\nAn example log entry (NDJSON formatted to make it readable):\r\n```\r\n{\r\n \"event\": {\r\n \"action\": \"ml_start_ad_datafeed\",\r\n \"type\": [\r\n \"change\"\r\n ],\r\n \"category\": [\r\n \"database\"\r\n ],\r\n \"outcome\": \"success\"\r\n },\r\n \"labels\": {\r\n \"application\": \"elastic/ml\"\r\n },\r\n \"user\": {\r\n \"id\": \"u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0\",\r\n \"name\": \"elastic\",\r\n \"roles\": [\r\n \"superuser\"\r\n ]\r\n },\r\n \"kibana\": {\r\n \"space_id\": \"default\",\r\n \"session_id\": \"U6HQCDkk+fAEUCXs7i4qM2/MZITPxE02pp8o7h09P68=\"\r\n },\r\n \"trace\": {\r\n \"id\": \"4f1b616b-8535-43e1-8516-32ea9fe76d19\"\r\n },\r\n \"client\": {\r\n \"ip\": \"127.0.0.1\"\r\n },\r\n \"http\": {\r\n \"request\": {\r\n \"headers\": {\r\n \"x-forwarded-for\": \"127.0.0.1\"\r\n }\r\n }\r\n },\r\n \"service\": {\r\n \"node\": {\r\n \"roles\": [\r\n \"background_tasks\",\r\n \"ui\"\r\n ]\r\n }\r\n },\r\n \"ecs\": {\r\n \"version\": \"8.11.0\"\r\n },\r\n \"@timestamp\": \"2024-10-11T09:07:47.933+01:00\",\r\n \"message\": \"Starting anomaly detection datafeed datafeed-11aaaa\",\r\n \"log\": {\r\n \"level\": \"INFO\",\r\n \"logger\": \"plugins.security.audit.ecs\"\r\n },\r\n \"process\": {\r\n \"pid\": 58305,\r\n \"uptime\": 100.982390291\r\n },\r\n \"transaction\": {\r\n \"id\": \"77c14aadc6901324\"\r\n }\r\n}\r\n```\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"923c450c1b044a12dd938c0c5ea380a895eeaf88"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: James Gowdy <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds a new
MlAuditLoggerservice for logging calls to elasticsearch in kibana's audit log.Not all calls are logged, only ones which make changes to ML jobs or trained models, e.g. creating, deleting, starting, stopping etc.
Calls to the es client are wrapped in a logging function so successes and failures can be caught and logged.
the audit log can be enabed by adding this to the kibana yml or dev.yml file
xpack.security.audit.enabled: trueAn example log entry (NDJSON formatted to make it readable):