diff --git a/packages/mongodb_atlas/_dev/build/docs/README.md b/packages/mongodb_atlas/_dev/build/docs/README.md index 10775b512a0..1d89ec5e88c 100644 --- a/packages/mongodb_atlas/_dev/build/docs/README.md +++ b/packages/mongodb_atlas/_dev/build/docs/README.md @@ -6,22 +6,25 @@ Use the MongoDB Atlas integration to: -- Collect metrics related to process. +- Collect MongoDB Audit logs and Process metrics for comprehensive monitoring and analysis. - Create informative visualizations to track usage trends, measure key metrics, and derive actionable business insights. - Set up alerts to minimize Mean Time to Detect (MTTD) and Mean Time to Resolve (MTTR) by quickly referencing relevant logs during troubleshooting. ## Data streams -The MongoDB Atlas integration collects metrics. +The MongoDB Atlas integration collects logs and metrics. + +Logs help you keep a record of events that happen on your machine. The `Log` data stream collected by MongoDB Atlas integration is `mongod_audit`. Metrics give you insight into the statistics of the MongoDB Atlas. The `Metric` data stream collected by the MongoDB Atlas integration is `process` so that the user can monitor and troubleshoot the performance of the MongoDB Atlas instance. Data streams: +- `mongod_audit`: The auditing facility allows administrators and users to track system activity for deployments with multiple users and applications. Mongod Audit logs capture events related to database operations such as insertions, updates, deletions, user authentication, etc., occurring within the mongod instances. - `process` : This data stream collects host metrics per process for all the hosts of the specified group. Metrics like measurements for the host, such as CPU usage, number of I/O operations and memory are available on this data stream. Note: -- Users can monitor and see the metrics inside the ingested documents for MongoDB Atlas in the `logs-*` index pattern from `Discover`. +- Users can monitor and see the log inside the ingested documents for MongoDB Atlas in the `logs-*` index pattern from `Discover`. ## Prerequisites @@ -29,25 +32,26 @@ You can store and search your data using Elasticsearch and visualize and manage ## Setup -### To collect data from MongoDB Atlas, the following parameters from your MongoDB Atlas instance are required: +### To collect data from MongoDB Atlas, the following parameters from your MongoDB Atlas instance are required 1. Public Key 2. Private Key 3. GroupId -### Steps to obtain Public Key, Private Key and GroupId: +### Steps to obtain Public Key, Private Key and GroupId 1. Generate programmatic API Keys with project owner permissions using the instructions in the Atlas [documentation](https://www.mongodb.com/docs/atlas/configure-api-access/#create-an-api-key-for-a-project). Then, copy the public key and private key. These serve the same function as a username and API Key respectively. -2. You can find your Project ID (Group ID) in the Atlas UI. To do this, navigate to your project, click on Settings, and copy the Project ID (Group ID). You can also programmatically find it using the Atlas Admin API or Atlas CLI as described in this Atlas [document](https://www.mongodb.com/docs/atlas/app-services/apps/metadata/#find-a-project-id). +2. Enable Database Auditing for the Atlas project you want to monitor logs. You can follow the instructions provided in this Atlas [document](https://www.mongodb.com/docs/atlas/database-auditing/#procedure). +3. You can find your Project ID (Group ID) in the Atlas UI. To do this, navigate to your project, click on Settings, and copy the Project ID (Group ID). You can also programmatically find it using the Atlas Admin API or Atlas CLI as described in this Atlas [document](https://www.mongodb.com/docs/atlas/app-services/apps/metadata/#find-a-project-id). -### Important terms of MongoDB Atlas API: +### Important terms of MongoDB Atlas API 1. Granularity: Duration that specifies the interval at which Atlas reports the metrics. 2. Period: Duration over which Atlas reports the metrics. Note: Both of above attributes can be set by using `period` in configuration parameters. -### Enabling the integration in Elastic: +### Steps to enable Integration in Elastic 1. In Kibana go to Management > Integrations 2. In "Search for integrations" search bar, type MongoDB Atlas @@ -56,6 +60,31 @@ Note: Both of above attributes can be set by using `period` in configuration par 5. Enter all the necessary configuration parameters, including Public Key, Private Key, and GroupId. 6. Finally, save the integration. +Note: +- The `mongod_audit` data stream gathers historical data spanning the previous 30 minutes. +- Mongod: Mongod is the primary daemon method for the MongoDB system. It helps in handling the data requests, managing the data access, performing background management operations, and other core database operations. + +## Troubleshooting + +If you encounter an error while ingesting data, it might be due to the data collected over a long time span. Generating a response in such cases may take longer and might cause a request timeout if the `HTTP Client Timeout` parameter is set to a small duration. To avoid this error, it is recommended to adjust the `HTTP Client Timeout` and `Interval` parameters based on the duration of data collection. +``` +{ + "error": { + "message": "failed eval: net/http: request canceled (Client.Timeout or context cancellation while reading body)" + } +} +``` + +## Logs reference + +### Mongod Audit + +This is the `mongod_audit` data stream. This data stream allows administrators and users to track system activity for deployments with multiple users and applications. + +{{event "mongod_audit"}} + +{{fields "mongod_audit"}} + ## Metrics reference ### Process diff --git a/packages/mongodb_atlas/_dev/deploy/docker/Dockerfile b/packages/mongodb_atlas/_dev/deploy/docker/Dockerfile index dfc9e6809c8..46682dba77e 100644 --- a/packages/mongodb_atlas/_dev/deploy/docker/Dockerfile +++ b/packages/mongodb_atlas/_dev/deploy/docker/Dockerfile @@ -1,6 +1,6 @@ FROM golang:1.19 -COPY ./test /go +COPY ./mongodb_atlas /go CMD ["./test"] HEALTHCHECK --interval=1s --retries=90 CMD curl localhost:7780/api/atlas/v2/groups/mongodb-group1/processes --digest -u admin:MongoDB@123 \ No newline at end of file diff --git a/packages/mongodb_atlas/_dev/deploy/docker/mongodb_atlas/mongod_audit_data.log b/packages/mongodb_atlas/_dev/deploy/docker/mongodb_atlas/mongod_audit_data.log new file mode 100755 index 00000000000..7660872b355 --- /dev/null +++ b/packages/mongodb_atlas/_dev/deploy/docker/mongodb_atlas/mongod_audit_data.log @@ -0,0 +1,5 @@ +{ "atype": "authenticate", "ts": { "$date": "2023-04-01T12:00:00.000Z" }, "uuid": { "$binary": "some-unique-identifier", "$type": "04" }, "local": { "ip": "127.0.0.1", "port": 27017 }, "remote": { "ip": "192.168.1.100", "port": 54320 }, "users": [{ "user": "auditUser", "db": "admin" }], "roles": [{ "role": "dbAdmin", "db": "admin" }], "result": 0 } +{ "atype": "authCheck", "ts": { "$date": "2023-04-01T12:05:00.000Z" }, "uuid": { "$binary": "another-unique-identifier", "$type": "04" }, "local": { "ip": "127.0.0.1", "port": 27017 }, "remote": { "ip": "192.168.1.101", "port": 54321 }, "users": [{ "user": "userTest", "db": "test" }], "roles": [{ "role": "read", "db": "test" }], "result": 13 } +{ "atype": "createIndex", "ts": { "$date": "2023-04-01T12:10:00.000Z" }, "uuid": { "$binary": "yet-another-unique-identifier", "$type": "04" }, "local": { "ip": "127.0.0.1", "port": 27017 }, "remote": { "ip": "192.168.1.102", "port": 54322 }, "users": [{ "user": "indexManager", "db": "test" }], "roles": [{ "role": "dbOwner", "db": "test" }], "result": 0 } +{ "atype": "dropCollection", "ts": { "$date": "2023-04-01T12:15:00.000Z" }, "uuid": { "$binary": "unique-identifier-drop-coll", "$type": "04" }, "local": { "ip": "127.0.0.1", "port": 27017 }, "remote": { "ip": "192.168.1.103", "port": 54323 }, "users": [{ "user": "adminUser", "db": "test" }], "roles": [{ "role": "dbAdmin", "db": "test" }], "result": 0 } +{ "atype": "createUser", "ts": { "$date": "2023-04-01T12:20:00.000Z" }, "uuid": { "$binary": "unique-identifier-create-user", "$type": "04" }, "local": { "ip": "127.0.0.1", "port": 27017 }, "remote": { "ip": "192.168.1.104", "port": 54324 }, "users": [{ "user": "admin", "db": "admin" }], "roles": [{ "role": "userAdmin", "db": "admin" }], "result": 0 } \ No newline at end of file diff --git a/packages/mongodb_atlas/_dev/deploy/docker/test b/packages/mongodb_atlas/_dev/deploy/docker/mongodb_atlas/test similarity index 56% rename from packages/mongodb_atlas/_dev/deploy/docker/test rename to packages/mongodb_atlas/_dev/deploy/docker/mongodb_atlas/test index 1a430970479..10c9c4b31e8 100755 Binary files a/packages/mongodb_atlas/_dev/deploy/docker/test and b/packages/mongodb_atlas/_dev/deploy/docker/mongodb_atlas/test differ diff --git a/packages/mongodb_atlas/changelog.yml b/packages/mongodb_atlas/changelog.yml index 3ee5328478c..10de1cb5417 100644 --- a/packages/mongodb_atlas/changelog.yml +++ b/packages/mongodb_atlas/changelog.yml @@ -1,4 +1,9 @@ # newer versions go on top +- version: "0.0.2" + changes: + - description: MongoDB Atlas integration package with "mongod_audit" data stream. + type: enhancement + link: https://github.com/elastic/integrations/pull/9020 - version: "0.0.1" changes: - description: MongoDB Atlas integration package with "process" data stream. diff --git a/packages/mongodb_atlas/data_stream/mongod_audit/_dev/test/pipeline/test-common-config.yml b/packages/mongodb_atlas/data_stream/mongod_audit/_dev/test/pipeline/test-common-config.yml new file mode 100644 index 00000000000..a06dda74051 --- /dev/null +++ b/packages/mongodb_atlas/data_stream/mongod_audit/_dev/test/pipeline/test-common-config.yml @@ -0,0 +1,5 @@ +dynamic_fields: + "event.ingested": ".*" +fields: + tags: + - preserve_original_event diff --git a/packages/mongodb_atlas/data_stream/mongod_audit/_dev/test/pipeline/test-mongod-audit.json b/packages/mongodb_atlas/data_stream/mongod_audit/_dev/test/pipeline/test-mongod-audit.json new file mode 100644 index 00000000000..a62eb488bca --- /dev/null +++ b/packages/mongodb_atlas/data_stream/mongod_audit/_dev/test/pipeline/test-mongod-audit.json @@ -0,0 +1,20 @@ +{ + "events": [ + { + "@timestamp": "2024-02-08T06:20:49.729Z", + "message": "No data for given time period or host is unreachable" + }, + { + "@timestamp": "2024-02-08T06:20:56.621Z", + "message": "{ \"atype\": \"logout\", \"ts\": { \"$date\": \"2024-01-29T06:57:15.366+00:00\" }, \"uuid\": { \"$binary\": \"bY/PMV8IR36q+hmAJZYyfw==\", \"$type\": \"04\" }, \"local\": { \"ip\": \"127.0.0.1\", \"port\": 27017 }, \"remote\": { \"ip\": \"127.0.0.1\", \"port\": 43714 }, \"users\":[ { \"user\":\"mms-monitoring-agent\", \"db\":\"admin\" } ], \"roles\": [ { \"role\": \"backup\", \"db\": \"admin\" }, { \"role\": \"clusterAdmin\", \"db\": \"admin\" }, { \"role\": \"dbAdminAnyDatabase\", \"db\": \"admin\" }, { \"role\": \"readWriteAnyDatabase\", \"db\": \"admin\" }, { \"role\": \"restore\", \"db\": \"admin\" }, { \"role\": \"userAdminAnyDatabase\", \"db\": \"admin\" } ], \"result\": 0 }" + }, + { + "@timestamp": "2024-02-08T07:20:00.123Z", + "message": "{\"atype\":\"authenticate\",\"ts\":{\"$date\":\"2023-04-01T12:00:00.000Z\"},\"uuid\":{\"$binary\":\"some-unique-identifier\",\"$type\":\"04\"},\"local\":{\"ip\":\"127.0.0.1\",\"port\":27017},\"remote\":{\"ip\":\"192.168.1.100\",\"port\":54320},\"users\":[{\"user\":\"auditUser\",\"db\":\"admin\"}],\"roles\":[{\"role\":\"dbAdmin\",\"db\":\"admin\"}],\"result\":0}" + }, + { + "@timestamp": "2024-02-08T07:30:56.234Z", + "message": "{\"atype\":\"authCheck\",\"ts\":{\"$date\":\"2023-04-01T12:05:00.000Z\"},\"uuid\":{\"$binary\":\"another-unique-identifier\",\"$type\":\"04\"},\"local\":{\"ip\":\"127.0.0.1\",\"port\":27017},\"remote\":{\"ip\":\"192.168.1.101\",\"port\":54321},\"users\":[{\"user\":\"userTest\",\"db\":\"test\"}],\"roles\":[{\"role\":\"read\",\"db\":\"test\"}],\"result\":13}" + } + ] +} \ No newline at end of file diff --git a/packages/mongodb_atlas/data_stream/mongod_audit/_dev/test/pipeline/test-mongod-audit.json-expected.json b/packages/mongodb_atlas/data_stream/mongod_audit/_dev/test/pipeline/test-mongod-audit.json-expected.json new file mode 100644 index 00000000000..73f8b0ee833 --- /dev/null +++ b/packages/mongodb_atlas/data_stream/mongod_audit/_dev/test/pipeline/test-mongod-audit.json-expected.json @@ -0,0 +1,204 @@ +{ + "expected": [ + null, + { + "@timestamp": "2024-01-29T06:57:15.366Z", + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "logout", + "category": [ + "network", + "authentication" + ], + "kind": "event", + "module": "mongodb_atlas", + "original": "{ \"atype\": \"logout\", \"ts\": { \"$date\": \"2024-01-29T06:57:15.366+00:00\" }, \"uuid\": { \"$binary\": \"bY/PMV8IR36q+hmAJZYyfw==\", \"$type\": \"04\" }, \"local\": { \"ip\": \"127.0.0.1\", \"port\": 27017 }, \"remote\": { \"ip\": \"127.0.0.1\", \"port\": 43714 }, \"users\":[ { \"user\":\"mms-monitoring-agent\", \"db\":\"admin\" } ], \"roles\": [ { \"role\": \"backup\", \"db\": \"admin\" }, { \"role\": \"clusterAdmin\", \"db\": \"admin\" }, { \"role\": \"dbAdminAnyDatabase\", \"db\": \"admin\" }, { \"role\": \"readWriteAnyDatabase\", \"db\": \"admin\" }, { \"role\": \"restore\", \"db\": \"admin\" }, { \"role\": \"userAdminAnyDatabase\", \"db\": \"admin\" } ], \"result\": 0 }", + "type": [ + "access", + "info" + ] + }, + "mongodb_atlas": { + "mongod_audit": { + "local": { + "ip": "127.0.0.1", + "port": 27017 + }, + "remote": { + "ip": "127.0.0.1", + "port": 43714 + }, + "result": "Success", + "user": { + "names": [ + { + "db": "admin", + "user": "mms-monitoring-agent" + } + ], + "roles": [ + { + "db": "admin", + "role": "backup" + }, + { + "db": "admin", + "role": "clusterAdmin" + }, + { + "db": "admin", + "role": "dbAdminAnyDatabase" + }, + { + "db": "admin", + "role": "readWriteAnyDatabase" + }, + { + "db": "admin", + "role": "restore" + }, + { + "db": "admin", + "role": "userAdminAnyDatabase" + } + ] + }, + "uuid": { + "binary": "bY/PMV8IR36q+hmAJZYyfw==", + "type": "04" + } + } + }, + "related": { + "ip": [ + "127.0.0.1" + ] + }, + "tags": [ + "preserve_original_event" + ] + }, + { + "@timestamp": "2023-04-01T12:00:00.000Z", + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "authenticate", + "category": [ + "network", + "authentication" + ], + "kind": "event", + "module": "mongodb_atlas", + "original": "{\"atype\":\"authenticate\",\"ts\":{\"$date\":\"2023-04-01T12:00:00.000Z\"},\"uuid\":{\"$binary\":\"some-unique-identifier\",\"$type\":\"04\"},\"local\":{\"ip\":\"127.0.0.1\",\"port\":27017},\"remote\":{\"ip\":\"192.168.1.100\",\"port\":54320},\"users\":[{\"user\":\"auditUser\",\"db\":\"admin\"}],\"roles\":[{\"role\":\"dbAdmin\",\"db\":\"admin\"}],\"result\":0}", + "type": [ + "access", + "info" + ] + }, + "mongodb_atlas": { + "mongod_audit": { + "local": { + "ip": "127.0.0.1", + "port": 27017 + }, + "remote": { + "ip": "192.168.1.100", + "port": 54320 + }, + "result": "Success", + "user": { + "names": [ + { + "db": "admin", + "user": "auditUser" + } + ], + "roles": [ + { + "db": "admin", + "role": "dbAdmin" + } + ] + }, + "uuid": { + "binary": "some-unique-identifier", + "type": "04" + } + } + }, + "related": { + "ip": [ + "127.0.0.1", + "192.168.1.100" + ] + }, + "tags": [ + "preserve_original_event" + ] + }, + { + "@timestamp": "2023-04-01T12:05:00.000Z", + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "authCheck", + "category": [ + "network", + "authentication" + ], + "kind": "event", + "module": "mongodb_atlas", + "original": "{\"atype\":\"authCheck\",\"ts\":{\"$date\":\"2023-04-01T12:05:00.000Z\"},\"uuid\":{\"$binary\":\"another-unique-identifier\",\"$type\":\"04\"},\"local\":{\"ip\":\"127.0.0.1\",\"port\":27017},\"remote\":{\"ip\":\"192.168.1.101\",\"port\":54321},\"users\":[{\"user\":\"userTest\",\"db\":\"test\"}],\"roles\":[{\"role\":\"read\",\"db\":\"test\"}],\"result\":13}", + "type": [ + "access", + "info" + ] + }, + "mongodb_atlas": { + "mongod_audit": { + "local": { + "ip": "127.0.0.1", + "port": 27017 + }, + "remote": { + "ip": "192.168.1.101", + "port": 54321 + }, + "result": "Unauthorized to perform the operation", + "user": { + "names": [ + { + "db": "test", + "user": "userTest" + } + ], + "roles": [ + { + "db": "test", + "role": "read" + } + ] + }, + "uuid": { + "binary": "another-unique-identifier", + "type": "04" + } + } + }, + "related": { + "ip": [ + "127.0.0.1", + "192.168.1.101" + ] + }, + "tags": [ + "preserve_original_event" + ] + } + ] +} \ No newline at end of file diff --git a/packages/mongodb_atlas/data_stream/mongod_audit/_dev/test/system/test-default-config.yml b/packages/mongodb_atlas/data_stream/mongod_audit/_dev/test/system/test-default-config.yml new file mode 100644 index 00000000000..19fe7ae9a87 --- /dev/null +++ b/packages/mongodb_atlas/data_stream/mongod_audit/_dev/test/system/test-default-config.yml @@ -0,0 +1,13 @@ +vars: + url: + - http://{{Hostname}}:{{Port}} + public_key: + - admin + private_key: + - MongoDB@123 +data_stream: + vars: + groupId: + - mongodb-group1 +input: cel +service: mongodbatlas diff --git a/packages/mongodb_atlas/data_stream/mongod_audit/agent/stream/input.yml.hbs b/packages/mongodb_atlas/data_stream/mongod_audit/agent/stream/input.yml.hbs new file mode 100644 index 00000000000..b45dff66716 --- /dev/null +++ b/packages/mongodb_atlas/data_stream/mongod_audit/agent/stream/input.yml.hbs @@ -0,0 +1,107 @@ +config_version: 2 +interval: {{interval}} +{{#if enable_request_tracer}} +resource.tracer.filename: "../../logs/cel/http-request-trace-*.ndjson" +{{/if}} +{{#if ssl}} +resource.ssl: {{ssl}} +{{/if}} +{{#if http_client_timeout}} +resource.timeout: {{http_client_timeout}} +{{/if}} +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +{{#if processors}} +processors: +{{processors}} +{{/if}} +auth.digest: + user: {{public_key}} + password: {{private_key}} +resource.url: {{url}} +state: + groupId: {{groupId}} + want_more: false + page_num: 1 +redact: + fields: ~ +program: | + ( + ( + has(state.hostlist) && size(state.hostlist) > 0 + ? + state + : + ( + state.page_num != 1 + ? + state + : + state.with({ + "startDate": state.?cursor.last_timestamp.orValue(int(now - duration("30m"))), + "endDate": int(now) + })).as(state, state.with(request("GET", state.url + "/api/atlas/v2/groups/" + state.groupId + "/processes?pageNum=" + string(state.page_num) + "&itemsPerPage=100").with({ + "Header": { + "Accept": ["application/vnd.atlas." + string(now.getFullYear()) + "-01-01+gzip"] + } + }).do_request().as(resp, + bytes(resp.Body).decode_json().as(body, { + "hostlist": body.results.map(e, e.hostname), + "next": 0, + "page_num": body.links.exists_one(res,res.rel=="next") ? (int(state.page_num)+1) : 1 + }) + ))) + ).as(state, state.next < size(state.hostlist) + ? + ( + request("GET", state.url + "/api/atlas/v2/groups/" + state.groupId + "/clusters/" + state.hostlist[state.next] + "/logs/mongodb-audit-log.gz?startDate=" + string(int(state.startDate)) + "&endDate=" + string(int(state.endDate)) + ).with({ + "Header": { + "Accept": ["application/vnd.atlas." + string(now.getFullYear()) + "-01-01+gzip"] + } + }).do_request().as(resp, ( + resp.StatusCode == 200 && resp.ContentLength != 0 + ? + string(bytes(resp.Body).mime('application/gzip')).as(body,{ + "events": body.trim_space().split("\n").map(value,{"message":value, "host_name": state.hostlist[state.next]}), + "cursor": { + "last_timestamp": state.endDate + }, + "hostlist": (int(state.next)+1) < size(state.hostlist) ? state.hostlist : [], + "next": (int(state.next)+1) < size(state.hostlist) ? (int(state.next)+1) : 0, + "want_more": ((int(state.next)+1) < size(state.hostlist) || state.page_num != 1), + "page_num": state.page_num, + "startDate": state.startDate, + "endDate": state.endDate, + "groupId": state.groupId, + }) + : + { + // If data is not available during this period, or if the host is unreachable, + // an event will be generated with the following message, and this event will be dropped during pipeline processing. + // keep this message in a sync with pipeline as it is being used there. + "events": [{"message":"No data for given time period or host is unreachable"}], + "cursor": { + "last_timestamp": state.endDate + }, + "hostlist": (int(state.next)+1) < size(state.hostlist) ? state.hostlist : [], + "next": (int(state.next)+1) < size(state.hostlist) ? (int(state.next)+1) : 0, + "want_more": ((int(state.next)+1) < size(state.hostlist) || state.page_num != 1), + "page_num": state.page_num, + "startDate": state.startDate, + "endDate": state.endDate, + "groupId": state.groupId, + } + ))) + : + {} + ) + ) diff --git a/packages/mongodb_atlas/data_stream/mongod_audit/elasticsearch/ingest_pipeline/default.yml b/packages/mongodb_atlas/data_stream/mongod_audit/elasticsearch/ingest_pipeline/default.yml new file mode 100644 index 00000000000..5e62d2ca2f5 --- /dev/null +++ b/packages/mongodb_atlas/data_stream/mongod_audit/elasticsearch/ingest_pipeline/default.yml @@ -0,0 +1,164 @@ +--- +description: Pipeline for processing MongoDB Atlas Mongod Audit logs. +processors: + - drop: + if: ctx.message == "No data for given time period or host is unreachable" + tag: drop_empty_events + - set: + field: ecs.version + value: 8.11.0 + - set: + field: event.kind + value: event + - set: + field: event.module + value: mongodb_atlas + - set: + field: event.category + value: ["network", "authentication"] + - set: + field: event.type + value: ["access", "info"] + - rename: + field: message + target_field: event.original + ignore_missing: true + if: 'ctx.event?.original == null' + - rename: + field: host_name + target_field: mongodb_atlas.mongod_audit.hostname + ignore_missing: true + - json: + field: event.original + target_field: json + tag: json_decoding + on_failure: + - append: + field: error.message + value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' + - date: + field: json.ts.$date + target_field: '@timestamp' + formats: [ISO8601] + on_failure: + - append: + field: error.message + value: '{{ _ingest.on_failure_message }}' + - append: + field: related.ip + value: '{{{json.local.ip}}}' + if: ctx.json?.local?.ip != null + allow_duplicates: false + - append: + field: related.ip + value: '{{{json.remote.ip}}}' + if: ctx.json?.remote?.ip != null + allow_duplicates: false + - rename: + field: json.atype + target_field: event.action + ignore_missing: true + - rename: + field: json.local.port + target_field: mongodb_atlas.mongod_audit.local.port + ignore_missing: true + - rename: + field: json.local.ip + target_field: mongodb_atlas.mongod_audit.local.ip + ignore_missing: true + - rename: + field: json.local.isSystemUser + target_field: mongodb_atlas.mongod_audit.local.is_system_user + ignore_missing: true + - rename: + field: json.local.unix + target_field: mongodb_atlas.mongod_audit.local.unix + ignore_missing: true + - rename: + field: json.remote.port + target_field: mongodb_atlas.mongod_audit.remote.port + ignore_missing: true + - rename: + field: json.remote.ip + target_field: mongodb_atlas.mongod_audit.remote.ip + ignore_missing: true + - rename: + field: json.remote.isSystemUser + target_field: mongodb_atlas.mongod_audit.remote.is_system_user + ignore_missing: true + - rename: + field: json.remote.unix + target_field: mongodb_atlas.mongod_audit.remote.unix + ignore_missing: true + - rename: + field: json.param + target_field: mongodb_atlas.mongod_audit.param + ignore_missing: true + - rename: + field: json.users + target_field: mongodb_atlas.mongod_audit.user.names + ignore_missing: true + - rename: + field: json.roles + target_field: mongodb_atlas.mongod_audit.user.roles + ignore_missing: true + - rename: + field: json.uuid.$binary + target_field: mongodb_atlas.mongod_audit.uuid.binary + ignore_missing: true + - rename: + field: json.uuid.$type + target_field: mongodb_atlas.mongod_audit.uuid.type + ignore_missing: true + - script: + description: Provide information of the error code. + lang: painless + ignore_failure: true + source: | + Map m = new HashMap(); + m.put(0, "Success"); + m.put(13, "Unauthorized to perform the operation"); + m.put(18, "Authentication Failed"); + m.put(26, "Namespace Not Found"); + m.put(276, "Index build aborted"); + m.put(334, "Unauthorized to perform the operation"); + ctx.mongodb_atlas.mongod_audit.result = m.get(ctx.json.result); + - script: + lang: painless + source: |- + boolean drop(Object o) { + if (o == null || o == '') { + return true; + } else if (o instanceof Map) { + ((Map) o).values().removeIf(v -> drop(v)); + return (((Map) o).size() == 0); + } else if (o instanceof List) { + ((List) o).removeIf(v -> drop(v)); + return (((List) o).length == 0); + } + return false; + } + drop(ctx); + description: Drops null/empty values recursively. + - remove: + field: + - event.original + ignore_missing: true + if: ctx.tags == null || !(ctx.tags.contains('preserve_original_event')) + - remove: + field: + - json + ignore_missing: true + - set: + field: event.kind + value: pipeline_error + if: ctx.error?.message != null +on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - append: + field: event.kind + value: pipeline_error + allow_duplicates: false + diff --git a/packages/mongodb_atlas/data_stream/mongod_audit/fields/base-fields.yml b/packages/mongodb_atlas/data_stream/mongod_audit/fields/base-fields.yml new file mode 100644 index 00000000000..abc42103fc8 --- /dev/null +++ b/packages/mongodb_atlas/data_stream/mongod_audit/fields/base-fields.yml @@ -0,0 +1,18 @@ +- name: data_stream.type + type: constant_keyword + description: Data stream type. +- name: data_stream.dataset + type: constant_keyword + description: Data stream dataset. +- name: data_stream.namespace + type: constant_keyword + description: Data stream namespace. +- name: input.type + type: keyword + description: Type of Filebeat input. +- name: tags + type: keyword + description: List of keywords used to tag each event. +- name: '@timestamp' + type: date + description: Event timestamp. diff --git a/packages/mongodb_atlas/data_stream/mongod_audit/fields/fields.yml b/packages/mongodb_atlas/data_stream/mongod_audit/fields/fields.yml new file mode 100644 index 00000000000..f2e70e42e98 --- /dev/null +++ b/packages/mongodb_atlas/data_stream/mongod_audit/fields/fields.yml @@ -0,0 +1,63 @@ +- name: mongodb_atlas.mongod_audit + type: group + fields: + - name: hostname + type: keyword + description: Hostname that stores the log files that you want to download. + - name: local + type: group + fields: + - name: ip + type: ip + description: IP address of the running instance. + - name: is_system_user + type: boolean + description: True if the event is caused by a system user, false otherwise. + - name: port + type: long + description: Port number of the running instance. + - name: unix + type: keyword + description: Unix that contains the MongoDB socket file path if the client connects through a Unix domain socket. + - name: param + type: object + object_type: keyword + description: Specific details for the event. + - name: remote + type: group + fields: + - name: is_system_user + type: boolean + description: True if the event is caused by a system user, false otherwise. + - name: ip + type: ip + description: IP address of the incoming connection associated with the event. + - name: port + type: long + description: Port number of the incoming connection associated with the event. + - name: unix + type: keyword + description: Unix that contains the MongoDB socket file path if the client connects through a Unix domain socket. + - name: result + type: keyword + description: Error code. + - name: user + type: group + fields: + - name: names + type: object + object_type: keyword + description: Array of user identification documents. + - name: roles + type: object + object_type: keyword + description: Array of documents that specify the roles granted to the user. + - name: uuid + type: group + fields: + - name: binary + type: keyword + description: Document that contains a universally unique identifier (UUID) for the audit message. + - name: type + type: keyword + description: The $type field specifies the BSON subtype for the $binary field. diff --git a/packages/mongodb_atlas/data_stream/mongod_audit/manifest.yml b/packages/mongodb_atlas/data_stream/mongod_audit/manifest.yml new file mode 100644 index 00000000000..70b8bc2c198 --- /dev/null +++ b/packages/mongodb_atlas/data_stream/mongod_audit/manifest.yml @@ -0,0 +1,62 @@ +title: Collect Mongod Audit logs from MongoDB Atlas +type: logs +streams: + - input: cel + title: MongoDB Atlas Mongod Audit logs + description: Collect Mongod Audit logs from MongoDB Atlas. + template_path: input.yml.hbs + vars: + - name: groupId + type: text + title: GroupId + description: Identifier string that identifies your project. Groups and projects are synonymous terms. Your group id is the same as your project id. Ex. 32b6e34b3d91647abb20e7b8 + multi: false + required: true + show_user: true + - name: interval + type: text + title: Interval + description: Duration between requests to the Atlas API. Supported units for this parameter are m/s. + default: 5m + multi: false + required: true + show_user: false + - name: http_client_timeout + type: text + title: HTTP Client Timeout + description: Duration before declaring that the HTTP client connection has timed out. Valid time units are m/s. If the data collection period is extensive, ensure that this parameter's value is appropriately long. + multi: false + required: true + show_user: false + default: 1m + - name: enable_request_tracer + type: bool + title: Enable request tracing + multi: false + required: false + show_user: false + description: The request tracer logs requests and responses to the agent's local file-system for debugging configurations. Enabling this request tracing compromises security and should only be used for debugging. See [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#_request_tracer_filename) for details. + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - mongodb_atlas-mongod_audit + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/fleet/current/elastic-agent-processor-configuration.html) for details. diff --git a/packages/mongodb_atlas/data_stream/mongod_audit/sample_event.json b/packages/mongodb_atlas/data_stream/mongod_audit/sample_event.json new file mode 100644 index 00000000000..bbdae79bffe --- /dev/null +++ b/packages/mongodb_atlas/data_stream/mongod_audit/sample_event.json @@ -0,0 +1,105 @@ +{ + "@timestamp": "2023-04-01T12:00:00.000Z", + "agent": { + "ephemeral_id": "82e5a7ce-c7ad-436f-90f2-a1cefbe22333", + "id": "498e0c10-4447-4a56-90f0-ba02c44a01c2", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.13.0" + }, + "data_stream": { + "dataset": "mongodb_atlas.mongod_audit", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.11.0" + }, + "elastic_agent": { + "id": "498e0c10-4447-4a56-90f0-ba02c44a01c2", + "snapshot": false, + "version": "8.13.0" + }, + "event": { + "action": "authenticate", + "agent_id_status": "verified", + "category": [ + "network", + "authentication" + ], + "dataset": "mongodb_atlas.mongod_audit", + "ingested": "2024-04-09T06:13:37Z", + "kind": "event", + "module": "mongodb_atlas", + "type": [ + "access", + "info" + ] + }, + "host": { + "architecture": "x86_64", + "containerized": true, + "hostname": "docker-fleet-agent", + "id": "8259e024976a406e8a54cdbffeb84fec", + "ip": [ + "192.168.245.7" + ], + "mac": [ + "02-42-C0-A8-F5-07" + ], + "name": "docker-fleet-agent", + "os": { + "codename": "focal", + "family": "debian", + "kernel": "3.10.0-1160.92.1.el7.x86_64", + "name": "Ubuntu", + "platform": "ubuntu", + "type": "linux", + "version": "20.04.6 LTS (Focal Fossa)" + } + }, + "input": { + "type": "cel" + }, + "mongodb_atlas": { + "mongod_audit": { + "hostname": "hostname1", + "local": { + "ip": "127.0.0.1", + "port": 27017 + }, + "remote": { + "ip": "192.168.1.100", + "port": 54320 + }, + "result": "Success", + "user": { + "names": [ + { + "db": "admin", + "user": "auditUser" + } + ], + "roles": [ + { + "db": "admin", + "role": "dbAdmin" + } + ] + }, + "uuid": { + "binary": "some-unique-identifier", + "type": "04" + } + } + }, + "related": { + "ip": [ + "127.0.0.1", + "192.168.1.100" + ] + }, + "tags": [ + "mongodb_atlas-mongod_audit" + ] +} \ No newline at end of file diff --git a/packages/mongodb_atlas/docs/README.md b/packages/mongodb_atlas/docs/README.md index c90ed4f1a20..cc17df46cd8 100644 --- a/packages/mongodb_atlas/docs/README.md +++ b/packages/mongodb_atlas/docs/README.md @@ -6,22 +6,25 @@ Use the MongoDB Atlas integration to: -- Collect metrics related to process. +- Collect MongoDB Audit logs and Process metrics for comprehensive monitoring and analysis. - Create informative visualizations to track usage trends, measure key metrics, and derive actionable business insights. - Set up alerts to minimize Mean Time to Detect (MTTD) and Mean Time to Resolve (MTTR) by quickly referencing relevant logs during troubleshooting. ## Data streams -The MongoDB Atlas integration collects metrics. +The MongoDB Atlas integration collects logs and metrics. + +Logs help you keep a record of events that happen on your machine. The `Log` data stream collected by MongoDB Atlas integration is `mongod_audit`. Metrics give you insight into the statistics of the MongoDB Atlas. The `Metric` data stream collected by the MongoDB Atlas integration is `process` so that the user can monitor and troubleshoot the performance of the MongoDB Atlas instance. Data streams: +- `mongod_audit`: The auditing facility allows administrators and users to track system activity for deployments with multiple users and applications. Mongod Audit logs capture events related to database operations such as insertions, updates, deletions, user authentication, etc., occurring within the mongod instances. - `process` : This data stream collects host metrics per process for all the hosts of the specified group. Metrics like measurements for the host, such as CPU usage, number of I/O operations and memory are available on this data stream. Note: -- Users can monitor and see the metrics inside the ingested documents for MongoDB Atlas in the `logs-*` index pattern from `Discover`. +- Users can monitor and see the log inside the ingested documents for MongoDB Atlas in the `logs-*` index pattern from `Discover`. ## Prerequisites @@ -29,25 +32,26 @@ You can store and search your data using Elasticsearch and visualize and manage ## Setup -### To collect data from MongoDB Atlas, the following parameters from your MongoDB Atlas instance are required: +### To collect data from MongoDB Atlas, the following parameters from your MongoDB Atlas instance are required 1. Public Key 2. Private Key 3. GroupId -### Steps to obtain Public Key, Private Key and GroupId: +### Steps to obtain Public Key, Private Key and GroupId 1. Generate programmatic API Keys with project owner permissions using the instructions in the Atlas [documentation](https://www.mongodb.com/docs/atlas/configure-api-access/#create-an-api-key-for-a-project). Then, copy the public key and private key. These serve the same function as a username and API Key respectively. -2. You can find your Project ID (Group ID) in the Atlas UI. To do this, navigate to your project, click on Settings, and copy the Project ID (Group ID). You can also programmatically find it using the Atlas Admin API or Atlas CLI as described in this Atlas [document](https://www.mongodb.com/docs/atlas/app-services/apps/metadata/#find-a-project-id). +2. Enable Database Auditing for the Atlas project you want to monitor logs. You can follow the instructions provided in this Atlas [document](https://www.mongodb.com/docs/atlas/database-auditing/#procedure). +3. You can find your Project ID (Group ID) in the Atlas UI. To do this, navigate to your project, click on Settings, and copy the Project ID (Group ID). You can also programmatically find it using the Atlas Admin API or Atlas CLI as described in this Atlas [document](https://www.mongodb.com/docs/atlas/app-services/apps/metadata/#find-a-project-id). -### Important terms of MongoDB Atlas API: +### Important terms of MongoDB Atlas API 1. Granularity: Duration that specifies the interval at which Atlas reports the metrics. 2. Period: Duration over which Atlas reports the metrics. Note: Both of above attributes can be set by using `period` in configuration parameters. -### Enabling the integration in Elastic: +### Steps to enable Integration in Elastic 1. In Kibana go to Management > Integrations 2. In "Search for integrations" search bar, type MongoDB Atlas @@ -56,6 +60,164 @@ Note: Both of above attributes can be set by using `period` in configuration par 5. Enter all the necessary configuration parameters, including Public Key, Private Key, and GroupId. 6. Finally, save the integration. +Note: +- The `mongod_audit` data stream gathers historical data spanning the previous 30 minutes. +- Mongod: Mongod is the primary daemon method for the MongoDB system. It helps in handling the data requests, managing the data access, performing background management operations, and other core database operations. + +## Troubleshooting + +If you encounter an error while ingesting data, it might be due to the data collected over a long time span. Generating a response in such cases may take longer and might cause a request timeout if the `HTTP Client Timeout` parameter is set to a small duration. To avoid this error, it is recommended to adjust the `HTTP Client Timeout` and `Interval` parameters based on the duration of data collection. +``` +{ + "error": { + "message": "failed eval: net/http: request canceled (Client.Timeout or context cancellation while reading body)" + } +} +``` + +## Logs reference + +### Mongod Audit + +This is the `mongod_audit` data stream. This data stream allows administrators and users to track system activity for deployments with multiple users and applications. + +An example event for `mongod_audit` looks as following: + +```json +{ + "@timestamp": "2023-04-01T12:00:00.000Z", + "agent": { + "ephemeral_id": "82e5a7ce-c7ad-436f-90f2-a1cefbe22333", + "id": "498e0c10-4447-4a56-90f0-ba02c44a01c2", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.13.0" + }, + "data_stream": { + "dataset": "mongodb_atlas.mongod_audit", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.11.0" + }, + "elastic_agent": { + "id": "498e0c10-4447-4a56-90f0-ba02c44a01c2", + "snapshot": false, + "version": "8.13.0" + }, + "event": { + "action": "authenticate", + "agent_id_status": "verified", + "category": [ + "network", + "authentication" + ], + "dataset": "mongodb_atlas.mongod_audit", + "ingested": "2024-04-09T06:13:37Z", + "kind": "event", + "module": "mongodb_atlas", + "type": [ + "access", + "info" + ] + }, + "host": { + "architecture": "x86_64", + "containerized": true, + "hostname": "docker-fleet-agent", + "id": "8259e024976a406e8a54cdbffeb84fec", + "ip": [ + "192.168.245.7" + ], + "mac": [ + "02-42-C0-A8-F5-07" + ], + "name": "docker-fleet-agent", + "os": { + "codename": "focal", + "family": "debian", + "kernel": "3.10.0-1160.92.1.el7.x86_64", + "name": "Ubuntu", + "platform": "ubuntu", + "type": "linux", + "version": "20.04.6 LTS (Focal Fossa)" + } + }, + "input": { + "type": "cel" + }, + "mongodb_atlas": { + "mongod_audit": { + "hostname": "hostname1", + "local": { + "ip": "127.0.0.1", + "port": 27017 + }, + "remote": { + "ip": "192.168.1.100", + "port": 54320 + }, + "result": "Success", + "user": { + "names": [ + { + "db": "admin", + "user": "auditUser" + } + ], + "roles": [ + { + "db": "admin", + "role": "dbAdmin" + } + ] + }, + "uuid": { + "binary": "some-unique-identifier", + "type": "04" + } + } + }, + "related": { + "ip": [ + "127.0.0.1", + "192.168.1.100" + ] + }, + "tags": [ + "mongodb_atlas-mongod_audit" + ] +} +``` + +**Exported fields** + +| Field | Description | Type | +|---|---|---| +| @timestamp | Event timestamp. | date | +| data_stream.dataset | Data stream dataset. | constant_keyword | +| data_stream.namespace | Data stream namespace. | constant_keyword | +| data_stream.type | Data stream type. | constant_keyword | +| input.type | Type of Filebeat input. | keyword | +| mongodb_atlas.mongod_audit.hostname | Hostname that stores the log files that you want to download. | keyword | +| mongodb_atlas.mongod_audit.local.ip | IP address of the running instance. | ip | +| mongodb_atlas.mongod_audit.local.is_system_user | True if the event is caused by a system user, false otherwise. | boolean | +| mongodb_atlas.mongod_audit.local.port | Port number of the running instance. | long | +| mongodb_atlas.mongod_audit.local.unix | Unix that contains the MongoDB socket file path if the client connects through a Unix domain socket. | keyword | +| mongodb_atlas.mongod_audit.param | Specific details for the event. | object | +| mongodb_atlas.mongod_audit.remote.ip | IP address of the incoming connection associated with the event. | ip | +| mongodb_atlas.mongod_audit.remote.is_system_user | True if the event is caused by a system user, false otherwise. | boolean | +| mongodb_atlas.mongod_audit.remote.port | Port number of the incoming connection associated with the event. | long | +| mongodb_atlas.mongod_audit.remote.unix | Unix that contains the MongoDB socket file path if the client connects through a Unix domain socket. | keyword | +| mongodb_atlas.mongod_audit.result | Error code. | keyword | +| mongodb_atlas.mongod_audit.user.names | Array of user identification documents. | object | +| mongodb_atlas.mongod_audit.user.roles | Array of documents that specify the roles granted to the user. | object | +| mongodb_atlas.mongod_audit.uuid.binary | Document that contains a universally unique identifier (UUID) for the audit message. | keyword | +| mongodb_atlas.mongod_audit.uuid.type | The $type field specifies the BSON subtype for the $binary field. | keyword | +| tags | List of keywords used to tag each event. | keyword | + + ## Metrics reference ### Process diff --git a/packages/mongodb_atlas/img/mongodb_atlas-mongod-audit-dashboard.png b/packages/mongodb_atlas/img/mongodb_atlas-mongod-audit-dashboard.png new file mode 100644 index 00000000000..dd51a2ea5f3 Binary files /dev/null and b/packages/mongodb_atlas/img/mongodb_atlas-mongod-audit-dashboard.png differ diff --git a/packages/mongodb_atlas/kibana/dashboard/mongodb_atlas-88554c14-2b94-424f-8a3e-b6f65722fd51.json b/packages/mongodb_atlas/kibana/dashboard/mongodb_atlas-88554c14-2b94-424f-8a3e-b6f65722fd51.json new file mode 100644 index 00000000000..a16e66af73d --- /dev/null +++ b/packages/mongodb_atlas/kibana/dashboard/mongodb_atlas-88554c14-2b94-424f-8a3e-b6f65722fd51.json @@ -0,0 +1,1490 @@ +{ + "attributes": { + "controlGroupInput": { + "chainingSystem": "HIERARCHICAL", + "controlStyle": "oneLine", + "ignoreParentSettingsJSON": "{\"ignoreFilters\":false,\"ignoreQuery\":false,\"ignoreTimerange\":false,\"ignoreValidations\":false}", + "panelsJSON": "{\"4414bfd2-bd4e-49f2-8f60-18cd73c07fb9\":{\"type\":\"optionsListControl\",\"order\":0,\"grow\":false,\"width\":\"medium\",\"explicitInput\":{\"id\":\"4414bfd2-bd4e-49f2-8f60-18cd73c07fb9\",\"fieldName\":\"mongodb_atlas.mongod_audit.hostname\",\"title\":\"Hostname\",\"grow\":true,\"width\":\"medium\",\"selectedOptions\":[],\"exclude\":false,\"enhancements\":{}}}}" + }, + "description": "", + "kibanaSavedObjectMeta": { + "searchSourceJSON": { + "filter": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "indexRefName": "kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "mongodb_atlas.mongod_audit" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "mongodb_atlas.mongod_audit" + } + } + } + ], + "query": { + "language": "kuery", + "query": "" + } + } + }, + "optionsJSON": { + "hidePanelTitles": false, + "syncColors": false, + "syncCursor": true, + "syncTooltips": false, + "useMargins": true + }, + "panelsJSON": [ + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-e6b79055-c39d-4d4a-bb19-bc3690eaebad", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "currentIndexPatternId": "logs-*", + "layers": { + "e6b79055-c39d-4d4a-bb19-bc3690eaebad": { + "columnOrder": [ + "b9e8f2f8-a4ba-4860-b61b-d4a10d802542", + "f9a31155-d385-4869-bc47-266f91786306", + "d73c3683-9dcc-481d-b8b3-19235f14959b" + ], + "columns": { + "b9e8f2f8-a4ba-4860-b61b-d4a10d802542": { + "dataType": "string", + "isBucketed": true, + "label": "Top 5 values of event.action", + "operationType": "terms", + "params": { + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": false, + "orderBy": { + "columnId": "d73c3683-9dcc-481d-b8b3-19235f14959b", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "size": 5 + }, + "scale": "ordinal", + "sourceField": "event.action" + }, + "d73c3683-9dcc-481d-b8b3-19235f14959b": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "___records___" + }, + "f9a31155-d385-4869-bc47-266f91786306": { + "customLabel": false, + "dataType": "date", + "isBucketed": true, + "label": "@timestamp", + "operationType": "date_histogram", + "params": { + "dropPartials": false, + "includeEmptyRows": false, + "interval": "auto" + }, + "scale": "interval", + "sourceField": "@timestamp" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "indexPatternId": "logs-*", + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "27f966a9-6db9-4f8e-a26f-ece17d0f59b4", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "mongodb_atlas.mongod_audit" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "mongodb_atlas.mongod_audit" + } + } + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "curveType": "LINEAR", + "emphasizeFitting": true, + "endValue": "None", + "fittingFunction": "Linear", + "layers": [ + { + "accessors": [ + "d73c3683-9dcc-481d-b8b3-19235f14959b" + ], + "layerId": "e6b79055-c39d-4d4a-bb19-bc3690eaebad", + "layerType": "data", + "position": "top", + "seriesType": "line", + "showGridlines": false, + "splitAccessor": "b9e8f2f8-a4ba-4860-b61b-d4a10d802542", + "xAccessor": "f9a31155-d385-4869-bc47-266f91786306" + } + ], + "legend": { + "isVisible": true, + "legendSize": "large", + "position": "right" + }, + "preferredSeriesType": "line", + "title": "Empty XY chart", + "valueLabels": "hide", + "valuesInLegend": true + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsXY" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 12, + "i": "d51a65b5-2b67-4e7d-8f67-39e0be60951c", + "w": 19, + "x": 0, + "y": 0 + }, + "panelIndex": "d51a65b5-2b67-4e7d-8f67-39e0be60951c", + "title": "Audit events by action [Logs MongoDB Atlas]", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-6a395f87-9046-43eb-9868-fcb5fef2d588", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "currentIndexPatternId": "logs-*", + "layers": { + "6a395f87-9046-43eb-9868-fcb5fef2d588": { + "columnOrder": [ + "081a5009-3056-4272-9368-06a5ff3512e3" + ], + "columns": { + "081a5009-3056-4272-9368-06a5ff3512e3": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Failed audit events", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "mongodb_atlas.mongod_audit.result" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "indexPatternId": "logs-*", + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "970ff4e8-40d3-4202-88fe-e30cb734fcf2", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "mongodb_atlas.mongod_audit" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "mongodb_atlas.mongod_audit" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "mongodb_atlas.mongod_audit.result", + "index": "6e46be5c-a178-44bf-bf43-2f30fe0e91d7", + "key": "mongodb_atlas.mongod_audit.result", + "negate": true, + "params": { + "query": "Success" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "mongodb_atlas.mongod_audit.result": "Success" + } + } + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "color": "#FF474C", + "layerId": "6a395f87-9046-43eb-9868-fcb5fef2d588", + "layerType": "data", + "metricAccessor": "081a5009-3056-4272-9368-06a5ff3512e3" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": {}, + "hidePanelTitles": true + }, + "gridData": { + "h": 4, + "i": "4c1f2cd2-c7a0-4c12-8700-1607880700b4", + "w": 10, + "x": 19, + "y": 0 + }, + "panelIndex": "4c1f2cd2-c7a0-4c12-8700-1607880700b4", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-b854c8f5-b3d4-4671-a815-22ce69800910", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "currentIndexPatternId": "logs-*", + "layers": { + "b854c8f5-b3d4-4671-a815-22ce69800910": { + "columnOrder": [ + "da19a79c-62e3-46fa-91b7-d07058b96a7f", + "7fb53a3d-c196-4fee-b2a3-b886a48093be", + "d60c6535-19d2-48c4-a5c8-6525780c1d28" + ], + "columns": { + "7fb53a3d-c196-4fee-b2a3-b886a48093be": { + "customLabel": false, + "dataType": "date", + "isBucketed": true, + "label": "@timestamp", + "operationType": "date_histogram", + "params": { + "dropPartials": false, + "includeEmptyRows": false, + "interval": "auto" + }, + "scale": "interval", + "sourceField": "@timestamp" + }, + "d60c6535-19d2-48c4-a5c8-6525780c1d28": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "___records___" + }, + "da19a79c-62e3-46fa-91b7-d07058b96a7f": { + "dataType": "string", + "isBucketed": true, + "label": "Top 5 values of mongodb_atlas.mongod_audit.hostname", + "operationType": "terms", + "params": { + "accuracyMode": true, + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": false, + "orderBy": { + "columnId": "d60c6535-19d2-48c4-a5c8-6525780c1d28", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "size": 5 + }, + "scale": "ordinal", + "sourceField": "mongodb_atlas.mongod_audit.hostname" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "indexPatternId": "logs-*", + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "638338d2-7c80-4da4-a43c-2bb67be66bbd", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "mongodb_atlas.mongod_audit" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "mongodb_atlas.mongod_audit" + } + } + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "curveType": "LINEAR", + "emphasizeFitting": true, + "fittingFunction": "Zero", + "layers": [ + { + "accessors": [ + "d60c6535-19d2-48c4-a5c8-6525780c1d28" + ], + "layerId": "b854c8f5-b3d4-4671-a815-22ce69800910", + "layerType": "data", + "position": "top", + "seriesType": "line", + "showGridlines": false, + "splitAccessor": "da19a79c-62e3-46fa-91b7-d07058b96a7f", + "xAccessor": "7fb53a3d-c196-4fee-b2a3-b886a48093be" + } + ], + "legend": { + "isVisible": true, + "position": "right" + }, + "preferredSeriesType": "line", + "title": "Empty XY chart", + "valueLabels": "hide", + "valuesInLegend": true + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsXY" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 12, + "i": "a7ce33e2-ac75-4503-b1bf-a1cfeeade0b5", + "w": 19, + "x": 29, + "y": 0 + }, + "panelIndex": "a7ce33e2-ac75-4503-b1bf-a1cfeeade0b5", + "title": "Audit events by hostname [Logs MongoDB Atlas]", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-df4300b8-d282-45db-8afb-bf4f2d4732bb", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "ea764634-d952-4afc-b0d0-6b2593e631d8", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "9d965676-699b-4da6-9773-12c586fe1451", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "layers": { + "df4300b8-d282-45db-8afb-bf4f2d4732bb": { + "columnOrder": [ + "fadcac49-354f-4993-955b-bf5856e20b24" + ], + "columns": { + "fadcac49-354f-4993-955b-bf5856e20b24": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Failed authentications", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "mongodb_atlas.mongod_audit.result" + } + }, + "incompleteColumns": {} + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "ea764634-d952-4afc-b0d0-6b2593e631d8", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "mongodb_atlas.mongod_audit" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "mongodb_atlas.mongod_audit" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "9d965676-699b-4da6-9773-12c586fe1451", + "negate": false, + "params": [ + { + "meta": { + "alias": null, + "disabled": false, + "field": "mongodb_atlas.mongod_audit.result", + "index": "logs-*", + "key": "mongodb_atlas.mongod_audit.result", + "negate": false, + "params": { + "query": "Authentication Failed" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "mongodb_atlas.mongod_audit.result": "Authentication Failed" + } + } + }, + { + "meta": { + "alias": null, + "disabled": false, + "field": "event.action", + "index": "logs-*", + "key": "event.action", + "negate": false, + "params": { + "query": "authenticate" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "event.action": "authenticate" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "color": "#FF474C", + "layerId": "df4300b8-d282-45db-8afb-bf4f2d4732bb", + "layerType": "data", + "metricAccessor": "fadcac49-354f-4993-955b-bf5856e20b24" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": {}, + "hidePanelTitles": true + }, + "gridData": { + "h": 4, + "i": "c9c47a9b-0732-4bd9-b7d9-9900f5651b8c", + "w": 10, + "x": 19, + "y": 4 + }, + "panelIndex": "c9c47a9b-0732-4bd9-b7d9-9900f5651b8c", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "description": "", + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-b11afd7f-ccde-4144-a5c4-c2bd5f0f133b", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "d26fabf1-07c9-440f-8ed7-36510678e8f2", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "c1d1916b-8c57-4c7e-82c0-a169b86bc58a", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "layers": { + "b11afd7f-ccde-4144-a5c4-c2bd5f0f133b": { + "columnOrder": [ + "49b33ec2-9a87-4f2f-87c9-ebae3e74110e" + ], + "columns": { + "49b33ec2-9a87-4f2f-87c9-ebae3e74110e": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Failed authorizations", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "mongodb_atlas.mongod_audit.result" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "d26fabf1-07c9-440f-8ed7-36510678e8f2", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "mongodb_atlas.mongod_audit" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "mongodb_atlas.mongod_audit" + } + } + }, + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "c1d1916b-8c57-4c7e-82c0-a169b86bc58a", + "negate": false, + "params": [ + { + "meta": { + "alias": null, + "disabled": false, + "field": "mongodb_atlas.mongod_audit.result", + "index": "logs-*", + "key": "mongodb_atlas.mongod_audit.result", + "negate": false, + "params": { + "query": "Unauthorized to perform the operation" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "mongodb_atlas.mongod_audit.result": "Unauthorized to perform the operation" + } + } + }, + { + "meta": { + "alias": null, + "disabled": false, + "field": "event.action", + "index": "logs-*", + "key": "event.action", + "negate": false, + "params": { + "query": "authCheck" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "event.action": "authCheck" + } + } + } + ], + "relation": "AND", + "type": "combined" + }, + "query": {} + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "color": "#FF474C", + "layerId": "b11afd7f-ccde-4144-a5c4-c2bd5f0f133b", + "layerType": "data", + "metricAccessor": "49b33ec2-9a87-4f2f-87c9-ebae3e74110e" + } + }, + "title": "Authentication Succussed", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": {}, + "hidePanelTitles": true + }, + "gridData": { + "h": 4, + "i": "e7947ccd-02d3-4787-9d37-a93f2eefe560", + "w": 10, + "x": 19, + "y": 8 + }, + "panelIndex": "e7947ccd-02d3-4787-9d37-a93f2eefe560", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-2738d1cf-d706-488d-87f1-4a312f1b0aec", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "currentIndexPatternId": "logs-*", + "layers": { + "2738d1cf-d706-488d-87f1-4a312f1b0aec": { + "columnOrder": [ + "6b003785-3e6b-43c3-8064-54597bf84392", + "b5d2e267-5389-49ec-82bd-9a41ea471c7e" + ], + "columns": { + "6b003785-3e6b-43c3-8064-54597bf84392": { + "customLabel": false, + "dataType": "string", + "isBucketed": true, + "label": "Top 5 values of event.action", + "operationType": "terms", + "params": { + "accuracyMode": true, + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": false, + "orderBy": { + "columnId": "b5d2e267-5389-49ec-82bd-9a41ea471c7e", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "size": 5 + }, + "scale": "ordinal", + "sourceField": "event.action" + }, + "b5d2e267-5389-49ec-82bd-9a41ea471c7e": { + "dataType": "number", + "isBucketed": false, + "label": "Count of records", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "___records___" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "indexPatternId": "logs-*", + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "85a41d10-1b4d-4711-8048-1100e601ea6b", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "mongodb_atlas.mongod_audit" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "mongodb_atlas.mongod_audit" + } + } + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "layers": [ + { + "categoryDisplay": "default", + "emptySizeRatio": 0.3, + "layerId": "2738d1cf-d706-488d-87f1-4a312f1b0aec", + "layerType": "data", + "legendDisplay": "default", + "metrics": [ + "b5d2e267-5389-49ec-82bd-9a41ea471c7e" + ], + "nestedLegend": false, + "numberDisplay": "percent", + "primaryGroups": [ + "6b003785-3e6b-43c3-8064-54597bf84392" + ] + } + ], + "shape": "donut" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsPie" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 13, + "i": "70400818-7c0c-4a77-95b4-fe7a2c03b937", + "w": 13, + "x": 0, + "y": 12 + }, + "panelIndex": "70400818-7c0c-4a77-95b4-fe7a2c03b937", + "title": "Audit actions [Logs MongoDB Atlas]", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-c7b75e0b-22bc-4f3a-800b-8c8046a809b0", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "currentIndexPatternId": "logs-*", + "layers": { + "c7b75e0b-22bc-4f3a-800b-8c8046a809b0": { + "columnOrder": [ + "92d5b28e-d452-4df3-91f8-aa2f69c81101", + "e3174c23-d638-4128-81ce-5ccc66e9cf6f", + "d5569916-5b66-403c-8849-75b76032b55a" + ], + "columns": { + "92d5b28e-d452-4df3-91f8-aa2f69c81101": { + "dataType": "string", + "isBucketed": true, + "label": "Top 5 values of mongodb_atlas.mongod_audit.user.names.user", + "operationType": "terms", + "params": { + "accuracyMode": true, + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": false, + "orderBy": { + "columnId": "d5569916-5b66-403c-8849-75b76032b55a", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "size": 5 + }, + "scale": "ordinal", + "sourceField": "mongodb_atlas.mongod_audit.user.names.user" + }, + "d5569916-5b66-403c-8849-75b76032b55a": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "count", + "params": { + "emptyAsNull": false + }, + "scale": "ratio", + "sourceField": "___records___" + }, + "e3174c23-d638-4128-81ce-5ccc66e9cf6f": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Action", + "operationType": "terms", + "params": { + "accuracyMode": true, + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": false, + "orderBy": { + "columnId": "d5569916-5b66-403c-8849-75b76032b55a", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "size": 5 + }, + "scale": "ordinal", + "sourceField": "event.action" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "indexPatternId": "logs-*", + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "8e5b3099-5943-441f-9a3c-6ec2573af079", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "mongodb_atlas.mongod_audit" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "mongodb_atlas.mongod_audit" + } + } + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "layers": [ + { + "accessors": [ + "d5569916-5b66-403c-8849-75b76032b55a" + ], + "layerId": "c7b75e0b-22bc-4f3a-800b-8c8046a809b0", + "layerType": "data", + "position": "top", + "seriesType": "bar", + "showGridlines": false, + "splitAccessor": "92d5b28e-d452-4df3-91f8-aa2f69c81101", + "xAccessor": "e3174c23-d638-4128-81ce-5ccc66e9cf6f" + } + ], + "legend": { + "isVisible": true, + "position": "right" + }, + "preferredSeriesType": "bar", + "title": "Empty XY chart", + "valueLabels": "show" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsXY" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 13, + "i": "3d0d8ae0-7d57-4f5d-8c52-9cfa0b864c80", + "w": 35, + "x": 13, + "y": 12 + }, + "panelIndex": "3d0d8ae0-7d57-4f5d-8c52-9cfa0b864c80", + "title": "Audit actions by user [Logs MongoDB Atlas]", + "type": "lens" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-bd4b3489-ed64-4108-a89c-7a1cbed8e8d0", + "type": "index-pattern" + } + ], + "state": { + "adHocDataViews": {}, + "datasourceStates": { + "formBased": { + "layers": { + "bd4b3489-ed64-4108-a89c-7a1cbed8e8d0": { + "columnOrder": [ + "a6ff9301-8681-440b-a05e-8ae4fba85856", + "f49d1bc0-32ce-4650-a761-beee77c27150", + "07d349a2-8f7d-4ca5-9c81-dfa0640e95c3", + "476b944b-ebf3-4fbb-9f4e-b38a14d6fadc", + "87aaa3f4-dc4e-4ae6-8af4-b220d9a6f36f", + "763c43fe-94cb-4f1d-a425-8836f1fb92b5", + "b63cfa8a-e073-42f7-8bf6-84eed0feff65" + ], + "columns": { + "07d349a2-8f7d-4ca5-9c81-dfa0640e95c3": { + "customLabel": true, + "dataType": "string", + "filter": { + "language": "kuery", + "query": "\"mongodb_atlas.mongod_audit.user.roles.role\": *" + }, + "isBucketed": false, + "label": "User role", + "operationType": "last_value", + "params": { + "sortField": "@timestamp" + }, + "scale": "ordinal", + "sourceField": "mongodb_atlas.mongod_audit.user.roles.role" + }, + "476b944b-ebf3-4fbb-9f4e-b38a14d6fadc": { + "customLabel": true, + "dataType": "ip", + "filter": { + "language": "kuery", + "query": "\"mongodb_atlas.mongod_audit.local.ip\": *" + }, + "isBucketed": false, + "label": "Local IP", + "operationType": "last_value", + "params": { + "sortField": "@timestamp" + }, + "scale": "ordinal", + "sourceField": "mongodb_atlas.mongod_audit.local.ip" + }, + "763c43fe-94cb-4f1d-a425-8836f1fb92b5": { + "customLabel": true, + "dataType": "string", + "filter": { + "language": "kuery", + "query": "\"event.action\": *" + }, + "isBucketed": false, + "label": "Action", + "operationType": "last_value", + "params": { + "sortField": "@timestamp" + }, + "scale": "ordinal", + "sourceField": "event.action" + }, + "87aaa3f4-dc4e-4ae6-8af4-b220d9a6f36f": { + "customLabel": true, + "dataType": "ip", + "filter": { + "language": "kuery", + "query": "\"mongodb_atlas.mongod_audit.remote.ip\": *" + }, + "isBucketed": false, + "label": "Remote IP", + "operationType": "last_value", + "params": { + "sortField": "@timestamp" + }, + "scale": "ordinal", + "sourceField": "mongodb_atlas.mongod_audit.remote.ip" + }, + "a6ff9301-8681-440b-a05e-8ae4fba85856": { + "dataType": "date", + "isBucketed": true, + "label": "@timestamp", + "operationType": "date_histogram", + "params": { + "dropPartials": false, + "includeEmptyRows": false, + "interval": "auto" + }, + "scale": "interval", + "sourceField": "@timestamp" + }, + "b63cfa8a-e073-42f7-8bf6-84eed0feff65": { + "customLabel": true, + "dataType": "string", + "filter": { + "language": "kuery", + "query": "\"mongodb_atlas.mongod_audit.result\": *" + }, + "isBucketed": false, + "label": "Result", + "operationType": "last_value", + "params": { + "sortField": "@timestamp" + }, + "scale": "ordinal", + "sourceField": "mongodb_atlas.mongod_audit.result" + }, + "f49d1bc0-32ce-4650-a761-beee77c27150": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "User", + "operationType": "terms", + "params": { + "accuracyMode": true, + "exclude": [], + "excludeIsRegex": false, + "include": [], + "includeIsRegex": false, + "missingBucket": false, + "orderBy": { + "fallback": true, + "type": "alphabetical" + }, + "orderDirection": "asc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "size": 10000 + }, + "scale": "ordinal", + "sourceField": "mongodb_atlas.mongod_audit.user.names.user" + } + }, + "ignoreGlobalFilters": false, + "incompleteColumns": {}, + "sampling": 1 + } + } + }, + "indexpattern": { + "layers": {} + }, + "textBased": { + "layers": {} + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "field": "data_stream.dataset", + "index": "b7eedcd9-3437-4fb3-9cd1-e68236cacee1", + "key": "data_stream.dataset", + "negate": false, + "params": { + "query": "mongodb_atlas.mongod_audit" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "data_stream.dataset": "mongodb_atlas.mongod_audit" + } + } + } + ], + "internalReferences": [], + "query": { + "language": "kuery", + "query": "" + }, + "visualization": { + "columns": [ + { + "alignment": "left", + "columnId": "f49d1bc0-32ce-4650-a761-beee77c27150", + "isMetric": false, + "isTransposed": false + }, + { + "alignment": "left", + "columnId": "07d349a2-8f7d-4ca5-9c81-dfa0640e95c3", + "isMetric": true, + "isTransposed": false + }, + { + "alignment": "center", + "columnId": "476b944b-ebf3-4fbb-9f4e-b38a14d6fadc", + "isMetric": true, + "isTransposed": false + }, + { + "alignment": "center", + "columnId": "87aaa3f4-dc4e-4ae6-8af4-b220d9a6f36f", + "isMetric": true, + "isTransposed": false + }, + { + "alignment": "center", + "columnId": "763c43fe-94cb-4f1d-a425-8836f1fb92b5", + "isMetric": true, + "isTransposed": false + }, + { + "alignment": "center", + "columnId": "b63cfa8a-e073-42f7-8bf6-84eed0feff65", + "isMetric": true, + "isTransposed": false + }, + { + "columnId": "a6ff9301-8681-440b-a05e-8ae4fba85856", + "isMetric": false, + "isTransposed": false + } + ], + "headerRowHeight": "auto", + "layerId": "bd4b3489-ed64-4108-a89c-7a1cbed8e8d0", + "layerType": "data", + "paging": { + "enabled": true, + "size": 10 + }, + "rowHeight": "auto" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsDatatable" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 17, + "i": "5a4ba5e1-49d3-4b5c-b99b-28cb53e25f62", + "w": 48, + "x": 0, + "y": 25 + }, + "panelIndex": "5a4ba5e1-49d3-4b5c-b99b-28cb53e25f62", + "title": "Audit overview [Logs MongoDB Atlas]", + "type": "lens" + } + ], + "timeRestore": false, + "title": "[Logs MongoDB Atlas] Mongod Audit", + "version": 1 + }, + "coreMigrationVersion": "8.8.0", + "created_at": "2024-03-13T10:52:35.457Z", + "id": "mongodb_atlas-88554c14-2b94-424f-8a3e-b6f65722fd51", + "managed": false, + "references": [ + { + "id": "logs-*", + "name": "kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "d51a65b5-2b67-4e7d-8f67-39e0be60951c:indexpattern-datasource-layer-e6b79055-c39d-4d4a-bb19-bc3690eaebad", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "4c1f2cd2-c7a0-4c12-8700-1607880700b4:indexpattern-datasource-layer-6a395f87-9046-43eb-9868-fcb5fef2d588", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "a7ce33e2-ac75-4503-b1bf-a1cfeeade0b5:indexpattern-datasource-layer-b854c8f5-b3d4-4671-a815-22ce69800910", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "c9c47a9b-0732-4bd9-b7d9-9900f5651b8c:indexpattern-datasource-layer-df4300b8-d282-45db-8afb-bf4f2d4732bb", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "c9c47a9b-0732-4bd9-b7d9-9900f5651b8c:ea764634-d952-4afc-b0d0-6b2593e631d8", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "c9c47a9b-0732-4bd9-b7d9-9900f5651b8c:9d965676-699b-4da6-9773-12c586fe1451", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "e7947ccd-02d3-4787-9d37-a93f2eefe560:indexpattern-datasource-layer-b11afd7f-ccde-4144-a5c4-c2bd5f0f133b", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "e7947ccd-02d3-4787-9d37-a93f2eefe560:d26fabf1-07c9-440f-8ed7-36510678e8f2", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "e7947ccd-02d3-4787-9d37-a93f2eefe560:c1d1916b-8c57-4c7e-82c0-a169b86bc58a", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "70400818-7c0c-4a77-95b4-fe7a2c03b937:indexpattern-datasource-layer-2738d1cf-d706-488d-87f1-4a312f1b0aec", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "3d0d8ae0-7d57-4f5d-8c52-9cfa0b864c80:indexpattern-datasource-layer-c7b75e0b-22bc-4f3a-800b-8c8046a809b0", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "5a4ba5e1-49d3-4b5c-b99b-28cb53e25f62:indexpattern-datasource-layer-bd4b3489-ed64-4108-a89c-7a1cbed8e8d0", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "controlGroup_4414bfd2-bd4e-49f2-8f60-18cd73c07fb9:optionsListDataView", + "type": "index-pattern" + } + ], + "type": "dashboard", + "typeMigrationVersion": "8.9.0" +} \ No newline at end of file diff --git a/packages/mongodb_atlas/manifest.yml b/packages/mongodb_atlas/manifest.yml index af64b32ef34..0d4cdce6d3e 100644 --- a/packages/mongodb_atlas/manifest.yml +++ b/packages/mongodb_atlas/manifest.yml @@ -1,10 +1,10 @@ format_version: 3.0.3 name: mongodb_atlas title: "MongoDB Atlas" -version: 0.0.1 +version: 0.0.2 source: license: "Elastic-2.0" -description: This Elastic integration collects metrics from MongoDB Atlas instance. +description: This Elastic integration collects logs and metrics from MongoDB Atlas instance. type: integration categories: - cloud @@ -15,6 +15,10 @@ conditions: elastic: subscription: "basic" screenshots: + - src: /img/mongodb_atlas-mongod-audit-dashboard.png + title: MongoDB Atlas Mongod Audit Dashboard + size: 600x600 + type: image/png - src: /img/mongodb_atlas-process-dashboard.png title: MongoDB Atlas Process Metrics Dashboard size: 600x600 @@ -26,12 +30,12 @@ icons: type: image/svg+xml policy_templates: - name: mongodb_atlas - title: MongoDB Atlas metrics - description: Collect MongoDB Atlas metrics. + title: MongoDB Atlas logs and metrics + description: Collect MongoDB Atlas logs and metrics. inputs: - type: cel - title: Collect MongoDB Atlas metrics - description: Collect MongoDB Atlas metrics via API. + title: Collect MongoDB Atlas logs and metrics + description: Collect MongoDB Atlas logs and metrics via API. vars: - name: url type: text