-
Notifications
You must be signed in to change notification settings - Fork 8.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[ML] Adds new security module to detect anomalous activity in host-ba…
…sed logs (#195582) ## Summary Adds a new security module `Security: Host` to the prebuilt security jobs. The module has the following jobs: - Detect Spike in Host based traffic - Detect Decrease in Host based traffic
- Loading branch information
1 parent
289bb16
commit 5ed4297
Showing
9 changed files
with
191 additions
and
1 deletion.
There are no files selected for viewing
3 changes: 3 additions & 0 deletions
3
x-pack/plugins/ml/server/models/data_recognizer/modules/security_host/logo.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
{ | ||
"icon": "logoSecurity" | ||
} |
60 changes: 60 additions & 0 deletions
60
x-pack/plugins/ml/server/models/data_recognizer/modules/security_host/manifest.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
{ | ||
"id": "security_host", | ||
"title": "Security: Host", | ||
"description": "Detect anomalous activity in your ECS-compatible host-based logs.", | ||
"type": "Host data", | ||
"logoFile": "logo.json", | ||
"defaultIndexPattern": "auditbeat-*,logs-*,filebeat-*,winlogbeat-*", | ||
"query": { | ||
"bool": { | ||
"filter": [ | ||
{ | ||
"exists": { | ||
"field": "event.category" | ||
} | ||
}, | ||
{ | ||
"exists": { | ||
"field": "host.name" | ||
} | ||
}, | ||
{ | ||
"exists": { | ||
"field": "event.dataset" | ||
} | ||
}, | ||
{ | ||
"term": { | ||
"event.outcome": "success" | ||
} | ||
} | ||
], | ||
"must_not": { "terms": { "_tier": ["data_frozen", "data_cold"] } } | ||
} | ||
}, | ||
"jobs": [ | ||
{ | ||
"id": "high_count_events_for_a_host_name", | ||
"file": "high_count_events_for_a_host_name.json" | ||
}, | ||
{ | ||
"id": "low_count_events_for_a_host_name", | ||
"file": "low_count_events_for_a_host_name.json" | ||
} | ||
], | ||
"datafeeds": [ | ||
{ | ||
"id": "datafeed-high_count_events_for_a_host_name", | ||
"file": "datafeed_high_count_events_for_a_host_name.json", | ||
"job_id": "high_count_events_for_a_host_name" | ||
}, | ||
{ | ||
"id": "datafeed-low_count_events_for_a_host_name", | ||
"file": "datafeed_low_count_events_for_a_host_name.json", | ||
"job_id": "low_count_events_for_a_host_name" | ||
} | ||
], | ||
"tags": [ | ||
"security" | ||
] | ||
} |
33 changes: 33 additions & 0 deletions
33
.../data_recognizer/modules/security_host/ml/datafeed_high_count_events_for_a_host_name.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
{ | ||
"job_id": "JOB_ID", | ||
"indices": [ | ||
"INDEX_PATTERN_NAME" | ||
], | ||
"max_empty_searches": 10, | ||
"query": { | ||
"bool": { | ||
"filter": [ | ||
{ | ||
"exists": { | ||
"field": "event.category" | ||
} | ||
}, | ||
{ | ||
"exists": { | ||
"field": "host.name" | ||
} | ||
}, | ||
{ | ||
"exists": { | ||
"field": "event.dataset" | ||
} | ||
}, | ||
{ | ||
"term": { | ||
"event.outcome": "success" | ||
} | ||
} | ||
] | ||
} | ||
} | ||
} |
33 changes: 33 additions & 0 deletions
33
...s/data_recognizer/modules/security_host/ml/datafeed_low_count_events_for_a_host_name.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
{ | ||
"job_id": "JOB_ID", | ||
"indices": [ | ||
"INDEX_PATTERN_NAME" | ||
], | ||
"max_empty_searches": 10, | ||
"query": { | ||
"bool": { | ||
"filter": [ | ||
{ | ||
"exists": { | ||
"field": "event.category" | ||
} | ||
}, | ||
{ | ||
"exists": { | ||
"field": "host.name" | ||
} | ||
}, | ||
{ | ||
"exists": { | ||
"field": "event.dataset" | ||
} | ||
}, | ||
{ | ||
"term": { | ||
"event.outcome": "success" | ||
} | ||
} | ||
] | ||
} | ||
} | ||
} |
29 changes: 29 additions & 0 deletions
29
...er/models/data_recognizer/modules/security_host/ml/high_count_events_for_a_host_name.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
{ | ||
"description": "Security: Host - Looks for a sudden spike in host based traffic. This can be due to a range of security issues, such as a compromised system, DDoS attacks, malware infections, privilege escalation, or data exfiltration.", | ||
"groups": ["security", "host"], | ||
"analysis_config": { | ||
"bucket_span": "3h", | ||
"detectors": [ | ||
{ | ||
"detector_description": "high count of host based events", | ||
"function": "high_count", | ||
"partition_field_name": "host.name", | ||
"detector_index": 0 | ||
} | ||
], | ||
"influencers": ["host.name", "host.ip", "event.dataset", "event.action", "event.category"] | ||
}, | ||
"allow_lazy_open": true, | ||
"analysis_limits": { | ||
"model_memory_limit": "128mb" | ||
}, | ||
"data_description": { | ||
"time_field": "@timestamp" | ||
}, | ||
"custom_settings": { | ||
"created_by": "ml-module-security-host", | ||
"security_app_display_name": "Spike in the Host Traffic", | ||
"managed": true, | ||
"job_revision": 1 | ||
} | ||
} |
29 changes: 29 additions & 0 deletions
29
...ver/models/data_recognizer/modules/security_host/ml/low_count_events_for_a_host_name.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
{ | ||
"description": "Security: Host - Looks for a sudden drop in host based traffic. This can be due to a range of security issues, such as a compromised system, a failed service, or a network misconfiguration.", | ||
"groups": ["security", "host"], | ||
"analysis_config": { | ||
"bucket_span": "3h", | ||
"detectors": [ | ||
{ | ||
"detector_description": "low count of host based events", | ||
"function": "low_count", | ||
"partition_field_name": "host.name", | ||
"detector_index": 0 | ||
} | ||
], | ||
"influencers": ["host.name", "host.ip", "event.dataset", "event.action", "event.category"] | ||
}, | ||
"allow_lazy_open": true, | ||
"analysis_limits": { | ||
"model_memory_limit": "128mb" | ||
}, | ||
"data_description": { | ||
"time_field": "@timestamp" | ||
}, | ||
"custom_settings": { | ||
"created_by": "ml-module-security-host", | ||
"security_app_display_name": "Decrease in the Host Traffic", | ||
"managed": true, | ||
"job_revision": 1 | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters