Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Alerts are not triggered for windows OS if user add the exception for MAC OS Only. #102613

Closed
ghost opened this issue Jun 18, 2021 · 13 comments · Fixed by #106494
Assignees
Labels
bug Fixes for quality problems that affect the customer experience impact:critical This issue should be addressed immediately due to a critical level of impact on the product. OLM Sprint Team:Defend Workflows “EDR Workflows” sub-team of Security Solution Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v7.14.0

Comments

@ghost
Copy link

ghost commented Jun 18, 2021

Description
Alerts are not triggered for windows OS if user add the exception for MAC OS Only.

Build Details:

Version: 7.14.0-SNAPSHOT
Commit: 9838db392e7fcfc12f004b68fb1b09739f131148
Build Hash: 41559
Artifact Page : https://artifacts-api.elastic.co/v1/search/7.14.0-SNAPSHOT

Browser Details:
All

Preconditions:

  1. Kibana user should be logged in.
  2. Alerts should be generated

Steps to Reproduce:

  1. Navigate to detection tab of security.
  2. Click on Endpoint security rule.
  3. Scroll down and click on "exception" tab.
  4. Click on "Add new Exception" and select "Endpoint exception".
  5. Select the Mac OS from opreating system filed.
  6. Add the exception with below fields:
    ["event.module" "is" "malicious_file"]
  7. Generate the alerts [Say: mimikatz] from Windows OS.
  8. Observe that alerts are not triggered for windows OS

Impacted Test case:
N/A

Actual Result:
Alerts are not triggered for windows OS if user add the exception for MAC OS Only.

Expected Result:
Alerts should be triggered for windows OS if user add the exception for MAC OS Only.

What's working:
N/A

What's not working:
N/A

Screenshot:
OS_Exception

mac_os

@ghost ghost added bug Fixes for quality problems that affect the customer experience Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. labels Jun 18, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@ghost ghost added the v7.14.0 label Jun 18, 2021
@ghost
Copy link
Author

ghost commented Jun 18, 2021

@manishgupta-qasource Please review!!

@manishgupta-qasource
Copy link

Reviewed & Assigned to @MadameSheema

@manishgupta-qasource manishgupta-qasource added the impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. label Jun 18, 2021
@MadameSheema MadameSheema removed their assignment Jun 18, 2021
@MadameSheema MadameSheema added the Team:Detections and Resp Security Detection Response Team label Jun 18, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@kevinlog kevinlog self-assigned this Jun 18, 2021
@kevinlog kevinlog added Team:Defend Workflows “EDR Workflows” sub-team of Security Solution and removed Team:Detections and Resp Security Detection Response Team labels Jun 18, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-onboarding-and-lifecycle-mgt (Team:Onboarding and Lifecycle Mgt)

@kevinlog
Copy link
Contributor

Chatted with @peluja1012 offline - our team can take this since we (me specifically) introduced this nuance.

Proposed solution is to introduce a third option "Windows and macOS" so that users have the flexibility to create Exceptions for both OSs from scratch. Linux still needs to remain exclusive because we must form file.path values in a way to make them case sensitive.

Options in the dropdown would be:

  • Windows
  • MacOS
  • Window & MaOS
  • Linux

Selected OSs for new Exceptions will show the OS they apply to in the UI as the do today, we'll just make sure users can still create them for Mac and Windows together in one action if they choose to preserve the older functionality.

@kevinlog
Copy link
Contributor

PR: #103404

FYI @peluja1012

@kevinlog kevinlog added the QA:Ready for Testing Code is merged and ready for QA to validate label Jun 28, 2021
@ghost
Copy link
Author

ghost commented Jul 15, 2021

Hi @kevinlog,

We are blocked to test this ticket due to #104442.

We will validate the same once the issue #104442 got resolved.

Thanks!!

@ghost
Copy link
Author

ghost commented Jul 21, 2021

Hi @kevinlog,

We have validated this ticket on 7.14.0 BC3 build and Please Find the below observations:

Build Details:

VERSION: 7.14.0 BC3
BUILD: 42545
COMMIT: c314921a9893e0b46d9a3958f5520e3d6b1ce7d5
ARTIFACT: https://staging.elastic.co/7.14.0-682a8012/summary-7.14.0.html

Observation 1: "Windows and mac OS" is displaying in drop down when create the exception from scratch and successfully able to create the exception.

Screenshot:
exception

Linux

Observation 2: We have same field "event.code" with same value "malicious_file" for all OS i.e, Windows, Mac OS, Linux. when we create the exception from scratch for only one OS e.g: Mac OS with field "event.code" then the alert is not trigger from Linux and windows as well. Please find the below steps:

Steps:

  1. Create the exception from scratch for Mac OS with field "event.code".
  2. After added the exception generate the alerts from all OS [Windows, Mac OS, Linux].
  3. Observe that alerts will not trigger from any OS.

Screenshot:
event code

Could you please confirm if it is expected.

Thanks!!

@ghost
Copy link
Author

ghost commented Jul 21, 2021

Hi @kevinlog,

Please Find the below observations for this issue:

Build Details:

VERSION: 7.14.0 BC3
BUILD: 42545
COMMIT: c314921a9893e0b46d9a3958f5520e3d6b1ce7d5
ARTIFACT: https://staging.elastic.co/7.14.0-682a8012/summary-7.14.0.html

Observation: When we add the exception from scratch for MacOS with field "event.code" value "malicious_file", and trigger the alert from Windows host then we get the notification on windows host but the windows alert did not streamed to Kibana.

Screenshot:
alert_notification

image

Thanks!!

@kevinlog
Copy link
Contributor

thanks for the update @deepikakeshav-qasource

I can verify that the alert documents from the Windows host are still successfully streaming to ES. See the below in Discover.

image

The Alert is triggering on the Endpoint and it's being streamed to ES. It seems like it might be a bug in the Detection engine.

FYI @peluja1012 on this - I'm digging more and going to reproduce. This seems like a critical issue.

@kevinlog kevinlog added impact:critical This issue should be addressed immediately due to a critical level of impact on the product. and removed QA:Ready for Testing Code is merged and ready for QA to validate impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. labels Jul 21, 2021
FrankHassanabad added a commit that referenced this issue Jul 22, 2021
… type (#106494)

## Summary

Fixes #102613, and targets `7.14.0` as a blocker/critical

Previously we never fully finished the plumbing for using the `os_types` (operating system type) in the exception lists to be able to filter out values based on this type. With the endpoint exceptions now having specific selections for os_type we have to filter it with exceptions and basically make it work.

Some caveats is that the endpoints utilize `host.os.name.casless` for filtering against os_type, while agents such as auditbeat, winlogbeat, etc... use `host.os.type`. Really `host.os.type` is the correct ECS field to use, but to retain compatibility with the current version of endpoint agents I support both in one query to where if either of these two matches, then that will trigger the exceptions.

* Adds e2e tests
* Enhances the e2e tooling to do endpoint exception testing with `os_types`.
* Adds the logic to handle os_type
* Updates the unit tests

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Jul 22, 2021
… type (elastic#106494)

## Summary

Fixes elastic#102613, and targets `7.14.0` as a blocker/critical

Previously we never fully finished the plumbing for using the `os_types` (operating system type) in the exception lists to be able to filter out values based on this type. With the endpoint exceptions now having specific selections for os_type we have to filter it with exceptions and basically make it work.

Some caveats is that the endpoints utilize `host.os.name.casless` for filtering against os_type, while agents such as auditbeat, winlogbeat, etc... use `host.os.type`. Really `host.os.type` is the correct ECS field to use, but to retain compatibility with the current version of endpoint agents I support both in one query to where if either of these two matches, then that will trigger the exceptions.

* Adds e2e tests
* Enhances the e2e tooling to do endpoint exception testing with `os_types`.
* Adds the logic to handle os_type
* Updates the unit tests

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Jul 22, 2021
… type (elastic#106494)

## Summary

Fixes elastic#102613, and targets `7.14.0` as a blocker/critical

Previously we never fully finished the plumbing for using the `os_types` (operating system type) in the exception lists to be able to filter out values based on this type. With the endpoint exceptions now having specific selections for os_type we have to filter it with exceptions and basically make it work.

Some caveats is that the endpoints utilize `host.os.name.casless` for filtering against os_type, while agents such as auditbeat, winlogbeat, etc... use `host.os.type`. Really `host.os.type` is the correct ECS field to use, but to retain compatibility with the current version of endpoint agents I support both in one query to where if either of these two matches, then that will trigger the exceptions.

* Adds e2e tests
* Enhances the e2e tooling to do endpoint exception testing with `os_types`.
* Adds the logic to handle os_type
* Updates the unit tests

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
kibanamachine added a commit that referenced this issue Jul 22, 2021
… type (#106494) (#106596)

## Summary

Fixes #102613, and targets `7.14.0` as a blocker/critical

Previously we never fully finished the plumbing for using the `os_types` (operating system type) in the exception lists to be able to filter out values based on this type. With the endpoint exceptions now having specific selections for os_type we have to filter it with exceptions and basically make it work.

Some caveats is that the endpoints utilize `host.os.name.casless` for filtering against os_type, while agents such as auditbeat, winlogbeat, etc... use `host.os.type`. Really `host.os.type` is the correct ECS field to use, but to retain compatibility with the current version of endpoint agents I support both in one query to where if either of these two matches, then that will trigger the exceptions.

* Adds e2e tests
* Enhances the e2e tooling to do endpoint exception testing with `os_types`.
* Adds the logic to handle os_type
* Updates the unit tests

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios

Co-authored-by: Frank Hassanabad <[email protected]>
kibanamachine added a commit that referenced this issue Jul 22, 2021
… type (#106494) (#106597)

## Summary

Fixes #102613, and targets `7.14.0` as a blocker/critical

Previously we never fully finished the plumbing for using the `os_types` (operating system type) in the exception lists to be able to filter out values based on this type. With the endpoint exceptions now having specific selections for os_type we have to filter it with exceptions and basically make it work.

Some caveats is that the endpoints utilize `host.os.name.casless` for filtering against os_type, while agents such as auditbeat, winlogbeat, etc... use `host.os.type`. Really `host.os.type` is the correct ECS field to use, but to retain compatibility with the current version of endpoint agents I support both in one query to where if either of these two matches, then that will trigger the exceptions.

* Adds e2e tests
* Enhances the e2e tooling to do endpoint exception testing with `os_types`.
* Adds the logic to handle os_type
* Updates the unit tests

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios

Co-authored-by: Frank Hassanabad <[email protected]>
@ghost
Copy link
Author

ghost commented Jul 26, 2021

Hi @MadameSheema,

We have validated this ticket on 7.14.0 SNAPSHOT build and observed that issue is Fixed.

Alerts are triggered when we add the exception from scratch for MacOS with field "event.code" value "malicious_file", and trigger the alert from Windows host and Linux. The alerts are triggered successfully.

Build Details:

VERSION: 7.14.0 SNAPSHOT
BUILD: 42683
COMMIT: 88eb5cdd510be2ad9bf1d50506a53d68db9c24c1
ARTIFACT: https://artifacts-api.elastic.co/v1/search/7.14.0-SNAPSHOT

Observations:

Exception Steps Result
Created the exception for MacOS with field "event.code" value "malicious_file" Trigger the alerts from Windows host and Linux Alerts are trigger only from Windows and linux ✔️
Created the exception for Windows with field "event.code" value "malicious_file" Trigger the alerts from MacOS and Linux Alerts are trigger only from MacOS and Linux ✔️
Created the exception for Linux with field "event.code" value "malicious_file" Trigger the alerts from MacOS and Windows Alerts are trigger only from Windows and MacOS ✔️
Created the exception for MacOS and Windows with field "event.code" value "malicious_file" Trigger the alerts from Linux Alerts are trigger only from Linux ✔️

Screenshot:
Windows_mac
Linux Alerts

Thanks!!

@ghost
Copy link

ghost commented Aug 16, 2021

Bug Conversion

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience impact:critical This issue should be addressed immediately due to a critical level of impact on the product. OLM Sprint Team:Defend Workflows “EDR Workflows” sub-team of Security Solution Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v7.14.0
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants