-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Audit/Computer/Distribution Groups Management Events - ECS related.user field mapping #15217
Conversation
Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually? |
1 similar comment
Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually? |
x-pack/winlogbeat/module/security/config/winlogbeat-security.js
Outdated
Show resolved
Hide resolved
x-pack/winlogbeat/module/security/config/winlogbeat-security.js
Outdated
Show resolved
Hide resolved
x-pack/winlogbeat/module/security/config/winlogbeat-security.js
Outdated
Show resolved
Hide resolved
x-pack/winlogbeat/module/security/config/winlogbeat-security.js
Outdated
Show resolved
Hide resolved
x-pack/winlogbeat/module/security/config/winlogbeat-security.js
Outdated
Show resolved
Hide resolved
x-pack/winlogbeat/module/security/config/winlogbeat-security.js
Outdated
Show resolved
Hide resolved
x-pack/winlogbeat/module/security/config/winlogbeat-security.js
Outdated
Show resolved
Hide resolved
x-pack/winlogbeat/module/security/config/winlogbeat-security.js
Outdated
Show resolved
Hide resolved
x-pack/winlogbeat/module/security/config/winlogbeat-security.js
Outdated
Show resolved
Hide resolved
x-pack/winlogbeat/module/security/config/winlogbeat-security.js
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks this is great.
I just had some thoughts about alternate names, let me know what you think.
Would you like me to generate the golden files for you?
// event.action Description Table | ||
var eventActionTypes = { | ||
"1100": "logging-shutdown", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe "logging-service-shutdown"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok! I'll change it
"4741": "added-computer-account", | ||
"4742": "changed-computer-account", | ||
"4743": "deleted-computer-account", | ||
"4744": "added-group-account", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"added-distribution-group-account"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok!
"4742": "changed-computer-account", | ||
"4743": "deleted-computer-account", | ||
"4744": "added-group-account", | ||
"4745": "changed-group-account", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"changed-distribution-group-account"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok!
"4743": "deleted-computer-account", | ||
"4744": "added-group-account", | ||
"4745": "changed-group-account", | ||
"4746": "added-group-account-to", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"added-member-to-distribution-group"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok!
"4744": "added-group-account", | ||
"4745": "changed-group-account", | ||
"4746": "added-group-account-to", | ||
"4747": "deleted-group-account-from", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"deleted-member-from-distribution-group"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok!
"4754": "added-group-account", | ||
"4755": "modified-group-account", | ||
"4756": "added-group-account-to", | ||
"4757": "deleted-group-account-from", | ||
"4758": "deleted-group-account", | ||
"4759": "added-group-account", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"added-distribution-group-account"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok!
"4754": "added-group-account", | ||
"4755": "modified-group-account", | ||
"4756": "added-group-account-to", | ||
"4757": "deleted-group-account-from", | ||
"4758": "deleted-group-account", | ||
"4759": "added-group-account", | ||
"4760": "changed-group-account", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"changed-distribution-group-account"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok!
"4754": "added-group-account", | ||
"4755": "modified-group-account", | ||
"4756": "added-group-account-to", | ||
"4757": "deleted-group-account-from", | ||
"4758": "deleted-group-account", | ||
"4759": "added-group-account", | ||
"4760": "changed-group-account", | ||
"4761": "added-group-account-to", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"added-member-to-distribution-group"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok!
"4759": "added-group-account", | ||
"4760": "changed-group-account", | ||
"4761": "added-group-account-to", | ||
"4762": "deleted-group-account-from", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"removed-member-from-distribution-group"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok!
"4760": "changed-group-account", | ||
"4761": "added-group-account-to", | ||
"4762": "deleted-group-account-from", | ||
"4763": "deleted-group-account", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"deleted-distribution-group-account"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok!
Hi @leehinman , Thank you! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Found a bug while generating the golden files.
x-pack/winlogbeat/module/security/config/winlogbeat-security.js
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Found a bug while generating the golden files.
Hi @leehinman . I have removed the Group Type related fields and functions. Now it should work when generating golden files.
I think it does not add valuable information at the moment. I'll keep thinking in how to add that information for a future PR
@janniten Thanks. I was able to produce the golden files. I noticed one odd thing. In security-windows2016_4723_Password_Change.evtx the Subject & Target user are the same, so when they get added to the related.user list we get a duplicates. That seems like a bug to me. What do you think? I was thinking we could use a Set in addRelatedUser to make sure the values are unique. Sound OK to you?
One other small thing. I noticed that
Was in a few of the golden files. I think if we add
to renameCommonAuthFields that will prevent the nulls |
x-pack/winlogbeat/module/security/config/winlogbeat-security.js
Outdated
Show resolved
Hide resolved
x-pack/winlogbeat/module/security/config/winlogbeat-security.js
Outdated
Show resolved
Hide resolved
Hi @leehinman
How are the golden files generated? Next time I can try to generate the golden files for myself and detect errors in an early stage. |
If the problem is that you are getting duplicate values in a string array, then you can use |
Thank you @andrewkroh. |
Sorry about the Set suggestion, luckily @andrewkroh had a much better suggestion :-) You can make the golden files by being in the "C:\Gopath\src\github.com\elastic\beats\x-pack\winlogbeat\module\security\test" directory and running "go test -update". I found one more issue, in win2019 the SubcategoryGuid is lowercase, not uppercase so the addAuditInfo function wasn't adding subcategor and AuditPolicyChangesDescription. Easy fix with adding toUpperCase before doing the lookup. I'll add that to the same commit as the golden files. |
Thank you @leehinman |
if (!computer) { | ||
return; | ||
} | ||
evt.Put("winlog.computer.name", computer.split(".")[0]); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would not make this assumption (and the next line) that these are the computer and domain by splitting .
- because many organizations can/do have multi level domains (ie: addomain.mydomain.local)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@neu5ron Does this make sense ?
evt.Put("winlog.computer.name", computer.split(".")[0]);
evt.Put("winlog.computer.domain", computer.split(".").slice(1).join('.');
If computer name is computer.addomain.mydomain.local
winlog.computer.name -> computer
winlog.computer.domain -> addomain.mydomain.local
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Many of these windows logs will already have the domain, and for the one's that don't - if the log does not have the domain, unless one can explicitly know with certainty what the domain is then I would not try to pull the domain from the computer name.. Otherwise there can be false/incorrect information..
Hope that makes sense, if not let me know :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@neu5ron you are correct. Thank you for your feedback.
If the computer is not joined or member of the domain I cannot fetch the domain name from the computer name.
In the case of the audit events like 1100, there is no other field in the event from where the domain name can be obtained; but to keep consistency and correctness I'll remove the addComputerData function
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@neu5ron @leehinman I have removed the addComputerData function.
Please let me know if I have to modify something else
Thank you!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@janniten I think the PR needs to be rebased so there aren't merge conflicts and the CI tests will run.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@leehinman . Done!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@leehinman Let me know I know is ok. Thank you!
jenkins, test this |
1 similar comment
jenkins, test this |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to update the list of event IDs in the module's asciidoc file. Can you do that in this PR please?
Thanks for your contribution.
|
||
var addRelatedUser= function(evt,user) { | ||
var related_user = evt.Get("related.user"); | ||
if (!related_user) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From and Elasticsearch document standpoint this is not necessary. You can just call evt.AppendTo
in both cases and it will work. Whether the value is a string or an array or strings it's going to be handled the same way in ES.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have modified the code in order to use directly the evt.AppentTo(related.user,user_to_add), less functions calls, less code, more efficiency :)
Thank you for the suggestion. I'll have this in mind for futures PRs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@andrewkroh some checks fail regarding to the documentation update.
I have added the events ID in \beats\x-pack\winlogbeat\module\security_meta\docs.asciidoc.
Shall I do something more?
@andrewkroh I have aldready added the events ID. I have also added the ones in #14299 that alse were missing. |
Pinging @elastic/siem (Team:SIEM) |
jenkins, test this |
jenkins, test this |
jenkins, test this |
🍾 Thank you @janniten for your contributions to Winlogbeat and ECS! |
…module (elastic#15217) Added Audit and Log Management related events, Computer Object Management Events, Distribution Groups Events. Changed user.name field for user management events and related.user mapping. New Events Due to that Windows events are the source of information for Winlogbeat the events 1100, 1102, 1104, 1105, 1108 and 4719 has been added in order to monitor changes in the audit policy configuration, log deletion and other failures in the log subsystem. For event 4719, a human readable description was added in order to know which setting was modified (winlog.event_data.SubCategory) and to which value (winlog.event_data.AuditPolicyChangesDescription). Distribution Groups (Security-Disabled) Management Events were added. Those events are processed in the same way and with the same function that Security Groups (elastic#14299). In order to add information about the nature of the group being managed the type (Security-Disabled/Security-Enabled) and scope (Local,Global,Universal) where added as winlog.group.type and winlog.group.scope. ComputerObject Management events were also added. Changes to ECS mappings In elastic/ecs#678 and elastic/ecs#589 we have been discussing how n-ary relationship between users in an event should be named and mapping into ECS. In elastic#13530 winlog.event_data.TargetUserName has been mapped to user.name but from the reasons exposed in elastic/ecs#678 and elastic/ecs#589 the mapping winlog.event_data.SubjectUserName -> user.name is more appropriate. This mapping was changed. Also, with the adding of related fields in ECS 1.3 and specifically the related.user field (elastic/ecs#694) all the user names appearing in one event were mapped to the related user events. Every time a SubjectUserName or TargetUserName is copied also is added to the related.user field, as well as other users appearing in the event. Event test data were added for all events with the exception of event 1108 which I was not able to reproduce. Co-authored-by: Lee Hinman <[email protected]> Co-authored-by: Andrew Kroh <[email protected]> (cherry picked from commit e624aef)
…ent Events - ECS related.user field mapping (#17090) Added Audit and Log Management related events, Computer Object Management Events, Distribution Groups Events. Changed user.name field for user management events and related.user mapping. New Events Due to that Windows events are the source of information for Winlogbeat the events 1100, 1102, 1104, 1105, 1108 and 4719 has been added in order to monitor changes in the audit policy configuration, log deletion and other failures in the log subsystem. For event 4719, a human readable description was added in order to know which setting was modified (winlog.event_data.SubCategory) and to which value (winlog.event_data.AuditPolicyChangesDescription). Distribution Groups (Security-Disabled) Management Events were added. Those events are processed in the same way and with the same function that Security Groups (#14299). In order to add information about the nature of the group being managed the type (Security-Disabled/Security-Enabled) and scope (Local,Global,Universal) where added as winlog.group.type and winlog.group.scope. ComputerObject Management events were also added. Changes to ECS mappings In elastic/ecs#678 and elastic/ecs#589 we have been discussing how n-ary relationship between users in an event should be named and mapping into ECS. In #13530 winlog.event_data.TargetUserName has been mapped to user.name but from the reasons exposed in elastic/ecs#678 and elastic/ecs#589 the mapping winlog.event_data.SubjectUserName -> user.name is more appropriate. This mapping was changed. Also, with the adding of related fields in ECS 1.3 and specifically the related.user field (elastic/ecs#694) all the user names appearing in one event were mapped to the related user events. Every time a SubjectUserName or TargetUserName is copied also is added to the related.user field, as well as other users appearing in the event. Event test data were added for all events with the exception of event 1108 which I was not able to reproduce. Co-authored-by: Lee Hinman <[email protected]> Co-authored-by: Andrew Kroh <[email protected]> Co-authored-by: Anabella Cristaldi <[email protected]> (cherry picked from commit e624aef)
…18775) This PR adds two new dashboards related to events added in PRs (#12906, #14299, #15217, #17517) and implements some improvements to existing winlogbeat security module's dashboard New Dashboards User Logon Dashboard shows all the logon information. It allow us to keep track between logon and admin logons event between RDP connections and disconnections. Failed and Blocked Accounts allow us to keep track to failed logons and locked out account Existing Dashboards Added Distribution groups Events (#15217) Found that Event 4625 can be generated by two different providers: Microsoft-Windows-Security-Auditing and Microsoft-Windows-EventSystem. Filters to use only the event.code=4625 from Microsoft-Windows-Security-Auditing where added All Dashboards Markdown with links all the winlogbeat security dashboards where added (following the idea of Filebeats, Auditbeat dashboards) image Visualization that use may events (like group management related visualizations) were modified in order to use a saved search with the relevant events for that visualization as a source (instead of using individual filters for each visualization) Removed the margin between panels to look in the same way that other beats dashboards TSVB metrics were modified to use the Entire Time Range and to use eye-friendlier colors Co-authored-by: Andrew Kroh <[email protected]>
…lastic#18775) This PR adds two new dashboards related to events added in PRs (elastic#12906, elastic#14299, elastic#15217, elastic#17517) and implements some improvements to existing winlogbeat security module's dashboard New Dashboards User Logon Dashboard shows all the logon information. It allow us to keep track between logon and admin logons event between RDP connections and disconnections. Failed and Blocked Accounts allow us to keep track to failed logons and locked out account Existing Dashboards Added Distribution groups Events (elastic#15217) Found that Event 4625 can be generated by two different providers: Microsoft-Windows-Security-Auditing and Microsoft-Windows-EventSystem. Filters to use only the event.code=4625 from Microsoft-Windows-Security-Auditing where added All Dashboards Markdown with links all the winlogbeat security dashboards where added (following the idea of Filebeats, Auditbeat dashboards) image Visualization that use may events (like group management related visualizations) were modified in order to use a saved search with the relevant events for that visualization as a source (instead of using individual filters for each visualization) Removed the margin between panels to look in the same way that other beats dashboards TSVB metrics were modified to use the Entire Time Range and to use eye-friendlier colors Co-authored-by: Andrew Kroh <[email protected]>
…lastic#18775) This PR adds two new dashboards related to events added in PRs (elastic#12906, elastic#14299, elastic#15217, elastic#17517) and implements some improvements to existing winlogbeat security module's dashboard New Dashboards User Logon Dashboard shows all the logon information. It allow us to keep track between logon and admin logons event between RDP connections and disconnections. Failed and Blocked Accounts allow us to keep track to failed logons and locked out account Existing Dashboards Added Distribution groups Events (elastic#15217) Found that Event 4625 can be generated by two different providers: Microsoft-Windows-Security-Auditing and Microsoft-Windows-EventSystem. Filters to use only the event.code=4625 from Microsoft-Windows-Security-Auditing where added All Dashboards Markdown with links all the winlogbeat security dashboards where added (following the idea of Filebeats, Auditbeat dashboards) image Visualization that use may events (like group management related visualizations) were modified in order to use a saved search with the relevant events for that visualization as a source (instead of using individual filters for each visualization) Removed the margin between panels to look in the same way that other beats dashboards TSVB metrics were modified to use the Entire Time Range and to use eye-friendlier colors Co-authored-by: Andrew Kroh <[email protected]> (cherry picked from commit 7b9c535)
…18775) (#22598) This PR adds two new dashboards related to events added in PRs (#12906, #14299, #15217, #17517) and implements some improvements to existing winlogbeat security module's dashboard New Dashboards User Logon Dashboard shows all the logon information. It allow us to keep track between logon and admin logons event between RDP connections and disconnections. Failed and Blocked Accounts allow us to keep track to failed logons and locked out account Existing Dashboards Added Distribution groups Events (#15217) Found that Event 4625 can be generated by two different providers: Microsoft-Windows-Security-Auditing and Microsoft-Windows-EventSystem. Filters to use only the event.code=4625 from Microsoft-Windows-Security-Auditing where added All Dashboards Markdown with links all the winlogbeat security dashboards where added (following the idea of Filebeats, Auditbeat dashboards) image Visualization that use may events (like group management related visualizations) were modified in order to use a saved search with the relevant events for that visualization as a source (instead of using individual filters for each visualization) Removed the margin between panels to look in the same way that other beats dashboards TSVB metrics were modified to use the Entire Time Range and to use eye-friendlier colors Co-authored-by: Andrew Kroh <[email protected]> (cherry picked from commit 7b9c535) Co-authored-by: Anabella Cristaldi <[email protected]>
Added Audit and Log Management related events, Computer Object Management Events, Distribution Groups Events - Change in user.name for user management events and related.user mapping
New Events
Due to that Windows events are the source of information for Winlogbeat the events 1100,1102,1104,1105,1108 and 4719 has been added in order to monitor changes in the audit policy configuration, log deletion and other failures in the log subsystem.
For event 4719 a human readable description was added in order to know which setting was modified (winlog.event_data.SubCategory) and to which value (winlog.event_data.AuditPolicyChangesDescription)
Distribution Groups (Security-Disabled) Management Events were added. Those events are processed in the same way and with the same function that Security Groups (#14299). In order to add information about the nature of the group being managed the type (Security-Disabled/Security-Enabled) and scope (Local,Global,Universal) where added as winlog.group.type and winlog.group.scope
ComputerObject Management events were added
Changes to ECS mappings
In elastic/ecs#678 and elastic/ecs#589 we have been discussing how n-ary relationship between users in an event should be named and mapping into ECS.
In #13530 winlog.event_data.TargetUserName has been mapped to user.name but from the reasons exposed in elastic/ecs#678 and elastic/ecs#589 the mapping winlog.event_data.SubjectUserName -> user.name is more appropriate. This mapping was changed.
Also, with the adding of related fields in ECS 1.3 and specifically the related.user field (elastic/ecs#694) all the user names appearing in one event were mapped to the related user events.
Every time a SubjectUserName or TargetUserName is copied also is added to the related.user field, as well as other users appearing in the event.
Event test data were added for all events with the exception of event 1108 which I was not able to reproduce