From 7b07b64ec0651f3ba96afa97e6bb1011e3184014 Mon Sep 17 00:00:00 2001 From: Mathieu Martin Date: Thu, 6 Aug 2020 16:20:29 -0400 Subject: [PATCH 01/24] Start the RFC for multiple users --- rfcs/text/0000-multiple-users.md | 386 +++++++++++++++++++++++++++++++ 1 file changed, 386 insertions(+) create mode 100644 rfcs/text/0000-multiple-users.md diff --git a/rfcs/text/0000-multiple-users.md b/rfcs/text/0000-multiple-users.md new file mode 100644 index 0000000000..8940390b91 --- /dev/null +++ b/rfcs/text/0000-multiple-users.md @@ -0,0 +1,386 @@ +# 0000: Multiple users in an event + + +- Stage: **2 (proposal)** +- Date: **TBD** + +Many log events refer to more than one user at the same time. +Examples of this are remote logons as someone else, user management and privilege escalation. +ECS supports some of these situations already, via the fact that the "user" fields are reused inside other field sets (e.g. `source.user.*` and `destination.user.*`). + +The purpose of this proposal is two-fold: + +1. Define additional places where the user fields can be used, to support situations that aren't currently covered. +2. Review and clarify the purpose of all the places the user fields are currently defined. If some of them appear unneeded, we will also consider removing them. + +## Fields + +As of ECS 1.5, user fields can be present in an event in all of the following places: + +* `user.*` +* `host.user.*` +* `source.user.*` +* `destination.user.*` +* `client.user.*` +* `server.user.*` + +The new fields proposed in this RFC are the following: + +* `user.effective.*` +* `user.target.*` +* `user.changes.*` + +Notice also that the user fields are now being nested as another name. +The purpose is to hint at their role, when used in these locations. + +It's also important to point out that the reuses of `user` inside other field sets +are not meant to inherit these new subsections inside the user field set. +For example: `source.user.*` will **not** contain `source.user.effective.*` and so on. + +The current reusable locations `user` will be amended to include a few more entries, +as demonstrated below. + +```YAML + reusable: + top_level: true + expected: + - client + - destination + - host + - server + - source + # Added for this RFC + - at: user + as: target + - at: user + as: effective + - at: user + as: changes +``` + +The `user` field set contains 9 leaf fields, 2 of which have a `.text` multi-field, +for a total of 11. These new nestings will therefore add a total of 33 fields. +This can also be seen on the pre-existing PR [ecs#869](https://github.com/elastic/ecs/pull/869). + +## Usage + +The examples below will mostly populate `user.name` inside the various `user` nestings, +for readability. Unless otherwise noted, all `user` fields that can reasonably be +populated in each location should be populated. + +### `user` at the root of an event + +Any event that only has one user should populate the user fields at the root of an +event. Any time more than one user is present in an event, the `user` fields +at the root of an event should be interpreted as the user performing the action. + +As many of the user fields as possible should be populated. + +In cases where a purpose-specific user field such as `url.username` is populated, +`user.name` should also be populated with the same user name. + +### Remote logons + +When users are crossing host boundaries, the users are captured at +`source.user` and `destination.user`. + +Examples of data sources where this is applicable: + +* Remote logons via ssh, kerberos +* Firewalls observing network traffic + +In order to align with ECS' design of having `user` at the root of the event as the +user performing the action, all `source.user` fields should be copied to `user` at the root. + +Here's an example where user "alice" logs on to another host as user "deus": + +```JSON +{ + "user": { + "name": "alice" + }, + "source": { + "user": { + "name": "alice" + }, + "ip": "10.42.42.42" + }, + "destination": { + "user": { + "name": "deus" + }, + "ip": "10.42.42.43" + } +} +``` + + + +(also applies to `client.user` and `server.user`) + +### Privilege Changes + +The `user.effective` fields are relevant when there's a privilege escalation or demotion +and it's possible to determine the user requesting/performing the escalation. + +Use the `user` fields at the root to capture who is requesting the privilege change, +and `user.effective` to capture the requested privilege level, whether or not the +privilege change was successful. + +Here are examples where this is applicable: + +* A user changing identity on a host. + * Examples: sudo, su, Run as. +* Running a program as a different user. Examples: + * A trusted user runs a specific admin command as root via a mechanism such as the Posix setuid/setgid. + * A service manager with administrator privileges starts child processes as limited + users, for security purposes (e.g. root runs Apache HTTPD as user "apache") + +In cases where the event source only gives information about the effective user +and not who requested different privileges, the `user` fields at the root of the +event should be used instead. + +Here's an example of user "alice" running a command as root via sudo: + +```JSON +{ + "user": { + "name": "alice", + "id": "1001", + "effective": { + "name": "root", + "id": "1" + } + } +} +``` + +### Identity and Access Management + +Whenever a user is performing an action that affects another user -- typically +in IAM scenarios -- the user affected by the action is captured at +`user.target`. The user performing the IAM activity is captured at the root +of the event. + +Examples of IAM activity include: + +* user-a creates or deletes user-b +* user-a modifies user-b + +In the create/delete scenarios, there's either no prior state (user creation) +or no post state (user deletion). In these cases, only `user` at the root and +`user.target` must be populated. + +Example where "root" creates user "bob": + +```JSON +{ + "user": { + "name": "root", + "id": "1", + "target": { + "name": "bob", + "id": "1002", + ... + } + } +} +``` + +When there's a change of state to an existing user, `user.target` must be used +to capture the prior state of the user, and `user.changes` should list only +the changes that were performed. + +Example where "root" renames user "bob" to "bob.barker": + +```JSON +{ + "user": { + "name": "root", + "id": "1", + "target": { + "name": "bob", + "id": "1002" + }, + "changes": { + "name": "bob.barker" + } + } +} +``` + +You'll note in the example above that the user ID is not repeated under changes, +since the ID didn't change. + +### Combining IAM and Privilege Escalation + +We've covered above how `user.target` and `user.changes` can be used at the same time. +If privilege escalation is captured in the same IAM event, `user.effective` +should of course be used as well. + +Here's the same "rename" example, where "alice" is renaming "bob" by escalating to "root": + +```JSON +{ + "user": { + "name": "alice", + "id": "1001", + "effective": { + "name": "root", + "id": "1" + }, + "target": { + "name": "bob", + "id": "1002" + }, + "changes": { + "name": "bob.barker" + } + } +} +``` + +### Pivoting via `related.user` + +Any event that has user(s) in it should always populate the array field `related.user` +with all usernames seen on the event. Note that this field is not a nesting of +all user fields, it's a flat array meant to contain user identifiers. + +Taking the example from `user.changes` again, and populating `related.user` as well, +the event now looks like: + +```JSON +{ + "user": { + "name": "alice", + "id": "1001", + "effective": { + "name": "root", + "id": "1" + }, + "target": { + "name": "bob", + "id": "1002" + }, + "changes": { + "name": "bob.barker" + } + }, + "related": { "user": ["alice", "root", "bob", "bob.barker"] } +} +``` + +## Source data + + + + + + + +## Scope of impact + + + +## Concerns + +As of ECS 1.5, the ECS documentation doesn't have a good place to explain at length +how to use the multiple nesting locations for `user`. This is already a problem +for the usage of `user` at the root vs its 5 reuse locations. The addition of these +3 new reuse locations adds to the situation. Adding a way to document field sets via +free form text is being worked on independently of this proposal. For now +the guidance on the meaning of each location where `user` can be used is in the +[Usage](#usage) section of this RFC. This guidance will be moved to the main ECS +documentation when the appropriate mechanism is available. + + + + + + + + + + + +## Real-world implementations + + + +## People + +The following are the people that consulted on the contents of this RFC. + +* @webmat | author +* TBD | sponsor +* @leehinman | subject matter expert +* @janniten | subject matter expert +* @willemdh | subject matter expert + + + + +## References + + + +* PR to add the fields described in this RFC: [ecs#869](https://github.com/elastic/ecs/pull/869) +* Past issues discussing this addition in ECS, starting with the most recent: + * https://github.com/elastic/ecs/issues/809 + * https://github.com/elastic/ecs/issues/678 + * https://github.com/elastic/ecs/issues/589 + * https://github.com/elastic/ecs/issues/234 + * https://github.com/elastic/ecs/issues/117 +* Discussions about this in Beats: + * https://github.com/elastic/beats/pull/10192 + * https://github.com/elastic/beats/issues/10111 + * https://github.com/elastic/beats/pull/9963 + +### RFC Pull Requests + + + +* Stage 2: https://github.com/elastic/ecs/pull/NNN + +Note: This RFC was initially proposed via a PR that targeted stage 2, +given the amount of discussion that has already has happened on this subject. From 02c75f38fe58a275778f4ac135efd81420155ac7 Mon Sep 17 00:00:00 2001 From: Mathieu Martin Date: Thu, 6 Aug 2020 16:36:39 -0400 Subject: [PATCH 02/24] Minor text adjustments --- rfcs/text/0000-multiple-users.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/rfcs/text/0000-multiple-users.md b/rfcs/text/0000-multiple-users.md index 8940390b91..5b9cbcba51 100644 --- a/rfcs/text/0000-multiple-users.md +++ b/rfcs/text/0000-multiple-users.md @@ -24,13 +24,13 @@ As of ECS 1.5, user fields can be present in an event in all of the following pl * `client.user.*` * `server.user.*` -The new fields proposed in this RFC are the following: +The new fields discussed in this RFC are the following: * `user.effective.*` * `user.target.*` * `user.changes.*` -Notice also that the user fields are now being nested as another name. +Notice that in these new additions, the user fields are now being nested as a different name. The purpose is to hint at their role, when used in these locations. It's also important to point out that the reuses of `user` inside other field sets @@ -59,7 +59,7 @@ as demonstrated below. ``` The `user` field set contains 9 leaf fields, 2 of which have a `.text` multi-field, -for a total of 11. These new nestings will therefore add a total of 33 fields. +for a total of 11 fields. These new nestings will therefore add a total of 33 fields. This can also be seen on the pre-existing PR [ecs#869](https://github.com/elastic/ecs/pull/869). ## Usage @@ -68,7 +68,7 @@ The examples below will mostly populate `user.name` inside the various `user` ne for readability. Unless otherwise noted, all `user` fields that can reasonably be populated in each location should be populated. -### `user` at the root of an event +### User fields at the Root of an Event Any event that only has one user should populate the user fields at the root of an event. Any time more than one user is present in an event, the `user` fields @@ -79,7 +79,7 @@ As many of the user fields as possible should be populated. In cases where a purpose-specific user field such as `url.username` is populated, `user.name` should also be populated with the same user name. -### Remote logons +### Remote Logons When users are crossing host boundaries, the users are captured at `source.user` and `destination.user`. @@ -240,7 +240,7 @@ Here's the same "rename" example, where "alice" is renaming "bob" by escalating } ``` -### Pivoting via `related.user` +### Pivoting via related.user Any event that has user(s) in it should always populate the array field `related.user` with all usernames seen on the event. Note that this field is not a nesting of From 00c963ffeb70cab78623039e8ad79eea85c32ba1 Mon Sep 17 00:00:00 2001 From: Mathieu Martin Date: Thu, 6 Aug 2020 16:41:00 -0400 Subject: [PATCH 03/24] Link RFC PR --- rfcs/text/0000-multiple-users.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/text/0000-multiple-users.md b/rfcs/text/0000-multiple-users.md index 5b9cbcba51..72d30acf74 100644 --- a/rfcs/text/0000-multiple-users.md +++ b/rfcs/text/0000-multiple-users.md @@ -380,7 +380,7 @@ e.g.: -* Stage 2: https://github.com/elastic/ecs/pull/NNN +* Stage 2: https://github.com/elastic/ecs/pull/914 Note: This RFC was initially proposed via a PR that targeted stage 2, given the amount of discussion that has already has happened on this subject. From 0e418f9dfb6e9341e8569a9f49a90756b947d083 Mon Sep 17 00:00:00 2001 From: Mathieu Martin Date: Mon, 10 Aug 2020 16:04:28 -0400 Subject: [PATCH 04/24] Typo in template --- rfcs/0000-rfc-template.md | 2 +- rfcs/text/0000-multiple-users.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/rfcs/0000-rfc-template.md b/rfcs/0000-rfc-template.md index 9b46087efa..d575443e27 100644 --- a/rfcs/0000-rfc-template.md +++ b/rfcs/0000-rfc-template.md @@ -83,7 +83,7 @@ The following are the people that consulted on the contents of this RFC. * TBD | author - Stage: **2 (proposal)** @@ -41,6 +41,7 @@ The current reusable locations `user` will be amended to include a few more entr as demonstrated below. ```YAML +# schemas/user.yml excerpt reusable: top_level: true expected: @@ -58,23 +59,26 @@ as demonstrated below. as: changes ``` -The `user` field set contains 9 leaf fields, 2 of which have a `.text` multi-field, -for a total of 11 fields. These new nestings will therefore add a total of 33 fields. -This can also be seen on the pre-existing PR [ecs#869](https://github.com/elastic/ecs/pull/869). +The `user` field set contains 6 leaf fields, 2 of which have a `.text` multi-field, +for a total of 8 fields. These new nestings will therefore add a total of 24 fields. +This can be seen in more detail on PR [ecs#869](https://github.com/elastic/ecs/pull/869). ## Usage -The examples below will mostly populate `user.name` inside the various `user` nestings, -for readability. Unless otherwise noted, all `user` fields that can reasonably be -populated in each location should be populated. +The examples below will only populate `user.name` and sometimes `user.id` inside +the various `user` nestings, for readability. +However in implementations, otherwise noted all `user` fields that can reasonably +be populated in each location should be populated. ### User fields at the Root of an Event -Any event that only has one user should populate the user fields at the root of an -event. Any time more than one user is present in an event, the `user` fields -at the root of an event should be interpreted as the user performing the action. +The user fields at the root of an event must be used to capture the user +performing the main action described by the event. This is especially important +when there's more than one user present on the event. `user.*` fields at the root +of the event represent the user performing the action. -As many of the user fields as possible should be populated. +In many cases, events that only mention one user are fine populating the user fields +at the root of the event. In cases where a purpose-specific user field such as `url.username` is populated, `user.name` should also be populated with the same user name. @@ -114,9 +118,8 @@ Here's an example where user "alice" logs on to another host as user "deus": } ``` - - -(also applies to `client.user` and `server.user`) +Whenever an event source populates the `client` and `server` fields in addition +to `source` and `destination`, the user fields should be copied accordingly as well. ### Privilege Changes @@ -209,7 +212,7 @@ Example where "root" renames user "bob" to "bob.barker": } ``` -You'll note in the example above that the user ID is not repeated under changes, +You'll note in the example above that the user ID is not repeated under `user.changes.*`, since the ID didn't change. ### Combining IAM and Privilege Escalation @@ -218,7 +221,8 @@ We've covered above how `user.target` and `user.changes` can be used at the same If privilege escalation is captured in the same IAM event, `user.effective` should of course be used as well. -Here's the same "rename" example, where "alice" is renaming "bob" by escalating to "root": +Here's the "rename" example from the IAM section above. In the following example, +we know "alice" is escalating privileges as "root", in order to modify user "bob": ```JSON { @@ -286,35 +290,58 @@ Stage 3: Add more real world example source documents so we have at least 2 tota ## Scope of impact - ## Concerns +### Deprecating host.user fields + +In past discussions and recent research, we have not identified a clear purpose +for the user fields nested at `host.user.*`. + +We are considering deprecating these fields with the intent to remove them completely. +Please let us know if you disagree with this, and share how you're using them. + +### Documenting the purpose of each usage of the user fields + As of ECS 1.5, the ECS documentation doesn't have a good place to explain at length how to use the multiple nesting locations for `user`. This is already a problem -for the usage of `user` at the root vs its 5 reuse locations. The addition of these +for the usage of `user` at the root vs its 5 reuse locations. The addition of 3 new reuse locations adds to the situation. Adding a way to document field sets via free form text is being worked on independently of this proposal. For now the guidance on the meaning of each location where `user` can be used is in the [Usage](#usage) section of this RFC. This guidance will be moved to the main ECS documentation when the appropriate mechanism is available. - - @@ -364,7 +391,7 @@ e.g.: -* PR to add the fields described in this RFC: [ecs#869](https://github.com/elastic/ecs/pull/869) +* PR to add the new fields described in this RFC: [ecs#869](https://github.com/elastic/ecs/pull/869) * Past issues discussing this addition in ECS, starting with the most recent: * https://github.com/elastic/ecs/issues/809 * https://github.com/elastic/ecs/issues/678 From 83537feca5c8c77e144090a3d8034c79d148c84c Mon Sep 17 00:00:00 2001 From: Mathieu Martin Date: Mon, 10 Aug 2020 16:37:38 -0400 Subject: [PATCH 06/24] Add a 3 --- rfcs/text/0000-multiple-users.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/text/0000-multiple-users.md b/rfcs/text/0000-multiple-users.md index d36c15fae4..a260ec5eef 100644 --- a/rfcs/text/0000-multiple-users.md +++ b/rfcs/text/0000-multiple-users.md @@ -60,7 +60,7 @@ as demonstrated below. ``` The `user` field set contains 6 leaf fields, 2 of which have a `.text` multi-field, -for a total of 8 fields. These new nestings will therefore add a total of 24 fields. +for a total of 8 fields. These 3 new nestings will therefore add a total of 24 fields. This can be seen in more detail on PR [ecs#869](https://github.com/elastic/ecs/pull/869). ## Usage From fe8e7598c2b0e428ff96dabf6a570bf557291d79 Mon Sep 17 00:00:00 2001 From: Mathieu Martin Date: Mon, 24 Aug 2020 14:16:23 -0400 Subject: [PATCH 07/24] Remove comma Co-authored-by: Eric Beahan --- rfcs/text/0000-multiple-users.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/text/0000-multiple-users.md b/rfcs/text/0000-multiple-users.md index a260ec5eef..c61f268b81 100644 --- a/rfcs/text/0000-multiple-users.md +++ b/rfcs/text/0000-multiple-users.md @@ -31,7 +31,7 @@ The new fields discussed in this RFC are the following: * `user.changes.*` Notice that in these new additions, the user fields are now being nested as a different name. -The purpose is to hint at their role, when used in these locations. +The purpose is to hint at their role when used in these locations. It's also important to point out that the reuses of `user` inside other field sets are not meant to inherit these new subsections inside the user field set. From 86eab562875535bc7ac9bac2f0c9a5389e39c588 Mon Sep 17 00:00:00 2001 From: Mathieu Martin Date: Mon, 24 Aug 2020 15:00:33 -0400 Subject: [PATCH 08/24] Change to 'As of ECS 1.6', since the release is imminent --- rfcs/text/0000-multiple-users.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/text/0000-multiple-users.md b/rfcs/text/0000-multiple-users.md index c61f268b81..84caa0af0f 100644 --- a/rfcs/text/0000-multiple-users.md +++ b/rfcs/text/0000-multiple-users.md @@ -333,7 +333,7 @@ Please let us know if you disagree with this, and share how you're using them. ### Documenting the purpose of each usage of the user fields -As of ECS 1.5, the ECS documentation doesn't have a good place to explain at length +As of ECS 1.6, the ECS documentation doesn't have a good place to explain at length how to use the multiple nesting locations for `user`. This is already a problem for the usage of `user` at the root vs its 5 reuse locations. The addition of 3 new reuse locations adds to the situation. Adding a way to document field sets via From d873044d744de431474364cf7401dfcd0341dfee Mon Sep 17 00:00:00 2001 From: Mathieu Martin Date: Mon, 24 Aug 2020 15:32:23 -0400 Subject: [PATCH 09/24] Link to PR for free form doc section per field set --- rfcs/text/0000-multiple-users.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/rfcs/text/0000-multiple-users.md b/rfcs/text/0000-multiple-users.md index 84caa0af0f..2aa83b3e49 100644 --- a/rfcs/text/0000-multiple-users.md +++ b/rfcs/text/0000-multiple-users.md @@ -337,8 +337,9 @@ As of ECS 1.6, the ECS documentation doesn't have a good place to explain at len how to use the multiple nesting locations for `user`. This is already a problem for the usage of `user` at the root vs its 5 reuse locations. The addition of 3 new reuse locations adds to the situation. Adding a way to document field sets via -free form text is being worked on independently of this proposal. For now -the guidance on the meaning of each location where `user` can be used is in the +free form text is being worked on independently of this proposal +([ecs#943](https://github.com/elastic/ecs/issues/943)). +For now the guidance on the meaning of each location where `user` can be used is in the [Usage](#usage) section of this RFC. This guidance will be moved to the main ECS documentation when the appropriate mechanism is available. @@ -402,6 +403,8 @@ e.g.: * https://github.com/elastic/beats/pull/10192 * https://github.com/elastic/beats/issues/10111 * https://github.com/elastic/beats/pull/9963 +* Adding a free form documentation section per field set, to allow documenting + them in a more holistic manner https://github.com/elastic/ecs/issues/943 ### RFC Pull Requests From 79432fb0c1557e0cae3742457ae4ddc5e722ac13 Mon Sep 17 00:00:00 2001 From: Mathieu Martin Date: Mon, 24 Aug 2020 15:45:04 -0400 Subject: [PATCH 10/24] Document explicitly when it's not possible to determine the audit user. --- rfcs/text/0000-multiple-users.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/rfcs/text/0000-multiple-users.md b/rfcs/text/0000-multiple-users.md index 2aa83b3e49..b9d496665a 100644 --- a/rfcs/text/0000-multiple-users.md +++ b/rfcs/text/0000-multiple-users.md @@ -158,6 +158,12 @@ Here's an example of user "alice" running a command as root via sudo: } ``` +When it's not possible (or it's prohibitive) to determine which user is requesting +different privilege levels, it's acceptable to capture the effective user at the +root of the event. Typically a privilege change event will already have happened, +for example: bob "su" as root; and subsequent events will show the root user +performing the actions. + ### Identity and Access Management Whenever a user is performing an action that affects another user -- typically From 5696aa24f1eb8620bcbe8ce3a8162ce3dbbb4bfe Mon Sep 17 00:00:00 2001 From: Mathieu Martin Date: Mon, 24 Aug 2020 15:51:13 -0400 Subject: [PATCH 11/24] Adjust another 'as of ECS 1.5' to 1.6 --- rfcs/text/0000-multiple-users.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/text/0000-multiple-users.md b/rfcs/text/0000-multiple-users.md index b9d496665a..eca1924f91 100644 --- a/rfcs/text/0000-multiple-users.md +++ b/rfcs/text/0000-multiple-users.md @@ -15,7 +15,7 @@ The purpose of this proposal is two-fold: ## Fields -As of ECS 1.5, user fields can be present in an event in all of the following places: +As of ECS 1.6, user fields can be present in an event in all of the following places: * `user.*` * `host.user.*` From adcd78af903d936ddc21530aa17fe327aa4b42ab Mon Sep 17 00:00:00 2001 From: Mathieu Martin Date: Mon, 24 Aug 2020 16:48:33 -0400 Subject: [PATCH 12/24] Remove a few more template comments --- rfcs/text/0000-multiple-users.md | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-) diff --git a/rfcs/text/0000-multiple-users.md b/rfcs/text/0000-multiple-users.md index eca1924f91..fa079a5347 100644 --- a/rfcs/text/0000-multiple-users.md +++ b/rfcs/text/0000-multiple-users.md @@ -323,7 +323,8 @@ and adjusted accordingly. @@ -349,14 +350,6 @@ For now the guidance on the meaning of each location where `user` can be used is [Usage](#usage) section of this RFC. This guidance will be moved to the main ECS documentation when the appropriate mechanism is available. - - - - @@ -396,8 +389,6 @@ e.g.: ## References - - * PR to add the new fields described in this RFC: [ecs#869](https://github.com/elastic/ecs/pull/869) * Past issues discussing this addition in ECS, starting with the most recent: * https://github.com/elastic/ecs/issues/809 @@ -414,8 +405,6 @@ e.g.: ### RFC Pull Requests - - * Stage 2: https://github.com/elastic/ecs/pull/914 Note: This RFC was initially proposed via a PR that targeted stage 2, From f9e0a52fd22ec0a116cf300abfbbc29bae2705b0 Mon Sep 17 00:00:00 2001 From: Mathieu Martin Date: Mon, 24 Aug 2020 16:49:52 -0400 Subject: [PATCH 13/24] Make the 'resolution' aspect of the concerns stand out more with an H4 heading --- rfcs/text/0000-multiple-users.md | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/rfcs/text/0000-multiple-users.md b/rfcs/text/0000-multiple-users.md index fa079a5347..3b42da3436 100644 --- a/rfcs/text/0000-multiple-users.md +++ b/rfcs/text/0000-multiple-users.md @@ -338,14 +338,22 @@ for the user fields nested at `host.user.*`. We are considering deprecating these fields with the intent to remove them completely. Please let us know if you disagree with this, and share how you're using them. +#### Resolution + +No resolution yet. + ### Documenting the purpose of each usage of the user fields As of ECS 1.6, the ECS documentation doesn't have a good place to explain at length -how to use the multiple nesting locations for `user`. This is already a problem -for the usage of `user` at the root vs its 5 reuse locations. The addition of -3 new reuse locations adds to the situation. Adding a way to document field sets via -free form text is being worked on independently of this proposal -([ecs#943](https://github.com/elastic/ecs/issues/943)). +how to properly use the multiple nesting locations for `user`. +This is already a problem for the usage of `user` at the root vs its 5 reuse locations. +The addition of 3 new reuse locations adds to this situation. + +#### Resolution + +Adding a way to document field sets via free form text is being worked on independently +of this proposal ([ecs#943](https://github.com/elastic/ecs/issues/943)). + For now the guidance on the meaning of each location where `user` can be used is in the [Usage](#usage) section of this RFC. This guidance will be moved to the main ECS documentation when the appropriate mechanism is available. From 938708370bd6db1331a22500381cc9840fc342e7 Mon Sep 17 00:00:00 2001 From: Mathieu Martin Date: Tue, 29 Sep 2020 14:09:06 -0400 Subject: [PATCH 14/24] Add the rfc field definition that makes this come to life. --- rfcs/text/0000/user.yml | 10 ++++++++++ schemas/user.yml | 9 --------- 2 files changed, 10 insertions(+), 9 deletions(-) create mode 100644 rfcs/text/0000/user.yml diff --git a/rfcs/text/0000/user.yml b/rfcs/text/0000/user.yml new file mode 100644 index 0000000000..23103f5b4a --- /dev/null +++ b/rfcs/text/0000/user.yml @@ -0,0 +1,10 @@ +--- +- name: user + reusable: + expected: + - at: user + as: target + - at: user + as: effective + - at: user + as: changes diff --git a/schemas/user.yml b/schemas/user.yml index 255e132f69..39a68ccd97 100644 --- a/schemas/user.yml +++ b/schemas/user.yml @@ -19,15 +19,6 @@ - server - source - # TODO Temporarily commented out to simplify initial rewrite review - - # - at: user - # as: target - # - at: user - # as: effective - # - at: user - # as: changes - type: group fields: From 2d364a71613adf6b0e291bf6e87ec8746055141d Mon Sep 17 00:00:00 2001 From: Mathieu Martin Date: Tue, 29 Sep 2020 17:14:21 -0400 Subject: [PATCH 15/24] First stab at source data for Linux events --- rfcs/text/0000-multiple-users.md | 183 ++++++++++++++++++++++++++++++- 1 file changed, 181 insertions(+), 2 deletions(-) diff --git a/rfcs/text/0000-multiple-users.md b/rfcs/text/0000-multiple-users.md index 3b42da3436..3246529a73 100644 --- a/rfcs/text/0000-multiple-users.md +++ b/rfcs/text/0000-multiple-users.md @@ -77,8 +77,8 @@ performing the main action described by the event. This is especially important when there's more than one user present on the event. `user.*` fields at the root of the event represent the user performing the action. -In many cases, events that only mention one user are fine populating the user fields -at the root of the event. +In many cases, events that only mention one user should populate the user fields +at the root of the event, even if the user is not the one performing the action. In cases where a purpose-specific user field such as `url.username` is populated, `user.name` should also be populated with the same user name. @@ -282,6 +282,185 @@ the event now looks like: ## Source data +Here are some concrete examples of events with multiple user and user roles. + +### Linux user creation + +Here's a typical set of log about a user creation on Linux. + +``` +Sep 29 19:55:09 localhost sudo: vagrant : TTY=pts/0 ; PWD=/home/vagrant ; USER=root ; COMMAND=/sbin/useradd test-user -p test-password +Sep 29 19:55:09 localhost sudo: pam_unix(sudo:session): session opened for user root by vagrant(uid=0) +Sep 29 19:55:09 localhost useradd[2097]: new group: name=test-user, GID=1001 +Sep 29 19:55:09 localhost useradd[2097]: new user: name=test-user, UID=1001, GID=1001, home=/home/test-user, shell=/bin/bash +Sep 29 19:55:09 localhost sudo: pam_unix(sudo:session): session closed for user root +``` + +#### Logical events + +A solution that coalesces log events to produce higher level logical events could +capture them all in the following way. + +Group creation: + +```JSON +{ + "event": { + "category": ["iam"], + "type": ["group", "creation"] + }, + "group": { + "name": "test-user", + "id": "1001" + }, + "user": { + "name": "vagrant", + "id": "1000", + "effective": { + "name": "root", + "id": "0" + }, + }, + "related": { "user": ["vagrant", "root"] } +} +``` + +User creation: + +```JSON +{ + "event": { + "category": ["iam"], + "type": ["user", "creation"] + }, + "user": { + "name": "vagrant", + "id": "1000", + "effective": { + "name": "root", + "id": "0" + }, + "target": { + "name": "test-user", + "id": "1001", + "group": { + "name": "test-user", + "id": "1001" + } + } + }, + "related": { "user": ["vagrant", "root", "test-user"] } +} +``` + +#### Raw events + +A solution that produces one event per log without coalescing would instead only +represent users with the information available in the given log event. + +event 1: + +```JSON +{ + "event": { + "category": ["process"], + "event": "creation" + }, + "user": { + "name": "vagrant", + "effective": { + "name": "root" + } + }, + "process": { + "name": "sudo", + "command_line": "/sbin/useradd test-user -p test-password" + } +} +``` + +event 2: + +```JSON +{ + "event": { + "category": ["session"], + "event": ["creation"], + "outcome": "success" + }, + "user": { + "name": "vagrant", + "effective": { + "name": "root" + } + }, + "process": { + "name": "sudo" + } +} +``` + +event 3: + +```JSON +{ + "event": { + "category": ["iam"], + "type": ["group", "creation"], + "outcome": "success" + }, + "group": { + "name": "test-user", + "id": "1001" + }, + "process": { + "name": "useradd", + "pid": 2097 + } +} +``` + +event 4: + +```JSON +{ + "event": { + "category": ["iam"], + "type": ["user", "creation"] + }, + "user": { + "name": "test-user", + "id": "1001" + }, + "process": { + "name": "useradd", + "pid": 2097 + } +} +``` + +Notice: in the event above, since the log mentions only the user being created, +we capture the user at the root of the event. We do this despite the fact that +they are not the one performing the action. + +event 5: + +```JSON +{ + "event": { + "category": ["session"], + "event": ["end"] + }, + "user": { + "name": "root" + }, + "process": { + "name": "sudo" + } +} +``` + + From 726edca8816e670faa33178c475145f008fa24e9 Mon Sep 17 00:00:00 2001 From: Mathieu Martin Date: Wed, 30 Sep 2020 10:53:31 -0400 Subject: [PATCH 16/24] Minor wording tweaks --- rfcs/text/0000-multiple-users.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/rfcs/text/0000-multiple-users.md b/rfcs/text/0000-multiple-users.md index 3246529a73..56b2784030 100644 --- a/rfcs/text/0000-multiple-users.md +++ b/rfcs/text/0000-multiple-users.md @@ -282,9 +282,9 @@ the event now looks like: ## Source data -Here are some concrete examples of events with multiple user and user roles. +Here are some concrete examples of events with multiple users and user roles. -### Linux user creation +### Linux IAM and privilege escalation Here's a typical set of log about a user creation on Linux. @@ -379,7 +379,7 @@ event 1: } ``` -event 2: +event 2 (privilege escalation): ```JSON { @@ -400,7 +400,7 @@ event 2: } ``` -event 3: +event 3 (IAM): ```JSON { @@ -420,7 +420,7 @@ event 3: } ``` -event 4: +event 4 (IAM): ```JSON { From 678682878dffb251fd9a7284ccc681418202c98c Mon Sep 17 00:00:00 2001 From: Mathieu Martin Date: Wed, 30 Sep 2020 13:58:42 -0400 Subject: [PATCH 17/24] Add 2 Windows examples --- rfcs/text/0000-multiple-users.md | 105 ++++++++++++++++++++++++++++++- 1 file changed, 102 insertions(+), 3 deletions(-) diff --git a/rfcs/text/0000-multiple-users.md b/rfcs/text/0000-multiple-users.md index 56b2784030..cad1683790 100644 --- a/rfcs/text/0000-multiple-users.md +++ b/rfcs/text/0000-multiple-users.md @@ -283,6 +283,9 @@ the event now looks like: ## Source data Here are some concrete examples of events with multiple users and user roles. +Note that the design of these fields is meant to allow all of their use at the +same time, *when needed*. However if events don't contain all user roles because +they're spread out across events, only the fields relevant to each event should be used. ### Linux IAM and privilege escalation @@ -460,10 +463,106 @@ event 5: } ``` +### Windows privilege escalation + +A successful local Windows Admin logon where user "testuser" escalates to Administrator: + +```XML + + + 4624 + ... + + + 5-1-5-21-202424912787-2692429404-2351956786-1000 + testuser + TEST + 0xb976c + S-1-5-21-2024912787-2692429404-2351956786-500 + Administrator + TEST + 0x11b621 + ... + +``` + +Would translate to + +```JSON +{ + "event": { + "code": "4624", + "provider": "Microsoft-Windows-Security-Auditing", + "category": ["authentication"], + "event": ["start"], + "outcome": "success" + }, + "user": { + "name": "testuser", + "domain": "TEST", + "id": "S-1-5-21-202424912787-2692429404-2351956786-1000", + "effective": { + "name": "Administrator", + "domain": "TEST", + "id": "S-1-5-21-2024912787-2692429404-2351956786-500" + } + }, + "related": { "user": ["testuser", "Administrator"] } +} +``` + +### Windows IAM + +Modifying an existing user account, where the administrator renames user John to John2: + +```XML + + + 4781 + ... + + + S-1-5-21-2024912787-2692429404-2351956786-500 + Administrator + TEST + 0x11b621 + S-1-5-21-2024912787-2692429404-2351956786-1000 + John + John2 + TEST + ... + +``` + +Would translate to + +```JSON +{ + "event": { + "code": "4781", + "provider": "Microsoft-Windows-Security-Auditing", + "category": ["iam"], + "event": ["user", "change"], + "outcome": "success" + }, + "user": { + "name": "Administrator", + "domain": "TEST", + "id": "S-1-5-21-2024912787-2692429404-2351956786-500", + "target": { + "name": "John", + "id": "S-1-5-21-2024912787-2692429404-2351956786-1000", + "domain": "TEST", + }, + "changes": { + "name": "John2" + } + }, + "related": { "user": ["John", "John2", "Administrator"] } +} +``` + - - Stage: **2 (proposal)** diff --git a/rfcs/text/0000/user.yml b/rfcs/text/0007/user.yml similarity index 100% rename from rfcs/text/0000/user.yml rename to rfcs/text/0007/user.yml From 29b7546c42a640f0ade3ad3a97eaaaba5fa95107 Mon Sep 17 00:00:00 2001 From: Mathieu Martin Date: Wed, 30 Sep 2020 15:45:18 -0400 Subject: [PATCH 22/24] Replace all event.event by the correct field, event.type --- rfcs/text/0007-multiple-users.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/rfcs/text/0007-multiple-users.md b/rfcs/text/0007-multiple-users.md index 83a0c80404..0548b96a71 100644 --- a/rfcs/text/0007-multiple-users.md +++ b/rfcs/text/0007-multiple-users.md @@ -370,7 +370,7 @@ event 1: "event": { "kind": "event", "category": ["process"], - "event": "creation" + "type": "creation" }, "user": { "name": "vagrant", @@ -392,7 +392,7 @@ event 2 (privilege escalation): "event": { "kind": "event", "category": ["session"], - "event": ["creation"], + "type": ["creation"], "outcome": "success" }, "user": { @@ -459,7 +459,7 @@ event 5: "event": { "kind": "event", "category": ["session"], - "event": ["end"] + "type": ["end"] }, "user": { "name": "root" @@ -502,7 +502,7 @@ Would translate to "provider": "Microsoft-Windows-Security-Auditing", "kind": "event", "category": ["authentication"], - "event": ["start"], + "type": ["start"], "outcome": "success" }, "user": { @@ -551,7 +551,7 @@ Would translate to "provider": "Microsoft-Windows-Security-Auditing", "kind": "event", "category": ["iam"], - "event": ["user", "change"], + "type": ["user", "change"], "outcome": "success" }, "user": { @@ -615,7 +615,7 @@ And would translate to: "action": "AssumeRole", "kind": "event", "category": ["authentication"], - "event": ["start"], + "type": ["start"], "outcome": "success" }, "user": { From 1353161c29822a25b770d35c35d45856e2bdb29a Mon Sep 17 00:00:00 2001 From: Mathieu Martin Date: Wed, 30 Sep 2020 15:45:52 -0400 Subject: [PATCH 23/24] Missing pluralization --- rfcs/text/0007-multiple-users.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/text/0007-multiple-users.md b/rfcs/text/0007-multiple-users.md index 0548b96a71..d07e1d0d84 100644 --- a/rfcs/text/0007-multiple-users.md +++ b/rfcs/text/0007-multiple-users.md @@ -289,7 +289,7 @@ they're spread out across events, only the fields relevant to each event should ### Linux IAM and privilege escalation -Here's a typical set of log about a user creation on Linux. +Here's a typical set of logs about a user creation on Linux. ``` Sep 29 19:55:09 localhost sudo: vagrant : TTY=pts/0 ; PWD=/home/vagrant ; USER=root ; COMMAND=/sbin/useradd test-user -p test-password From 1892fa6d73e7d1de9f50722097e852b50e20015f Mon Sep 17 00:00:00 2001 From: Mathieu Martin Date: Fri, 2 Oct 2020 11:17:14 -0400 Subject: [PATCH 24/24] This is it. Merge date for this RFC is today. --- rfcs/text/0007-multiple-users.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/text/0007-multiple-users.md b/rfcs/text/0007-multiple-users.md index d07e1d0d84..344255f5f7 100644 --- a/rfcs/text/0007-multiple-users.md +++ b/rfcs/text/0007-multiple-users.md @@ -2,7 +2,7 @@ - Stage: **2 (proposal)** -- Date: **TBD** +- Date: **2020-09-02** Many log events refer to more than one user at the same time. Examples of this are remote logons as someone else, user management and privilege escalation.