[Discuss] Increasing availability of profile_uid outside of interactive sessions

[marshallmain]

Use Case

Alerts generated by the Security Solution's detection rules have a few fields that can be modified by users - status, assignee, and tags. Until now, we've had no record of who modified these fields and when. We would like to start keeping these records and profile_uid appears to be the best single identifier to track which user made some modification. We envision 2 types of records in this use case: (1) a record of the last update, stored in the alert document, and (2) a record of all updates, where each update to the alert gets a separate document in some log index. Since we will likely want to store a record of the last update for each mutable field in an alert (status, assignee, tags, potentially more in the future) it would be ideal to minimize the number of fields we need to store for each "last update" record in the alert document.

In this plan, the "record of all updates" in a separate log index could contain more detailed information about the user's authentication, like auth type, realm, etc. This would provide the authoritative audit info and allow us to distinguish between different auth methods for the same profile_uid. Even with the enhancements proposed in this issue, profile_uid may still be unavailable in some cases - the separate log index with detailed auth info would provide the audit trail for these cases.

Even if profile_uid is not available in all cases, keeping a record of the last update in the alert document itself is useful to enable simple queries involving alerts and these update records, e.g. "show me alerts updated by user X", "show me alerts updated in the last 24 hours", etc. We would like to use profile_uid as the unique user identifier in the alert document for consistency, since it will also be used for tracking the user assigned to an alert, and it has good integration with other aspects of Kibana, like avatars. I expect we'll be able to re-use UI elements that can translate from profile_uid to a readable username or add a query filter by profile_uid without requiring a user to directly interact with the ID value. These use cases are improved if profile_uid is available more often, even if it is not always available.

Proposal

profile_uid is currently only available within interactive user sessions. Security Solution users are known to use the HTTP API and automation scripts to manage alerts. We would like to explore ways to make profile_uid available even outside of interactive sessions so that we can still capture the profile_uid for API users. One approach might be to allow APIs to specify if a profile should be activated for that API even outside of interactive sessions. We could then pay the cost of profile activation only for specific APIs that we know will rely on profile_uid, like the API that changes the status of an alert.

In general, auditing APIs that write data would be our primary use case for obtaining a profile_uid outside of existing interactive sessions. APIs that only read data would generally be lower priority for obtaining profile_uid outside of interactive sessions.

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)