[RAC] Add external/related/associated Case IDs and 3rd party incidence link

[hbharding]

Part of #132772

Summary

We want to introduce human readable Case IDs so that users can easily reference a specific case. We will use an "incremental" pattern similar to tools like Github where the first case created is labeled #1, the 2nd is #2, and so on. When users create a new case, an ID will automatically be assigned and cannot be changed. Cases originating from different apps like Observability and Security would share the same sequence. For example, if cases #1 and #2 were created in Observability, and then a user creates a third case in Security, that case ID would be #3.

There are several places in our UI where it makes sense to show this information, documented below.

AC

• Add "Case ID" as a field for Case data structure
• Automatically assign a Case ID when a new case is created
• Add human readable case IDs in the places documented below
• These changes should also be reflected in the security solution as well.

Alerts list view

We have a separate issue for this since we want to append the Case ID to the alert index. The "Cases" column shows the Case ID with a link to view the case. If multiple Cases exist for a single alert, we can use a comma separated list of Case IDs. https://github.com/elastic/observability-design/issues/102

Alerts detail flyout

We should provide a link to the Case ID in the alert details flyout. If more than one Case exists, use a comma separated list. If no cases exist, the value can just use a - to denote that a case does not exist. These links should use the same hover tooltip from the alerts table (more about this below).

Modal: Add to existing case

We can show the Case ID in the table users see in order to select a case to add an alert to.

Cases list view

We can append the case ID to the Case name on the Cases table. Users should be able to search for a specific case by referencing it's number. i.e. 254 or #254.

Case detail page

We can append the Case ID to the Case detail page's title text. This way the relationship is clear when a user clicks on a Case ID and land on this page. If the user clicks the "edit" button icon to change the Case's name, the Case ID portion should disappear while they are editing. The Case ID should not change if a user changes the case name.

Tooltips

We should provide tooltips when users hover on the case ID to provide additional context about the cases. This way, we can set expectations for what the user will see if they click the case ID link. These tooltips should appear everywhere that we show the Case ID as a link. For now, this includes the Alerts list page and the Alert detail flyout.

[elasticmachine]

Pinging @elastic/security-threat-hunting-cases (Team:Threat Hunting:Cases)

[cnasikas]

Hi @hbharding,

I would like to make it apparent that providing that feature is not trivial. The problems we will face are a) concurrency and b) migrations (cases uses the SavedObjects API).

Concurrency

Imagine there are two Kibana nodes. The nodes do not share any memory or communicate. Two users simultaneously create a case and each request is routed to one of the nodes accordingly. Each Kibana node will query ES to get the latest Case to deduct the latest human-readable ID (let's say is a field saved in ES for each case). Both of the nodes will get the same ID and both of them will increase the ID by one. That will result in two different cases with the same ID. To resolve this problem we need to think of a way of creating distributed human-readable IDs.

Migrations

All cases created before this feature need to be migrated so each case is assigned an ID. The problem is it is not possible to keep state during migrations and the order of documents is random. The state is needed to keep a counter so each case is assigned the next ID. The order is needed to assign the IDs chronologically (created_at).

[oatkiller]

Thanks for the excellent write up. I'd like to add a few points along the lines of what @cnasikas has brought up:

How unique should these IDs be?

Assume we start with a brand new deployment. If the first ever case is created in space 'A' and the second case is created in space 'B', would each have the id: "#1", or would the second case have the id: "#2"?

Today SOs (and therefore cases) aren't shareable between spaces, but the functionality is planned. If we want to eventually have the option to share cases between spaces, we might want to make them unique across the entire kibana instance.

However there is a downside to this approach. Users with read/write privileges in a single space could infer how many cases are being created in other spaces. They could do this by creating a case and noting its ID, then waiting some time and creating another case. By subtracting the first id from the second they could infer the number of cases created in all other spaces during that time.

If that raises no security concerns, then I propose that we make the IDs unique to the kibana instance, not the space. Otherwise we'll have to deduplicate IDs between spaces if we decide to make cases shareable in the future.

Sub question

I'd like to ask someone on the ES team about other gotchas we should be aware of.

Re: Considering an Optimistic Concurrency Control approach

Assuming that the case IDs are unique within a space, we could have a single document per space that holds the last used space ID. The document would be a special, singleton, saved object that keeps a the space ID that was last used to create a case. It could store this in a keyword field. When a new case is created, we could use optimistic concurrency control (as described here) to get a new ID to use for the new case.

This approach has a serious downside though. As cases can be created programmatically, it's possible that there could be high contention on this check. If the system tries to create n cases concurrently, then n-1 of those concurrency checks will fail. If several long running automated processes try to do this in parallel, the system won't handle it well. For example, if many actions fired, creating many alerts all at once, that could cause contention on the check.

On situations where getting an incrementing integer is impractical

There are several known cases where getting an incremented integer id would be impractical:

• if many cases would be created concurrently, there would be too much contention on the concurrency check.
  • this could happen if many cases were created, concurrently, by actions, or api requests.
  • it could theoretically happen if there were many users collaborating on the system at the same time
• (TODO, explain) when a case is imported

We could mitigate this limitation by adding this requirement:

Cases that are created programmatically (without a 1:1 user interaction) will have an ID that includes a UUID in place of the usual incremented, easy to read, number. for example: '321B963B-26A9-4FD3-9932-A6E009FACC74'`.

We can use the same approach in any situation where getting a unique, incremented, ID would be impractical.

Migrations

In the above section on concurrency I described an approach where we'd use a hash in place of the ID in certain cases. We could use that approach here.

Import / Export

What would the human readable id of a case by after import? We could keep the human readable id from the imported data when importing the case. That could introduce duplicates. Otherwise we could replace the human readable id with a less human readable one. For example if the id was #1, we could change it to imported as #1 (fa302efaf) where the last bit of random numbers and letters would be unique. That would have the advantage of being unique, but still human readable-ish.

[oatkiller]

What about copying off of the github's commit hash link idea instead? TODO, explore

[hbharding]

Thanks @cnasikas and @oatkiller. Great questions and ideas! Clearly we have some more thinking to do on how to handle all these situations. I want to briefly summarize the points I see so far:

• Concurrency
• Migrations
• How does this work across Spaces? and concerns about privileges / security
• Bulk creation of cases
• Importing / exporting cases

I'll work with @vinaychandrasekhar (and maybe some help from @jasonrhodes) to add more detail to the AC. It might be useful for us all to have a brainstorming meeting. Stay tuned. cc @katrin-freihofner

[jasonrhodes]

I think this could easily eat up many many dev cycles on trying to figure out the non-functional requirements of making these IDs work when we already have UUIDs that do what we need, so I want to push back a bit on the importance here before we commit to that much eng work. I totally understand why we want these, but I like @oatkiller's suggestion of using GitHub's model of a shortened version of the UUID as a reasonably safe-unique value that's more readable than the full UUID (though not as nice as "#23", obviously).

[jasonrhodes]

This is the GitHub method that @oatkiller and I are referring to, for reference (I think they are just using the first 6-7 chars):

GitHub Docs

[jonathan-buttner]

What would the human readable id of a case by after import? We could keep the human readable id from the imported data when importing the case. That could introduce duplicates.

@oatkiller just wanted to note that the current implementation of the saved object management's page import (which cases leverages) doesn't allow custom functionality to be executed. So currently, we can't perform validation or manipulate the data being imported.

[elasticmachine]

Pinging @elastic/response-ops-cases (Feature:Cases)

[elasticmachine]

Pinging @elastic/response-ops (Team:ResponseOps)

[cnasikas]

Hi everyone! In #146864 we added the cases column to the alerts table.