[Cases] Attach files to cases backend design

[jonathan-buttner]

This issue will explain the backend changes needed to support attaching files to cases.

Background

Storing metadata

To facilitate finding files attached to case and RBAC operations, we will store the case owner and id within the metadata of the file. When using the files services find api we can pass in the case id to retrieve all files associated with that case. Storing the owner within the file metadata is necessary to perform RBAC checks.

We will also store the file saved object id in the externalReferenceAttachmentTypeId field of the attachment framework comment. This gives us flexibility in the future if we need to export the file information when exporting a case or retrieving specific case comments that contain a file saved object id.

RBAC needs

When a user is attaching a file to a case, we'd like to isolate the files attached to an observability case and a security or stack case. The files service is designed such that a file kind is registered during a plugin's setup. This means that if cases were to only register a single file kind id (e.g. cases), then all three instances of cases would be able to view the files uploaded (not necessarily attached) by any user with permissions to attach files to cases. When a user wants to attach a file to a case, first they would select a file on their computer and upload it to kibana. Once the file is uploaded it will be shown in the file picker component provided by the files service. These files will be visible to all three solutions.

Since we'd like security users to only see files uploaded to security, we have two options.

Register multiple file kinds

This option requires the cases backend to hard code the knowledge of the three owners. When the cases plugin performs its setup, cases will register a unique file kind of each owner type. On the UI when the user interacts with the files page to request all the files or view them, cases will deduce which plugin we are operating within and use that particular file kind for finding/opening the files.

The files service leverages HTTP tags to enforce RBAC over the API routes. Security solution, Observability, and Stack cases will all have separate HTTP tags so that they won't be able to view each others uploaded files. These tags will be specified separately within each plugin's cases feature registration process.

The downside of this implementation is that we have to hard code the three owners in the backend and this is something we've tried to avoid. In the future if we still need to use the owners on the backend we can implementation a plugin registration model where each plugin would register their own file kind with cases when they register their owner.

Callbacks

Another option proposed by the files service team was to allow registration of callbacks for each of the http operations when a file kind is registered. This would allow cases to register a series of callbacks that would be called before any operation is completed. When a file is retrieved, the callback will be execute and cases can check to see if the user performing the request is authorized for the operation for a particular owner. Cases will need to receive the owner information during the callback.

MVP Backend changes

File kind registration

The file services team is discussing whether the Callback RBAC solution is feasible. So for the time being we will move forward with registering three separate file kinds, one for each plugin that uses cases.

• We will support the following file types: images, plain text, logs, txt, json, zip, csv, pdf
• File size limit: 100 mb

The size limit and support file types will be made configurable through the kibana yml file for on-prem and in cloud.

Attachment framework

The files attachment will leverage the cases attachment framework. Using the framework avoids needing to implement saved object id migration logic into the references array, having to create a new file attachment schema, and makes implementing the UI easier. The file attachment within cases will store the following information. These fields need to exist within cases to preserve the user activity information when a case is exported (even though the file contents will not be exported).

Non-searchable fields

• File name
• File saved object ID
• Date when the file was created (This is the date it was created within elasticsearch)

Searchable fields (Needed for filtering the file view by file type/mime type)

• Extension
• Mime type

File types route

A new internal route will be created to facilitate retrieving all unique file types and mime types. The file service does not currently support this behavior so cases will duplicate the file type and mime type information such that an aggregation can be used to determine this information. These values will be used to allow filtering of the files by type/mime type.

Request

GET /internal/cases/<case id>/files/_extensions

Response

{
  types: Array<{
    value: string
    total: number - Do we want the totals for each type?
  }>
  total: number - total number of unique extensions
}

Not included in the MVP

• Publicly sharing files
• Exporting the file contents when a case is exported
• Searching for files by file size

[elasticmachine]

Pinging @elastic/response-ops (Team:ResponseOps)

[elasticmachine]

Pinging @elastic/response-ops-cases (Feature:Cases)

[jonathan-buttner]

I talked with Christos and we decided to keep as little information in the attachment framework for files as possible. So the size will not be included within cases. If we need the functionality to search for file sizes we'll add that within the file service.

[adcoelho]

There is something I realized while working on the front end.

It would be nice if the Find route API could also include a list of all mimeTypes or extensions in the response.

This will be useful for the filter-by-type functionality.

Since the API returns a paginated response I cannot just iterate over all existing attachments to get that info in the front end.

[adcoelho]

Another question regarding the API.

I noticed in your specification that the response already has camel case.

{
  files: Array<{
    id: string
    name: string
    extension: string
    mimeType: string
    created: string
  }>
  page: number
  perPage: number
  total: number
}

but mimeType and perPage will probably be snake case, right?

It is just that usually our API responses return snake and then we always call convertCaseToCamelCase.

[jonathan-buttner]

I was trying to model it after the response of the files service which leaves the mimeType as camel case. My default is to just leave everything as camel case but I'm happy to do whatever. I think there was a push to make things consistent with how Elasticsearch does it which is snake case but we haven't been super consistent with that.

[jonathan-buttner]

@adcoelho would you rather have the response of the unique file/mime types structured like this:

{
  types: Array<{
    value: string
    total: number
  }>
  mimeTypes: Array<{
    value: string
    total: number
  }>
}

Or as a single array?

{
  fileTypes: Array<{
    type: string - Either "extension" or "mimeType"
    value: string
    total: number
  }>
}

And do you think we'd want to show the total for each type in the dropdown filtering component?