[Cases] Implement files backend changes #151933
Labels
Feature:Cases
Cases feature
Team:ResponseOps
Label for the ResponseOps team (formerly the Cases and Alerting teams)
Comments
jonathan-buttner
added
Team:ResponseOps
|
Pinging @elastic/response-ops (Team:ResponseOps) |
|
Pinging @elastic/response-ops-cases (Feature:Cases) |
This was referenced Feb 22, 2023
jonathan-buttner
added a commit
that referenced
this issue
Mar 2, 2023
This PR registers three file kinds for cases. One for each instance of cases (stack, observability, and security). Each solution needs separate http tags for the routes that are generated by the file service to implement RBAC. I refactored the logic to remove some duplication across the three plugins since we're essentially registering the same http tags with slightly different names. This PR shouldn't affect any of the current functionality. Notable changes: - I split up the constants.ts file, really the only change is adding the file kinds logic to generate the http tags the rest is copy/paste - Refactored the logic to generate the `api` http tags for each plugin - Registered the three file kinds Issues: #151780 #151933 --------- Co-authored-by: kibanamachine <[email protected]>
jonathan-buttner
added a commit
that referenced
this issue
Mar 6, 2023
This PR registers the file attachment type `.files` within the cases attachment framework. Notable changes: - Attachment framework accepts an optional validation function for use when a request is received for that attachment type, this allows validating that the file metadata is correct - Refactored the logic for enforcing that only 1000 alerts can be attached to a case and 100 files Issue: #151933 <details><summary>cUrl request to create file attachments</summary> ``` curl --location --request POST 'http://elastic:changeme@localhost:5601/internal/cases/<case id>/attachments/_bulk_create' \ --header 'kbn-xsrf: hello' \ --header 'Content-Type: application/json' \ --data-raw '[ { "type": "externalReference", "externalReferenceStorage": { "type": "savedObject", "soType": "file" }, "externalReferenceId": "my-id", "externalReferenceAttachmentTypeId": ".files", "externalReferenceMetadata": { "file": { "name": "test_file", "extension": "png", "mimeType": "image/png", "createdAt": "2023-02-27T20:26:54.345Z" } }, "owner": "cases" }, { "type": "externalReference", "externalReferenceStorage": { "type": "savedObject", "soType": "file" }, "externalReferenceId": "my-id", "externalReferenceAttachmentTypeId": ".files", "externalReferenceMetadata": { "file": { "name": "test_file", "extension": "png", "mimeType": "image/png", "createdAt": "2023-02-27T20:26:54.345Z" } }, "owner": "cases" }, { "type": "externalReference", "externalReferenceStorage": { "type": "savedObject", "soType": "file" }, "externalReferenceId": "my-id", "externalReferenceAttachmentTypeId": ".files", "externalReferenceMetadata": { "file": { "name": "test_file", "extension": "jpeg", "mimeType": "image/jpeg", "createdAt": "2023-02-27T20:26:54.345Z" } }, "owner": "cases" } ] ' ``` </details>
bmorelli25
pushed a commit
to bmorelli25/kibana
that referenced
this issue
Mar 10, 2023
…152031) This PR registers three file kinds for cases. One for each instance of cases (stack, observability, and security). Each solution needs separate http tags for the routes that are generated by the file service to implement RBAC. I refactored the logic to remove some duplication across the three plugins since we're essentially registering the same http tags with slightly different names. This PR shouldn't affect any of the current functionality. Notable changes: - I split up the constants.ts file, really the only change is adding the file kinds logic to generate the http tags the rest is copy/paste - Refactored the logic to generate the `api` http tags for each plugin - Registered the three file kinds Issues: elastic#151780 elastic#151933 --------- Co-authored-by: kibanamachine <[email protected]>
bmorelli25
pushed a commit
to bmorelli25/kibana
that referenced
this issue
Mar 10, 2023
This PR registers the file attachment type `.files` within the cases attachment framework. Notable changes: - Attachment framework accepts an optional validation function for use when a request is received for that attachment type, this allows validating that the file metadata is correct - Refactored the logic for enforcing that only 1000 alerts can be attached to a case and 100 files Issue: elastic#151933 <details><summary>cUrl request to create file attachments</summary> ``` curl --location --request POST 'http://elastic:changeme@localhost:5601/internal/cases/<case id>/attachments/_bulk_create' \ --header 'kbn-xsrf: hello' \ --header 'Content-Type: application/json' \ --data-raw '[ { "type": "externalReference", "externalReferenceStorage": { "type": "savedObject", "soType": "file" }, "externalReferenceId": "my-id", "externalReferenceAttachmentTypeId": ".files", "externalReferenceMetadata": { "file": { "name": "test_file", "extension": "png", "mimeType": "image/png", "createdAt": "2023-02-27T20:26:54.345Z" } }, "owner": "cases" }, { "type": "externalReference", "externalReferenceStorage": { "type": "savedObject", "soType": "file" }, "externalReferenceId": "my-id", "externalReferenceAttachmentTypeId": ".files", "externalReferenceMetadata": { "file": { "name": "test_file", "extension": "png", "mimeType": "image/png", "createdAt": "2023-02-27T20:26:54.345Z" } }, "owner": "cases" }, { "type": "externalReference", "externalReferenceStorage": { "type": "savedObject", "soType": "file" }, "externalReferenceId": "my-id", "externalReferenceAttachmentTypeId": ".files", "externalReferenceMetadata": { "file": { "name": "test_file", "extension": "jpeg", "mimeType": "image/jpeg", "createdAt": "2023-02-27T20:26:54.345Z" } }, "owner": "cases" } ] ' ``` </details>
jonathan-buttner
added a commit
that referenced
this issue
Mar 24, 2023
This PR adds telemetry for the attachment framework and files attachments by plugin and for all cases. Issue: #151933 Notable changes: - We now send information for each registered attachment type - We send average file size - We leverage the files saved object to aggregate this information - We use an array type so that we don't have to hard code each attachment type, they are dynamically discovered instead - Stats for the attachments are done for all cases and broken down by plugin <details><summary>Example telemetry payload</summary> ``` "cases": { "cases": { "all": { "total": 1, "daily": 1, "weekly": 1, "monthly": 1, "status": { "open": 1, "inProgress": 0, "closed": 0 }, "syncAlertsOn": 1, "syncAlertsOff": 0, "totalUsers": 1, "totalParticipants": 1, "totalTags": 1, "totalWithAlerts": 0, "totalWithConnectors": 0, "latestDates": { "createdAt": "2023-03-09T15:20:46.399Z", "updatedAt": "2023-03-09T15:21:02.399Z", "closedAt": null }, "assignees": { "total": 0, "totalWithZero": 1, "totalWithAtLeastOne": 0 }, "attachmentFramework": { "externalAttachments": [ { "type": ".files", "average": 4, "maxOnACase": 4, "total": 4 } ], "persistableAttachments": [], "files": { "averageSize": { "value": 3 }, "average": 4, "maxOnACase": 4, "total": 4 } } }, "sec": { "total": 0, "daily": 0, "weekly": 0, "monthly": 0, "attachmentFramework": { "externalAttachments": [], "persistableAttachments": [], "files": { "average": 0, "averageSize": 0, "maxOnACase": 0, "total": 0 } }, "assignees": { "total": 0, "totalWithZero": 0, "totalWithAtLeastOne": 0 } }, "obs": { "total": 0, "daily": 0, "weekly": 0, "monthly": 0, "attachmentFramework": { "externalAttachments": [], "persistableAttachments": [], "files": { "average": 0, "averageSize": 0, "maxOnACase": 0, "total": 0 } }, "assignees": { "total": 0, "totalWithZero": 0, "totalWithAtLeastOne": 0 } }, "main": { "total": 1, "daily": 1, "weekly": 1, "monthly": 1, "attachmentFramework": { "externalAttachments": [ { "type": ".files", "average": 4, "maxOnACase": 4, "total": 4 } ], "persistableAttachments": [], "files": { "averageSize": { "value": 3 }, "average": 4, "maxOnACase": 4, "total": 4 } }, "assignees": { "total": 0, "totalWithZero": 1, "totalWithAtLeastOne": 0 } } }, "userActions": { "all": { "total": 5, "daily": 5, "weekly": 5, "monthly": 5, "maxOnACase": 5 } }, "comments": { "all": { "total": 0, "daily": 0, "weekly": 0, "monthly": 0, "maxOnACase": 0 } }, "alerts": { "all": { "total": 0, "daily": 0, "weekly": 0, "monthly": 0, "maxOnACase": 0 } }, "connectors": { "all": { "all": { "totalAttached": 0 }, "itsm": { "totalAttached": 0 }, "sir": { "totalAttached": 0 }, "jira": { "totalAttached": 0 }, "resilient": { "totalAttached": 0 }, "swimlane": { "totalAttached": 0 }, "maxAttachedToACase": 0 } }, "pushes": { "all": { "total": 0, "maxOnACase": 0 } }, "configuration": { "all": { "closure": { "manually": 0, "automatic": 0 } } } }, ``` </details> ## Testing To test modify this file: https://github.com/elastic/kibana/blob/main/x-pack/plugins/cases/server/telemetry/schedule_telemetry_task.ts With: ``` export const scheduleCasesTelemetryTask = ( taskManager: TaskManagerStartContract, logger: Logger ) => { (async () => { await taskManager .ensureScheduled({ id: CASES_TELEMETRY_TASK_NAME, taskType: CASES_TELEMETRY_TASK_NAME, schedule: { interval: `${MINUTES_ON_HALF_DAY}m`, }, scope: ['cases'], params: {}, state: {}, }) .catch((err) => { logger.debug( `Error scheduling cases task with ID ${CASES_TELEMETRY_TASK_NAME} and type ${CASES_TELEMETRY_TASK_NAME}. Received ${err.message}` ); }); await taskManager.runSoon(CASES_TELEMETRY_TASK_NAME); })(); }; ``` This will cause the telemetry to be sent as soon as the server is restarted. To generate files and attachments to add stats to the telemetry I created this python script: https://github.com/elastic/cases-files-generator To retrieve the telemetry: ``` POST http://localhost:5601/api/telemetry/v2/clusters/_stats { "refreshCache": true, "unencrypted": true } ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This issue will outline the tasks to complete for adding file attachments to the cases backend
Add file stats API (how many files are attached to a case)Using the file service list api directlyTechnical design issue: #151780
The text was updated successfully, but these errors were encountered: