[Cases] Adding file telemetry #152968

jonathan-buttner · 2023-03-08T22:22:08Z

This PR adds telemetry for the attachment framework and files attachments by plugin and for all cases.

Issue: #151933

Notable changes:

We now send information for each registered attachment type
We send average file size
- We leverage the files saved object to aggregate this information
- We use an array type so that we don't have to hard code each attachment type, they are dynamically discovered instead
Stats for the attachments are done for all cases and broken down by plugin

Example telemetry payload

                        "cases": {
                            "cases": {
                                "all": {
                                    "total": 1,
                                    "daily": 1,
                                    "weekly": 1,
                                    "monthly": 1,
                                    "status": {
                                        "open": 1,
                                        "inProgress": 0,
                                        "closed": 0
                                    },
                                    "syncAlertsOn": 1,
                                    "syncAlertsOff": 0,
                                    "totalUsers": 1,
                                    "totalParticipants": 1,
                                    "totalTags": 1,
                                    "totalWithAlerts": 0,
                                    "totalWithConnectors": 0,
                                    "latestDates": {
                                        "createdAt": "2023-03-09T15:20:46.399Z",
                                        "updatedAt": "2023-03-09T15:21:02.399Z",
                                        "closedAt": null
                                    },
                                    "assignees": {
                                        "total": 0,
                                        "totalWithZero": 1,
                                        "totalWithAtLeastOne": 0
                                    },
                                    "attachmentFramework": {
                                        "externalAttachments": [
                                            {
                                                "type": ".files",
                                                "average": 4,
                                                "maxOnACase": 4,
                                                "total": 4
                                            }
                                        ],
                                        "persistableAttachments": [],
                                        "files": {
                                            "averageSize": {
                                                "value": 3
                                            },
                                            "average": 4,
                                            "maxOnACase": 4,
                                            "total": 4
                                        }
                                    }
                                },
                                "sec": {
                                    "total": 0,
                                    "daily": 0,
                                    "weekly": 0,
                                    "monthly": 0,
                                    "attachmentFramework": {
                                        "externalAttachments": [],
                                        "persistableAttachments": [],
                                        "files": {
                                            "average": 0,
                                            "averageSize": 0,
                                            "maxOnACase": 0,
                                            "total": 0
                                        }
                                    },
                                    "assignees": {
                                        "total": 0,
                                        "totalWithZero": 0,
                                        "totalWithAtLeastOne": 0
                                    }
                                },
                                "obs": {
                                    "total": 0,
                                    "daily": 0,
                                    "weekly": 0,
                                    "monthly": 0,
                                    "attachmentFramework": {
                                        "externalAttachments": [],
                                        "persistableAttachments": [],
                                        "files": {
                                            "average": 0,
                                            "averageSize": 0,
                                            "maxOnACase": 0,
                                            "total": 0
                                        }
                                    },
                                    "assignees": {
                                        "total": 0,
                                        "totalWithZero": 0,
                                        "totalWithAtLeastOne": 0
                                    }
                                },
                                "main": {
                                    "total": 1,
                                    "daily": 1,
                                    "weekly": 1,
                                    "monthly": 1,
                                    "attachmentFramework": {
                                        "externalAttachments": [
                                            {
                                                "type": ".files",
                                                "average": 4,
                                                "maxOnACase": 4,
                                                "total": 4
                                            }
                                        ],
                                        "persistableAttachments": [],
                                        "files": {
                                            "averageSize": {
                                                "value": 3
                                            },
                                            "average": 4,
                                            "maxOnACase": 4,
                                            "total": 4
                                        }
                                    },
                                    "assignees": {
                                        "total": 0,
                                        "totalWithZero": 1,
                                        "totalWithAtLeastOne": 0
                                    }
                                }
                            },
                            "userActions": {
                                "all": {
                                    "total": 5,
                                    "daily": 5,
                                    "weekly": 5,
                                    "monthly": 5,
                                    "maxOnACase": 5
                                }
                            },
                            "comments": {
                                "all": {
                                    "total": 0,
                                    "daily": 0,
                                    "weekly": 0,
                                    "monthly": 0,
                                    "maxOnACase": 0
                                }
                            },
                            "alerts": {
                                "all": {
                                    "total": 0,
                                    "daily": 0,
                                    "weekly": 0,
                                    "monthly": 0,
                                    "maxOnACase": 0
                                }
                            },
                            "connectors": {
                                "all": {
                                    "all": {
                                        "totalAttached": 0
                                    },
                                    "itsm": {
                                        "totalAttached": 0
                                    },
                                    "sir": {
                                        "totalAttached": 0
                                    },
                                    "jira": {
                                        "totalAttached": 0
                                    },
                                    "resilient": {
                                        "totalAttached": 0
                                    },
                                    "swimlane": {
                                        "totalAttached": 0
                                    },
                                    "maxAttachedToACase": 0
                                }
                            },
                            "pushes": {
                                "all": {
                                    "total": 0,
                                    "maxOnACase": 0
                                }
                            },
                            "configuration": {
                                "all": {
                                    "closure": {
                                        "manually": 0,
                                        "automatic": 0
                                    }
                                }
                            }
                        },

Testing

To test modify this file: https://github.com/elastic/kibana/blob/main/x-pack/plugins/cases/server/telemetry/schedule_telemetry_task.ts

With:

export const scheduleCasesTelemetryTask = (
  taskManager: TaskManagerStartContract,
  logger: Logger
) => {
  (async () => {
    await taskManager
      .ensureScheduled({
        id: CASES_TELEMETRY_TASK_NAME,
        taskType: CASES_TELEMETRY_TASK_NAME,
        schedule: {
          interval: `${MINUTES_ON_HALF_DAY}m`,
        },
        scope: ['cases'],
        params: {},
        state: {},
      })
      .catch((err) => {
        logger.debug(
          `Error scheduling cases task with ID ${CASES_TELEMETRY_TASK_NAME} and type ${CASES_TELEMETRY_TASK_NAME}. Received ${err.message}`
        );
      });

    await taskManager.runSoon(CASES_TELEMETRY_TASK_NAME);
  })();
};

This will cause the telemetry to be sent as soon as the server is restarted.

To generate files and attachments to add stats to the telemetry I created this python script: https://github.com/elastic/cases-files-generator

To retrieve the telemetry:

POST http://localhost:5601/api/telemetry/v2/clusters/_stats
{
   "refreshCache": true,
   "unencrypted": true
}

jonathan-buttner · 2023-03-09T17:27:44Z

x-pack/plugins/cases/server/telemetry/constants.ts

-/**
- * This should only be used within telemetry
- */
-export const OWNERS = [OBSERVABILITY_OWNER, SECURITY_SOLUTION_OWNER, GENERAL_CASES_OWNER] as const;


We have these values in common now.

jonathan-buttner · 2023-03-09T17:28:54Z

x-pack/plugins/cases/server/telemetry/index.ts

@@ -42,7 +43,7 @@ export const createCasesTelemetry = async ({
 }: CreateCasesTelemetryArgs) => {
  const getInternalSavedObjectClient = async (): Promise<ISavedObjectsRepository> => {
    const [coreStart] = await core.getStartServices();
-    return coreStart.savedObjects.createInternalRepository(SAVED_OBJECT_TYPES);
+    return coreStart.savedObjects.createInternalRepository([...SAVED_OBJECT_TYPES, FILE_SO_TYPE]);


Just calling this out, to retrieve the average file size we have to do an average aggregation over the file service's saved objects that contain a metadata field caseId. Hopefully it's ok that we're accessing the file service's saved object internals.

The advantage of doing it this way is that we don't need to duplicate the file size within case comments.

jonathan-buttner · 2023-03-09T17:29:45Z

x-pack/plugins/cases/server/telemetry/queries/cases.ts

-    createdAt: savedObjects?.[0].saved_objects?.[0].attributes?.created_at ?? null,
-    updatedAt: savedObjects?.[1].saved_objects?.[0].attributes?.updated_at ?? null,
-    closedAt: savedObjects?.[2].saved_objects?.[0].attributes?.closed_at ?? null,
+    createdAt: savedObjects?.[0]?.saved_objects?.[0]?.attributes?.created_at ?? null,


When the telemetry runs initially and there are no cases this was throwing an error

jonathan-buttner · 2023-03-09T17:30:32Z

x-pack/plugins/cases/server/telemetry/queries/cases.ts

  };
 };

 export const getCasesTelemetryData = async ({
  savedObjectsClient,
  logger,
 }: CollectTelemetryDataParams): Promise<CasesTelemetry['cases']> => {
-  const byOwnerAggregationQuery = OWNERS.reduce(
+  try {


Did some refactoring here, wrapped it all in a try/catch so we can more easily identify errors when we're developing additional telemetry and now doing a promise.all

…emetry

jonathan-buttner · 2023-03-09T21:58:48Z

x-pack/plugins/cases/server/telemetry/queries/cases.ts

-    createdAt: savedObjects?.[0].saved_objects?.[0].attributes?.created_at ?? null,
-    updatedAt: savedObjects?.[1].saved_objects?.[0].attributes?.updated_at ?? null,
-    closedAt: savedObjects?.[2].saved_objects?.[0].attributes?.closed_at ?? null,
+    createdAt: savedObjects?.[0]?.saved_objects?.[0]?.attributes?.created_at ?? '',


Prior to adding the ? this was causing an exception if saved_objects[0] was undefined which was skipping returning null. Now that this doesn't throw an error I'm getting test failures because a keyword field can't be mapped to null 🤷‍♂️ .

elasticmachine · 2023-03-10T01:10:19Z

Pinging @elastic/response-ops (Team:ResponseOps)

elasticmachine · 2023-03-10T01:10:20Z

Pinging @elastic/response-ops-cases (Feature:Cases)

afharo

Approving to unblock. I added a nit, though.

afharo · 2023-03-16T21:37:24Z

x-pack/plugins/cases/server/telemetry/schema.ts

+  persistableAttachments: attachmentRegistrySchema,
+  externalAttachments: attachmentRegistrySchema,
+  files: {
+    average: long,


Nit: do we want to keep the decimals in the avg? If so, we might need to mark it as double or float. WDYT?

Great question, I was thinking integers since I don't know if care about the precision that much for this telemetry. I'll ask my PM though 👍

I checked with my PM and we're going to keep them as longs.

kibana-ci · 2023-03-21T20:28:46Z

💚 Build Succeeded

Buildkite Build
Commit: 5a992b7

Metrics [docs]

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
`securitySolution`	433	436	+3

Total ESLint disabled count

id	before	after	diff
`securitySolution`	513	516	+3

History

💚 Build #113179 succeeded a6bbf17
💔 Build #113155 failed 73c89da
💔 Build #113130 failed c12d75a
💔 Build #113120 failed 913070c
💔 Build #113116 failed 89229fa
💔 Build #112900 failed 8268027

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cnasikas

LGTM! Thanks for the refactor. Much cleaner.

Adding telemetry changes

8268027

jonathan-buttner added release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) Feature:Cases Cases feature v8.8.0 labels Mar 8, 2023

jonathan-buttner mentioned this pull request Mar 8, 2023

[Cases] Implement files backend changes #151933

Closed

13 tasks

Adding telemetry for all cases

89229fa

jonathan-buttner commented Mar 9, 2023

View reviewed changes

jonathan-buttner added 5 commits March 9, 2023 12:35

Merge branch 'main' of github.com:elastic/kibana into cases-files-tel…

913070c

…emetry

Fixing type errors

960e3dd

Forgot type

c12d75a

Fixing telemetry mapping errors

73c89da

Fixing schema failures from null

a6bbf17

jonathan-buttner commented Mar 9, 2023

View reviewed changes

jonathan-buttner marked this pull request as ready for review March 10, 2023 01:10

jonathan-buttner requested review from a team as code owners March 10, 2023 01:10

afharo approved these changes Mar 16, 2023

View reviewed changes

Merge branch 'main' into cases-files-telemetry

5a992b7

cnasikas approved these changes Mar 22, 2023

View reviewed changes

jonathan-buttner merged commit f17fe1b into elastic:main Mar 24, 2023

kibanamachine added the backport:skip This commit does not require backporting label Mar 24, 2023

jonathan-buttner deleted the cases-files-telemetry branch March 24, 2023 19:43

cnasikas mentioned this pull request Apr 3, 2023

[Cases][Meta]: File attachments #151595

Closed

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Cases] Adding file telemetry #152968

[Cases] Adding file telemetry #152968

jonathan-buttner commented Mar 8, 2023 •

edited

Loading

jonathan-buttner Mar 9, 2023

jonathan-buttner Mar 9, 2023

jonathan-buttner Mar 9, 2023

jonathan-buttner Mar 9, 2023

jonathan-buttner Mar 9, 2023

elasticmachine commented Mar 10, 2023

elasticmachine commented Mar 10, 2023

afharo left a comment

afharo Mar 16, 2023

jonathan-buttner Mar 20, 2023

jonathan-buttner Mar 21, 2023

kibana-ci commented Mar 21, 2023

ESLint disabled line counts

Total ESLint disabled count

cnasikas left a comment •

edited

Loading

[Cases] Adding file telemetry #152968

[Cases] Adding file telemetry #152968

Conversation

jonathan-buttner commented Mar 8, 2023 • edited Loading

Testing

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elasticmachine commented Mar 10, 2023

elasticmachine commented Mar 10, 2023

afharo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kibana-ci commented Mar 21, 2023

💚 Build Succeeded

Metrics [docs]

ESLint disabled line counts

Total ESLint disabled count

History

cnasikas left a comment • edited Loading

Choose a reason for hiding this comment

jonathan-buttner commented Mar 8, 2023 •

edited

Loading

cnasikas left a comment •

edited

Loading