Adds telemetry support to alerting and actions plugins #58081

YulNaumenko · 2020-02-20T03:08:23Z

Summary

Checklist

Delete any items that are not applicable to this PR.

Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios

elasticmachine · 2020-02-20T03:08:26Z

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

…telemetry # Please enter a commit message to explain why this merge is necessary, # especially if it merges an updated upstream into a topic branch. # # Lines starting with '#' will be ignored, and an empty message aborts # the commit.

pmuellr

code LGTM, made some comments; will have a few more comments outside of the review, in the main part of the PR

x-pack/plugins/actions/server/plugin.ts

x-pack/plugins/actions/server/usage/actions_telemetry.ts

x-pack/legacy/plugins/actions/server/mappings.json

pmuellr · 2020-02-27T16:18:03Z

x-pack/plugins/actions/server/plugin.ts

@@ -162,6 +170,22 @@ export class ActionsPlugin implements Plugin<Promise<PluginSetupContract>, Plugi
      actionsConfigUtils,
    });

+    const usageCollection = plugins.usageCollection;
+    if (usageCollection) {
+      core.getStartServices().then(async ([coreStart, startPlugins]: [CoreStart, any]) => {


oh neat, didn't realized you could get start services (async) in setup!

pmuellr · 2020-02-27T16:31:12Z

x-pack/plugins/actions/server/usage/actions_usage_collector.ts

+  taskManager: TaskManagerStartContract
+) {
+  return usageCollection.makeUsageCollector({
+    type: 'actions',


so does usageCollection take type: actions and then look for actions-telemetry types in the SO? I'm not seeing any other references to actions-telemetry in the code, other than in the SO mappings at the top, so guessing this must be how it's done.

x-pack/plugins/actions/server/usage/task.ts

x-pack/plugins/alerting/server/alerts_client.ts

x-pack/plugins/alerting/server/usage/alerts_telemetry.ts

…telemetry # Please enter a commit message to explain why this merge is necessary, # especially if it merges an updated upstream into a topic branch. # # Lines starting with '#' will be ignored, and an empty message aborts # the commit.

…hedule and connectors count

mikecote

It's coming along, great work! 👍 Just a few comments, some of them are duplicates between alerts and actions and then a few nits.

x-pack/plugins/actions/server/usage/task.ts

x-pack/plugins/alerting/server/usage/task.ts

x-pack/plugins/actions/server/usage/actions_usage_collector.ts

x-pack/plugins/alerting/server/usage/alerts_usage_collector.ts

x-pack/plugins/alerting/server/usage/alerts_telemetry.ts

x-pack/plugins/actions/server/usage/actions_telemetry.ts

…telemetry # Please enter a commit message to explain why this merge is necessary, # especially if it merges an updated upstream into a topic branch. # # Lines starting with '#' will be ignored, and an empty message aborts # the commit.

mikecote

This is awesome! LGTM 👍I like how the number of requests is reduced.

Just wanted to confirm we have follow up issues created for the missing telemetry items? (executions, usage count, etc)

…telemetry # Please enter a commit message to explain why this merge is necessary, # especially if it merges an updated upstream into a topic branch. # # Lines starting with '#' will be ignored, and an empty message aborts # the commit.

pmuellr

LGTM, made some comments, seems fine to ship as-is

pmuellr · 2020-03-12T16:23:09Z

x-pack/plugins/actions/server/usage/actions_telemetry.ts

+import { APICaller } from 'kibana/server';
+
+export async function getTotalCount(callCluster: APICaller, kibanaIndex: string) {
+  const scriptedMetric = {


I'm slightly surprised we can't use some existing agg here, instead of having to script it.

Existing aggregations is designed to work on the document fields, but in case it is a child document like a relation or alert.actions list - we should use nested path and in this case we still able to aggregate only fields on the nested level. I did a deep research over documentation and didn't find any other option https://www.elastic.co/guide/en/elasticsearch/reference/7.6/search-aggregations-bucket-nested-aggregation.html. The most pain here that we don't have an actionTypeId in 'relations', but have it on alert.actions and both of this objects are the separate branches of nesting which is not accessible for each other.

pmuellr · 2020-03-12T16:26:49Z

x-pack/plugins/actions/server/usage/actions_telemetry.ts

+  };
+}
+
+export async function getInUseTotalCount(callCluster: APICaller, kibanaIndex: string) {


It seems like we could probably do all these aggs in a single search, but I doubt there's a great reason to do that, since they won't be running all that often. And it no doubt easier to deal with a small set of semi-complex searchs, rather than run a single search with a bunch of semi-complex aggs in it.

I had the same thoughts, when was writing this code, but decided to keep it separately for now.

pmuellr · 2020-03-12T16:36:34Z

.../triggers_actions_ui/public/application/sections/alerts_list/components/alerts_list.test.tsx

@@ -4,7 +4,7 @@
 * you may not use this file except in compliance with the Elastic License.
 */
 import * as React from 'react';
-import { mountWithIntl } from 'test_utils/enzyme_helpers';
+import { mountWithIntl, nextTick } from 'test_utils/enzyme_helpers';


Are the changes in this file required for the PR? Like, the server plugin wasn't completely initialized before, or something?

Yeah, it is the last weird test I found. Decided to fix it, because it cause jest tests failing with time out.

kibanamachine · 2020-03-12T17:43:33Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: edef1a3

History

💚 Build #32663 succeeded b799870
💚 Build #31934 succeeded 6a236af
💔 Build #31859 failed 7ac9035
💚 Build #29598 succeeded aafa6c1
💛 Build #29522 was flaky 491c7df

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* Added base telemetry functionality * Fixed actions collector due to NP plugin changes * Fixed type checks * Fixed alerting plugin usage collector * Added actions and alerting telemetry tasks * Fixed failing tests and resolved comments * Extended telemetry for alerts by adding aggregations for throttle, schedule and connectors count * Fixed tests * Refactored using callCluster aggregations * Fixed compare * Fixed time convertion

* master: Endpoint: Change the input type for @kbn/config-schema to work with more schemas (elastic#60007) Using re2 for Timelion regular expressions (elastic#55208) [Monitoring] Re-enable logstash tests (elastic#59815) fix karma debug typo (elastic#60029) Adds telemetry support to alerting and actions plugins (elastic#58081)

* Added base telemetry functionality * Fixed actions collector due to NP plugin changes * Fixed type checks * Fixed alerting plugin usage collector * Added actions and alerting telemetry tasks * Fixed failing tests and resolved comments * Extended telemetry for alerts by adding aggregations for throttle, schedule and connectors count * Fixed tests * Refactored using callCluster aggregations * Fixed compare * Fixed time convertion

TinaHeiligers · 2020-03-25T19:29:37Z

x-pack/plugins/alerting/server/usage/alerts_usage_collector.ts

+            avg: 0,
+            max: 0,
+          },
+          schedule_time: {


After checking off line with @YulNaumenko , the mapping for the schedule_time and throttle_time properties should be strings.

Added base telemetry functionality

6980b46

YulNaumenko added Feature:Alerting v8.0.0 release_note:skip Skip the PR/issue when compiling release notes v7.7.0 Feature:Actions Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) labels Feb 20, 2020

YulNaumenko requested a review from a team as a code owner February 20, 2020 03:08

YulNaumenko self-assigned this Feb 20, 2020

YulNaumenko added 4 commits February 21, 2020 09:19

Fixed actions collector due to NP plugin changes

41aedb9

Fixed type checks

3e7dee2

Fixed alerting plugin usage collector

e9a101c

YulNaumenko changed the title ~~Adds telemetry support to alerting and actions plugins #49832 #54214~~ Adds telemetry support to alerting and actions plugins #49832 Feb 25, 2020

YulNaumenko changed the title ~~Adds telemetry support to alerting and actions plugins #49832~~ Adds telemetry support to alerting and actions plugins Feb 25, 2020

Added actions and alerting telemetry tasks

82e42d8

pmuellr mentioned this pull request Feb 27, 2020

[alerting event log] add query support #55633

Closed

pmuellr approved these changes Feb 27, 2020

View reviewed changes

YulNaumenko added 3 commits February 27, 2020 09:51

Fixed failing tests and resolved comments

491c7df

Extended telemetry for alerts by adding aggregations for throttle, sc…

aafa6c1

…hedule and connectors count

mikecote self-requested a review February 28, 2020 13:29

mikecote reviewed Feb 28, 2020

View reviewed changes

dgieselaar mentioned this pull request Mar 6, 2020

[APM] Threshold alerts #59566

Merged

YulNaumenko added 4 commits March 9, 2020 10:59

Fixed tests

6a236af

Refactored using callCluster aggregations

62af6b8

YulNaumenko requested a review from mikecote March 12, 2020 04:58

Fixed compare

b799870

mikecote approved these changes Mar 12, 2020

View reviewed changes

YulNaumenko added 2 commits March 12, 2020 08:57

Fixed time convertion

c2ceb31

pmuellr approved these changes Mar 12, 2020

View reviewed changes

YulNaumenko merged commit d3f53f6 into elastic:master Mar 12, 2020

YulNaumenko mentioned this pull request Mar 12, 2020

[7.x] Adds telemetry support to alerting and actions plugins (#58081) #60043

Merged

YulNaumenko mentioned this pull request Mar 12, 2020

[Actions] Implement number of active connectors by action type #60046

Closed

mikecote mentioned this pull request Mar 16, 2020

More alerting services telemetry #60315

Closed

3 tasks

TinaHeiligers reviewed Mar 25, 2020

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds telemetry support to alerting and actions plugins #58081

Adds telemetry support to alerting and actions plugins #58081

YulNaumenko commented Feb 20, 2020 •

edited

Loading

elasticmachine commented Feb 20, 2020

pmuellr left a comment

pmuellr Feb 27, 2020

pmuellr Feb 27, 2020

mikecote left a comment

mikecote left a comment

pmuellr left a comment

pmuellr Mar 12, 2020

YulNaumenko Mar 12, 2020

pmuellr Mar 12, 2020

YulNaumenko Mar 12, 2020

pmuellr Mar 12, 2020

YulNaumenko Mar 12, 2020 •

edited

Loading

kibanamachine commented Mar 12, 2020

TinaHeiligers Mar 25, 2020

Adds telemetry support to alerting and actions plugins #58081

Adds telemetry support to alerting and actions plugins #58081

Conversation

YulNaumenko commented Feb 20, 2020 • edited Loading

Summary

Checklist

elasticmachine commented Feb 20, 2020

pmuellr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mikecote left a comment

Choose a reason for hiding this comment

mikecote left a comment

Choose a reason for hiding this comment

pmuellr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

YulNaumenko Mar 12, 2020 • edited Loading

Choose a reason for hiding this comment

kibanamachine commented Mar 12, 2020

💚 Build Succeeded

History

Choose a reason for hiding this comment

YulNaumenko commented Feb 20, 2020 •

edited

Loading

YulNaumenko Mar 12, 2020 •

edited

Loading