Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds new contextual attributes to Infrastructure - Inventory Rule #140598

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
730cd37
adding context for nodetype Hosts for Inventory rule
benakansara Sep 13, 2022
510589b
Merge branch 'main' of https://github.com/elastic/kibana into feature…
benakansara Sep 13, 2022
6370e18
removing unused import
benakansara Sep 13, 2022
6febb7a
Merge branch 'main' of https://github.com/elastic/kibana into feature…
benakansara Sep 13, 2022
e73adf6
Merge branch 'main' of https://github.com/elastic/kibana into feature…
benakansara Sep 13, 2022
8bff527
Merge branch 'main' of https://github.com/elastic/kibana into feature…
benakansara Sep 22, 2022
270e7b7
Merge branch 'main' of https://github.com/elastic/kibana into feature…
benakansara Sep 23, 2022
28ef8ac
adding more action variables, fixing failing tests
benakansara Sep 28, 2022
f9f72a8
Merge branch 'main' of https://github.com/elastic/kibana into feature…
benakansara Sep 28, 2022
8b639f3
minor changes
benakansara Sep 28, 2022
9a61c03
refactoring
benakansara Sep 29, 2022
61f9db1
Merge branch 'main' of https://github.com/elastic/kibana into feature…
benakansara Sep 29, 2022
9c9d01d
removing extra fields being added with context
benakansara Oct 4, 2022
2374b89
Merge branch 'main' of https://github.com/elastic/kibana into feature…
benakansara Oct 4, 2022
61fb092
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Oct 4, 2022
2b02c20
fixing failing tests
benakansara Oct 5, 2022
9c7ae7f
Merge branch 'feature/contextual-attr-inventory-rule' of https://gith…
benakansara Oct 5, 2022
c5f7cb6
Merge branch 'main' of https://github.com/elastic/kibana into feature…
benakansara Oct 11, 2022
69f92e7
refactoring
benakansara Oct 11, 2022
5223d55
Merge branch 'main' of https://github.com/elastic/kibana into feature…
benakansara Oct 11, 2022
b01f85f
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Oct 11, 2022
a5df51d
adding component template ref
benakansara Oct 11, 2022
32a03f3
Merge branch 'feature/contextual-attr-inventory-rule' of https://gith…
benakansara Oct 11, 2022
0e5fa80
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Oct 11, 2022
cb12847
Merge branch 'main' into feature/contextual-attr-inventory-rule
kibanamachine Oct 11, 2022
920aa98
[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…
kibanamachine Oct 11, 2022
f98d7f3
changed description
benakansara Oct 12, 2022
6b616f7
Merge branch 'feature/contextual-attr-inventory-rule' of https://gith…
benakansara Oct 12, 2022
a1d66dd
Merge branch 'main' into feature/contextual-attr-inventory-rule
benakansara Oct 12, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions x-pack/plugins/infra/server/lib/alerting/common/messages.ts
Original file line number Diff line number Diff line change
Expand Up @@ -214,3 +214,45 @@ export const viewInAppUrlActionVariableDescription = i18n.translate(
'Link to the view or feature within Elastic that can be used to investigate the alert and its context further',
}
);

export const cloudActionVariableDescription = i18n.translate(
'xpack.infra.metrics.alerting.cloudActionVariableDescription',
{
defaultMessage: 'The cloud object defined by ECS if available in the source.',
}
);

export const hostActionVariableDescription = i18n.translate(
'xpack.infra.metrics.alerting.hostActionVariableDescription',
{
defaultMessage: 'The host object defined by ECS if available in the source.',
}
);

export const containerActionVariableDescription = i18n.translate(
'xpack.infra.metrics.alerting.containerActionVariableDescription',
{
defaultMessage: 'The container object defined by ECS if available in the source.',
}
);

export const orchestratorActionVariableDescription = i18n.translate(
'xpack.infra.metrics.alerting.orchestratorActionVariableDescription',
{
defaultMessage: 'The orchestrator object defined by ECS if available in the source.',
}
);

export const labelsActionVariableDescription = i18n.translate(
'xpack.infra.metrics.alerting.labelsActionVariableDescription',
{
defaultMessage: 'List of labels associated with the entity where this alert triggered.',
}
);

export const tagsActionVariableDescription = i18n.translate(
'xpack.infra.metrics.alerting.tagsActionVariableDescription',
{
defaultMessage: 'List of tags associated with the entity where this alert triggered.',
}
);
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ export type ConditionResult = InventoryMetricConditions & {
currentValue: number;
isNoData: boolean;
isError: boolean;
context: [x: string];
};

export const evaluateCondition = async ({
Expand Down Expand Up @@ -82,6 +83,14 @@ export const evaluateCondition = async ({
isNoData: value === null,
isError: value === undefined,
currentValue: value.value,
context: {
cloud: value.cloud,
host: value.host,
container: value.container,
orchestrator: value.orchestrator,
labels: value.labels,
tags: value.tags,
},
};
}) as unknown; // Typescript doesn't seem to know what `throw` is doing

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ type InventoryMetricThresholdAlert = Alert<
type InventoryMetricThresholdAlertFactory = (
id: string,
reason: string,
additionalContext?: [x: string] | null,
threshold?: number | undefined,
value?: number | undefined
) => InventoryMetricThresholdAlert;
Expand All @@ -66,12 +67,13 @@ export const createInventoryMetricThresholdExecutor = (libs: InfraBackendLibs) =
if (criteria.length === 0) throw new Error('Cannot execute an alert with 0 conditions');
const logger = createScopedLogger(libs.logger, 'inventoryRule', { alertId, executionId });
const { alertWithLifecycle, savedObjectsClient, getAlertStartedDate } = services;
const alertFactory: InventoryMetricThresholdAlertFactory = (id, reason) =>
const alertFactory: InventoryMetricThresholdAlertFactory = (id, reason, additionalContext) =>
alertWithLifecycle({
id,
fields: {
[ALERT_REASON]: reason,
[ALERT_RULE_PARAMETERS]: params as any, // the type assumes the object is already flattened when writing the same way as when reading https://github.com/elastic/kibana/blob/main/x-pack/plugins/rule_registry/common/field_map/runtime_type_from_fieldmap.ts#L60
...additionalContext,
},
});

Expand Down Expand Up @@ -184,7 +186,9 @@ export const createInventoryMetricThresholdExecutor = (libs: InfraBackendLibs) =
const actionGroupId =
nextState === AlertStates.WARNING ? WARNING_ACTIONS.id : FIRED_ACTIONS.id;

const alert = alertFactory(group, reason);
const additionalContext = results && results.length > 0 ? results[0][group].context : null;

const alert = alertFactory(group, reason, additionalContext);
const indexedStartedDate = getAlertStartedDate(group) ?? startedAt.toISOString();
const viewInAppUrl = getViewInAppUrlInventory(
criteria,
Expand All @@ -193,6 +197,7 @@ export const createInventoryMetricThresholdExecutor = (libs: InfraBackendLibs) =
libs.basePath
);
scheduledActionsCount++;

const context = {
group,
alertState: stateToAlertMessage[nextState],
Expand All @@ -204,6 +209,7 @@ export const createInventoryMetricThresholdExecutor = (libs: InfraBackendLibs) =
),
threshold: mapToConditionsLookup(criteria, (c) => c.threshold),
metric: mapToConditionsLookup(criteria, (c) => c.metric),
...additionalContext,
};
alert.scheduleActions(actionGroupId, context);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,18 @@ export const createRequest = (
const metricAggregations = createMetricAggregations(timerange, nodeType, metric, customMetric);
const bucketSelector = createBucketSelector(metric, condition, customMetric);

const additionalContextAgg = {
additionalContext: {
top_hits: {
size: 1,
_source: {
includes: ['host.*', 'labels.*', 'tags', 'cloud.*', 'orchestrator.*', 'container.*'],
excludes: ['host.cpu.*', 'host.disk.*', 'host.network.*'],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❔ For curiosity: why are we excluding some of them? How did you come up with that list?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The excluded fields are "metric" fields like host.cpu.usage or host.network.ingress.bytes. They wouldn't be very useful as search fields.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@simianhacker, in which case do we need to perform a search?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly for strings like host.name or host.os.architecture, any context that might be useful for search. A good rule of thumb would be "exclude anything that changes from event to event and include anything that is common across all events for an entity".

},
},
},
};

const request: ESSearchRequest = {
allow_no_indices: true,
ignore_unavailable: true,
Expand All @@ -65,7 +77,7 @@ export const createRequest = (
aggs: {
nodes: {
composite,
aggs: { ...metricAggregations, ...bucketSelector },
aggs: { ...metricAggregations, ...bucketSelector, ...additionalContextAgg },
},
},
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,10 @@
* 2.0.
*/

import { AggregationsAggregate, SearchResponse } from '@elastic/elasticsearch/lib/api/types';
import { ElasticsearchClient } from '@kbn/core/server';
import type { Logger } from '@kbn/logging';
import { EcsFieldsResponse } from '@kbn/rule-registry-plugin/common/search_strategy';
import { InventoryMetricConditions } from '../../../../../common/alerting/metrics';
import { InfraTimerangeInput, SnapshotCustomMetricInput } from '../../../../../common/http_api';
import {
Expand All @@ -20,13 +22,28 @@ import { createRequest } from './create_request';
interface BucketKey {
node: string;
}
type Response = Record<string, { value: number | null; warn: boolean; trigger: boolean }>;

interface AdditionalContext {
[x: string]: any;
}

type Response = Record<
string,
{
value: number | null;
warn: boolean;
trigger: boolean;
} & AdditionalContext
>;

type Metric = Record<string, { value: number | null }>;

interface Bucket {
key: BucketKey;
doc_count: number;
shouldWarn: { value: number };
shouldTrigger: { value: number };
additionalContext: SearchResponse<EcsFieldsResponse, Record<string, AggregationsAggregate>>;
}
type NodeBucket = Bucket & Metric;
interface ResponseAggregations {
Expand Down Expand Up @@ -56,10 +73,15 @@ export const getData = async (
const nextAfterKey = nodes.after_key;
for (const bucket of nodes.buckets) {
const metricId = customMetric && customMetric.field ? customMetric.id : metric;
const bucketHits = bucket.additionalContext?.hits?.hits;
const additionalContextSource =
bucketHits && bucketHits.length > 0 ? bucketHits[0]._source : null;

previous[bucket.key.node] = {
value: bucket?.[metricId]?.value ?? null,
warn: bucket?.shouldWarn.value > 0 ?? false,
trigger: bucket?.shouldTrigger.value > 0 ?? false,
...additionalContextSource,
};
}
if (nextAfterKey) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,15 @@ import {
import { InfraBackendLibs } from '../../infra_types';
import {
alertStateActionVariableDescription,
cloudActionVariableDescription,
containerActionVariableDescription,
groupActionVariableDescription,
hostActionVariableDescription,
labelsActionVariableDescription,
metricActionVariableDescription,
orchestratorActionVariableDescription,
reasonActionVariableDescription,
tagsActionVariableDescription,
thresholdActionVariableDescription,
timestampActionVariableDescription,
valueActionVariableDescription,
Expand Down Expand Up @@ -102,6 +108,12 @@ export async function registerMetricInventoryThresholdRuleType(
{ name: 'metric', description: metricActionVariableDescription },
{ name: 'threshold', description: thresholdActionVariableDescription },
{ name: 'viewInAppUrl', description: viewInAppUrlActionVariableDescription },
{ name: 'cloud', description: cloudActionVariableDescription },
{ name: 'host', description: hostActionVariableDescription },
{ name: 'container', description: containerActionVariableDescription },
{ name: 'orchestrator', description: orchestratorActionVariableDescription },
{ name: 'labels', description: labelsActionVariableDescription },
{ name: 'tags', description: tagsActionVariableDescription },
],
},
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { mappingFromFieldMap } from '@kbn/rule-registry-plugin/common/mapping_fr
import { experimentalRuleFieldMap } from '@kbn/rule-registry-plugin/common/assets/field_maps/experimental_rule_field_map';

import { Dataset, RuleRegistryPluginSetupContract } from '@kbn/rule-registry-plugin/server';
import { ECS_COMPONENT_TEMPLATE_NAME } from '@kbn/rule-registry-plugin/common/assets';
import type { InfraFeatureId } from '../../../common/constants';
import { RuleRegistrationContext, RulesServiceStartDeps } from './types';

Expand All @@ -30,7 +31,7 @@ export const createRuleDataClient = ({
feature: ownerFeatureId,
registrationContext,
dataset: Dataset.alerts,
componentTemplateRefs: [],
componentTemplateRefs: [ECS_COMPONENT_TEMPLATE_NAME],
componentTemplates: [
{
name: 'mappings',
Expand Down
Loading