Skip to content

Commit

Permalink
[RAM] Adds auto-incrementing revision field to rules (elastic#147398)
Browse files Browse the repository at this point in the history
## Summary

Resolves elastic#137164. 

This PR adds a new `revision` field (number) to Alerting Rules that
auto-increments when relevant content changes have been made via the
`Rules Client`. _Relevant content changes_ are defined as any content
change that the user may either want to be notified about, or have the
option to later revert to. This will include most all changes, except
the enabling/disabling of the rule, or updates to metadata fields like
`executionStatus` or `monitoring`. This `revision` field is not intended
to be user editable, and should be ignored if ever provided via the API.

See elastic#136213 for additional
details. To be followed-up by
elastic#137168, which will remove the
version bump logic from security solution.

## Details

### Migrations

Includes SO migration to default `revision` to `0` when upgrading a
cluster, or when importing `pre-8.8.0` rules via the SO Management UI.
For consistency, security rule import follows the same logic as SO
Management and will not reset the `revision` to `0` when overriding or
creating a new rule.

### Downstream Index Updates

* EventLog _has not_ been updated to include `revision` along with basic
rule fields currently being written. Should we?
* AAD Schema will be updated in
elastic#151388 (as this one is getting
pretty big) to include `revision` so alerts written will include which
specific revision of the rule created the alert.

### Reference Fields

Any creation of or modification to `actions` will result in a revision
increment. More typical reference fields like `exception lists` on the
security side will only result in a revision increment when the list is
initially associated/deleted from the rule (as subsequent updates will
be done directly against the list).

### RuleClient Updates

The following methods within the RuleClient have been updated to support
incrementing revision when relevant field changes have been detected:

* `clone()` - resets to 0 currently (see open question)
* `update()` - increments `revision` so long a change has been made to
relevant fields (fields not in [ignore
list](https://github.com/elastic/kibana/pull/147398/files#diff-6736e143ede2dc06e825bddcdc23b4d088a6620805751db4eddc5900d586c4dfR69-R85))
* `bulkEdit()` - increments `revision` for relevant fields (all current
bulk edit fields minus api key/snooze/mute)

Mutation methods not updated to include revision log:
* `snooze()`
* `unsnooze()`
*  `clearExpiredSnoozes()`
*  `muteAll()`
*  `unmuteAll()`
*  `muteInstance()`
*  `unmuteInstance()`
* `updateApiKey()` - increments revision as rule functionality could be
impacted


## Open questions:
- [X] Should `clone()` in RulesClient reset revision to 0 as if it's a
new rule, or keep the current value? (see
[comment](https://github.com/elastic/kibana/pull/147398/files#r1106484105))
- [X] What about snooze/un-snooze, and mute/unmute? Should we update
revision on these field changes as well? (see
[comment](https://github.com/elastic/kibana/pull/147398/files#r1106431966))
- Discussed with @XavierM and determined to not update on
snooze/mute/API key changes as this actions could be plentiful and don't
necessarily represent a version of the rule a user would want to revert
to, thus polluting the revision history.
- [ ] Should we write `revision` to EventLog?


---

### Checklist

Delete any items that are not applicable to this PR.

- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
  - [ ] To work with docs team 
- [X] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios


### For maintainers

- [X] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
  • Loading branch information
spong authored and bmorelli25 committed Mar 10, 2023
1 parent 51d50c2 commit 2c71631
Show file tree
Hide file tree
Showing 112 changed files with 663 additions and 20 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ describe('checking migration metadata changes on all registered SO types', () =>
Object {
"action": "6cfc277ed3211639e37546ac625f4a68f2494215",
"action_task_params": "5f419caba96dd8c77d0f94013e71d43890e3d5d6",
"alert": "785240e3137f5eb1a0f8986e5b8eff99780fc04f",
"alert": "1e4cd6941f1eb39c729c646e91fbfb9700de84b9",
"api_key_pending_invalidation": "16e7bcf8e78764102d7f525542d5b616809a21ee",
"apm-indices": "d19dd7fb51f2d2cbc1f8769481721e0953f9a6d2",
"apm-server-schema": "1d42f17eff9ec6c16d3a9324d9539e2d123d0a9a",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ const mockRule: RuleTableItem = {
ruleType: 'Test Rule Type',
isEditable: true,
enabledInLicense: true,
revision: 0,
};

export const RulesListNotifyBadgeSandbox = ({ triggersActionsUi }: SandboxProps) => {
Expand Down
1 change: 1 addition & 0 deletions x-pack/plugins/alerting/common/rule.ts
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,7 @@ export interface Rule<Params extends RuleTypeParams = never> {
isSnoozedUntil?: Date | null;
lastRun?: RuleLastRun | null;
nextRun?: Date | null;
revision: number;
running?: boolean | null;
viewInAppRelativeUrl?: string;
}
Expand Down
3 changes: 3 additions & 0 deletions x-pack/plugins/alerting/public/alert_api.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ describe('loadRule', () => {
"params": Object {
"x": 42,
},
"revision": 0,
"schedule": Object {
"interval": "1s",
},
Expand Down Expand Up @@ -268,6 +269,7 @@ function getApiRule() {
updated_by: '2889684073',
mute_all: false,
muted_alert_ids: [],
revision: 0,
schedule: {
interval: '1s',
},
Expand Down Expand Up @@ -333,5 +335,6 @@ function getRule(): Rule<{ x: number }> {
lastExecutionDate: RuleExecuteDate,
lastDuration: 1194,
},
revision: 0,
};
}
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ describe('common_transformations', () => {
notify_when: 'onActiveAlert',
mute_all: false,
muted_alert_ids: ['bob', 'jim'],
revision: 0,
execution_status: {
last_execution_date: dateExecuted.toISOString(),
last_duration: 42,
Expand Down Expand Up @@ -185,6 +186,7 @@ describe('common_transformations', () => {
],
},
},
"revision": 0,
"schedule": Object {
"interval": "1s",
},
Expand Down Expand Up @@ -229,6 +231,7 @@ describe('common_transformations', () => {
notify_when: 'onActiveAlert',
mute_all: false,
muted_alert_ids: ['bob', 'jim'],
revision: 0,
execution_status: {
last_execution_date: dateExecuted.toISOString(),
status: 'error',
Expand Down Expand Up @@ -344,6 +347,7 @@ describe('common_transformations', () => {
"nextRun": 2021-12-15T12:34:55.789Z,
"notifyWhen": "onActiveAlert",
"params": Object {},
"revision": 0,
"schedule": Object {
"interval": "1s",
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -784,4 +784,5 @@ const BaseRule: SanitizedRule<{ bar: boolean }> = {
status: 'unknown',
lastExecutionDate: new Date('2020-08-20T19:23:38Z'),
},
revision: 0,
};
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ describe('bulkEditInternalRulesRoute', () => {
status: 'unknown',
lastExecutionDate: new Date('2020-08-20T19:23:38Z'),
},
revision: 0,
};

const mockedAlerts: Array<SanitizedRule<{}>> = [mockedAlert];
Expand Down
2 changes: 2 additions & 0 deletions x-pack/plugins/alerting/server/routes/clone_rule.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ describe('cloneRuleRoute', () => {
status: 'unknown',
lastExecutionDate: new Date('2020-08-20T19:23:38Z'),
},
revision: 0,
};

const ruleToClone: AsApiContract<CreateOptions<{ bar: boolean }>['data']> = {
Expand All @@ -90,6 +91,7 @@ describe('cloneRuleRoute', () => {
created_at: mockedRule.createdAt,
updated_at: mockedRule.updatedAt,
id: mockedRule.id,
revision: 0,
execution_status: {
status: mockedRule.executionStatus.status,
last_execution_date: mockedRule.executionStatus.lastExecutionDate,
Expand Down
2 changes: 2 additions & 0 deletions x-pack/plugins/alerting/server/routes/create_rule.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ describe('createRuleRoute', () => {
status: 'unknown',
lastExecutionDate: new Date('2020-08-20T19:23:38Z'),
},
revision: 0,
};

const ruleToCreate: AsApiContract<CreateOptions<{ bar: boolean }>['data']> = {
Expand All @@ -93,6 +94,7 @@ describe('createRuleRoute', () => {
created_at: mockedAlert.createdAt,
updated_at: mockedAlert.updatedAt,
id: mockedAlert.id,
revision: mockedAlert.revision,
execution_status: {
status: mockedAlert.executionStatus.status,
last_execution_date: mockedAlert.executionStatus.lastExecutionDate,
Expand Down
2 changes: 2 additions & 0 deletions x-pack/plugins/alerting/server/routes/get_rule.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ describe('getRuleRoute', () => {
status: 'unknown',
lastExecutionDate: new Date('2020-08-20T19:23:38Z'),
},
revision: 0,
};

const getResult: AsApiContract<SanitizedRule<{ bar: boolean }>> = {
Expand All @@ -76,6 +77,7 @@ describe('getRuleRoute', () => {
created_at: mockedAlert.createdAt,
updated_at: mockedAlert.updatedAt,
id: mockedAlert.id,
revision: mockedAlert.revision,
execution_status: {
status: mockedAlert.executionStatus.status,
last_execution_date: mockedAlert.executionStatus.lastExecutionDate,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ describe('createAlertRoute', () => {
status: 'unknown',
lastExecutionDate: new Date('2020-08-20T19:23:38Z'),
},
revision: 0,
};

it('creates an alert with proper parameters', async () => {
Expand Down
1 change: 1 addition & 0 deletions x-pack/plugins/alerting/server/routes/legacy/get.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ describe('getAlertRoute', () => {
status: 'unknown',
lastExecutionDate: new Date('2020-08-20T19:23:38Z'),
},
revision: 0,
};

it('gets an alert with proper parameters', async () => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ const sampleRule: SanitizedRule<RuleTypeParams> & { activeSnoozes?: string[] } =
},
},
nextRun: DATE_2020,
revision: 0,
};

describe('rewriteRule', () => {
Expand Down
2 changes: 2 additions & 0 deletions x-pack/plugins/alerting/server/routes/resolve_rule.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ describe('resolveRuleRoute', () => {
},
outcome: 'aliasMatch',
alias_target_id: '2',
revision: 0,
};

const resolveResult: AsApiContract<ResolvedSanitizedRule<{ bar: boolean }>> = {
Expand All @@ -88,6 +89,7 @@ describe('resolveRuleRoute', () => {
created_at: mockedRule.createdAt,
updated_at: mockedRule.updatedAt,
id: mockedRule.id,
revision: mockedRule.revision,
execution_status: {
status: mockedRule.executionStatus.status,
last_execution_date: mockedRule.executionStatus.lastExecutionDate,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { UpdateOptions } from '..';
import { mockedDateString } from '../tests/lib';
import { incrementRevision } from './increment_revision';
import { SavedObject } from '@kbn/core/server';
import { RawRule, RuleTypeParams } from '../../types';

describe('incrementRevision', () => {
const currentRule: SavedObject<RawRule> = {
id: '1',
type: 'alert',
attributes: {
enabled: true,
name: 'rule-name',
tags: ['tag-1', 'tag-2'],
alertTypeId: '123',
consumer: 'rule-consumer',
legacyId: null,
schedule: { interval: '1s' },
actions: [],
params: {},
createdBy: null,
updatedBy: null,
createdAt: mockedDateString,
updatedAt: mockedDateString,
apiKey: null,
apiKeyOwner: null,
throttle: null,
notifyWhen: null,
muteAll: false,
mutedInstanceIds: [],
executionStatus: {
status: 'unknown',
lastExecutionDate: '2020-08-20T19:23:38Z',
error: null,
warning: null,
},
revision: 0,
},
references: [],
};

const updateOptions: UpdateOptions<RuleTypeParams> = {
id: '1',
data: {
schedule: {
interval: '1m',
},
name: 'abc',
tags: ['foo'],
params: {
bar: true,
risk_score: 40,
severity: 'low',
},
throttle: null,
notifyWhen: 'onActiveAlert',
actions: [],
},
};
const updatedParams: RuleTypeParams = { bar: true, risk_score: 40, severity: 'low' };

it('should return the current revision if no attrs or params are updated', () => {
// @ts-expect-error
expect(incrementRevision(currentRule, { data: {} }, {})).toBe(0);
});

it('should increment the revision if a root level attr is updated', () => {
expect(incrementRevision(currentRule, updateOptions, {})).toBe(1);
});

it('should increment the revision if a rule param is updated', () => {
// @ts-expect-error
expect(incrementRevision(currentRule, { data: {} }, updatedParams)).toBe(1);
});

it('should not increment the revision if an excluded attr is updated', () => {
// @ts-expect-error
expect(incrementRevision(currentRule, { data: { activeSnoozes: 'excludedValue' } }, {})).toBe(
0
);
});

it('should not increment the revision if an excluded param is updated', () => {
expect(
incrementRevision(
currentRule,
// @ts-expect-error
{ data: {} },
{ isSnoozedUntil: '1970-01-02T00:00:00.000Z' }
)
).toBe(0);
});
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { SavedObject } from '@kbn/core/server';
import { get, isEqual } from 'lodash';
import { RawRule, RuleTypeParams } from '../../types';
import { fieldsToExcludeFromRevisionUpdates, UpdateOptions } from '..';

export function incrementRevision<Params extends RuleTypeParams>(
currentRule: SavedObject<RawRule>,
{ data }: UpdateOptions<Params>,
updatedParams: RuleTypeParams
): number {
// Diff root level attrs
for (const [field, value] of Object.entries(data).filter(([key]) => key !== 'params')) {
if (
!fieldsToExcludeFromRevisionUpdates.has(field) &&
!isEqual(value, get(currentRule.attributes, field))
) {
return currentRule.attributes.revision + 1;
}
}

// Diff rule params
for (const [field, value] of Object.entries(updatedParams)) {
if (
!fieldsToExcludeFromRevisionUpdates.has(field) &&
!isEqual(value, get(currentRule.attributes.params, field))
) {
return currentRule.attributes.revision + 1;
}
}
return currentRule.attributes.revision;
}
1 change: 1 addition & 0 deletions x-pack/plugins/alerting/server/rules_client/lib/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,4 @@ export { scheduleTask } from './schedule_task';
export { createNewAPIKeySet } from './create_new_api_key_set';
export { recoverRuleAlerts } from './recover_rule_alerts';
export { addUuid } from './add_uuid';
export { incrementRevision } from './increment_revision';
Loading

0 comments on commit 2c71631

Please sign in to comment.