Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RAC][Security Solution] Add base Security Rule Type #105096

Merged
merged 100 commits into from
Aug 3, 2021
Merged
Show file tree
Hide file tree
Changes from 87 commits
Commits
Show all changes
100 commits
Select commit Hold shift + click to select a range
e2467ac
injects bulkCreate and wrapHits to individual rule executors
marshallmain May 11, 2021
ae41613
WIP create_security_rule_type_factory based on Marshall's work in #d3…
ecezalp Jun 4, 2021
83a2f0f
removes ruleStatusService from old rule executors, fixes executor uni…
ecezalp Jun 7, 2021
5fd3f60
fixes rebase
ecezalp Jun 30, 2021
b95b6b6
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 6, 2021
2c0236e
Rename reference_rules to rule_types
madirey Jul 6, 2021
637246c
Fix type errors
madirey Jul 6, 2021
c336531
Fix type errors in base security rule factory
madirey Jul 7, 2021
0645902
Additional improvements to types and interfaces
madirey Jul 7, 2021
72032d7
More type alignment
madirey Jul 8, 2021
7ea0928
Fix remaining type errors in query rule
madirey Jul 8, 2021
c92dbe6
Add validation / inject lists plugin
madirey Jul 9, 2021
849a428
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 9, 2021
6fb0fc8
Formatting
madirey Jul 9, 2021
b302674
Improvements to typing
madirey Jul 9, 2021
3cd9ee7
Static typing on executors
madirey Jul 9, 2021
44eb2de
cleanup
madirey Jul 11, 2021
b4b7b56
Hook up params for query/threshold rules... includes exceptionsList a…
madirey Jul 12, 2021
14b0b6b
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 12, 2021
f9922fa
Scaffolding for wrapHits and bulkCreate
madirey Jul 12, 2021
78601da
Add error handling / status reporting
madirey Jul 12, 2021
47f0f9c
Fixup alert type state
madirey Jul 13, 2021
5450d1b
Begin threshold
madirey Jul 13, 2021
a6a9efc
Begin work on threshold state
madirey Jul 13, 2021
a22c321
Organize rule types
madirey Jul 13, 2021
dc4f5bf
Export base security rule types
madirey Jul 13, 2021
b8185f2
Fixup lifecycle static typing
madirey Jul 13, 2021
a8c0b4e
WrapHits / bulk changes
madirey Jul 14, 2021
e25b32b
Field mappings (partial)
madirey Jul 14, 2021
a7771bd
whoops
madirey Jul 14, 2021
3daa823
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 15, 2021
87aff9c
Remove redundant params
madirey Jul 15, 2021
5f64f3b
More flexibile implementation of bulkCreateFactory
madirey Jul 15, 2021
aa60279
Add mappings
madirey Jul 16, 2021
bc50b42
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 19, 2021
cf13ad6
Finish query rule
madirey Jul 21, 2021
1a3393c
Revert "Remove redundant params"
madirey Jul 21, 2021
0570a37
Revert "whoops"
madirey Jul 21, 2021
ccbc66a
Fixup return types
madirey Jul 21, 2021
978984e
Use alertWithPersistence
madirey Jul 21, 2021
d6d5025
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 21, 2021
f384e49
Fix import
madirey Jul 21, 2021
e7ee2a7
End-to-end rule mostly working
madirey Jul 22, 2021
edc4578
Fix bulkCreate
madirey Jul 22, 2021
3f1dfe0
Bug fixes
madirey Jul 24, 2021
f44c2cf
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 24, 2021
66fdd74
Bug fixes and mapping changes
madirey Jul 24, 2021
058e576
Fix indexing
madirey Jul 25, 2021
f8ed661
cleanup
madirey Jul 25, 2021
7be4690
Fix type errors
madirey Jul 26, 2021
677659c
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 26, 2021
484cc00
Test fixes
madirey Jul 26, 2021
69b0007
Fix query tests
madirey Jul 27, 2021
c5eca53
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 27, 2021
15d671e
cleanup / rename kibana.rac to kibana
madirey Jul 27, 2021
95d70dc
Remove eql/threshold (for now)
madirey Jul 27, 2021
7617a3b
Move technical fields to package
madirey Jul 27, 2021
6ee2f85
Add indexAlias and buildRuleMessageFactory
madirey Jul 28, 2021
794f029
imports
madirey Jul 28, 2021
6b9cf53
type errors
madirey Jul 28, 2021
0328fe4
Change 'kibana.rac.*' to 'kibana.*'
madirey Jul 28, 2021
5bfb66c
Fix lifecycle tests
madirey Jul 28, 2021
d9b2d4c
Single alert instance
madirey Jul 28, 2021
774277e
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 28, 2021
9b16d22
fix import
madirey Jul 28, 2021
cb46500
Fix type error
madirey Jul 28, 2021
164f549
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 28, 2021
07a2f7e
Fix more type errors
madirey Jul 28, 2021
4bafdda
Fix query rule type test
madirey Jul 28, 2021
4c83aa3
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 28, 2021
d115a95
revert to previous ts-expect-error
madirey Jul 28, 2021
aeb052f
type errors again
madirey Jul 28, 2021
b93ed2c
types / linting
madirey Jul 28, 2021
44ce886
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 28, 2021
3284812
General readability improvements
madirey Jul 29, 2021
f7dfd3f
Add invariant function from Dmitrii's branch
madirey Jul 29, 2021
2ccc6c2
Use invariant and constants
madirey Jul 29, 2021
8c12651
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 29, 2021
7af0175
Improvements to field mappings
madirey Jul 29, 2021
aa1a49f
More test failure fixes
madirey Jul 29, 2021
1a65e63
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 29, 2021
7646f5e
Add refresh param for bulk create
madirey Jul 29, 2021
3175fed
Update more field refs
madirey Jul 29, 2021
dcae14d
Actually use refresh param
madirey Jul 29, 2021
5e3e3bb
cleanup
madirey Jul 29, 2021
eb1b900
test fixes
madirey Jul 30, 2021
c44bd32
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 30, 2021
d76eecd
changes to rule creation script
madirey Jul 30, 2021
01529fc
Fix created signals count
madirey Jul 30, 2021
668dddc
Use ruleId
madirey Jul 30, 2021
8d19387
Updates to bulk indexing
madirey Jul 30, 2021
44d0a2a
Mapping updates
madirey Jul 30, 2021
091b64a
Cannot use 'strict' for dynamic setting
madirey Jul 30, 2021
7f519e8
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Jul 30, 2021
3ef3de9
Merge branch 'master' into security-rule-type
kibanamachine Aug 2, 2021
cfbdd20
Merge branch 'master' into security-rule-type
kibanamachine Aug 2, 2021
75260cf
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Aug 3, 2021
a4b37b0
Merge branch 'security-rule-type' of github.com:madirey/kibana into s…
madirey Aug 3, 2021
9c02627
Merge branch 'master' of github.com:elastic/kibana into security-rule…
madirey Aug 3, 2021
1882710
Fix type errors from master
madirey Aug 3, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions api_docs/observability.json

Large diffs are not rendered by default.

12 changes: 6 additions & 6 deletions api_docs/rule_registry.json

Large diffs are not rendered by default.

199 changes: 156 additions & 43 deletions packages/kbn-rule-data-utils/src/technical_field_names.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,79 +8,192 @@

import { ValuesType } from 'utility-types';

const ALERT_NAMESPACE = 'kibana.rac.alert' as const;
const KIBANA_NAMESPACE = 'kibana' as const;

const TIMESTAMP = '@timestamp' as const;
const EVENT_KIND = 'event.kind' as const;
const ALERT_NAMESPACE = `${KIBANA_NAMESPACE}.alert` as const;
const ALERT_RULE_NAMESPACE = `${ALERT_NAMESPACE}.rule` as const;

const CONSUMERS = `${KIBANA_NAMESPACE}.consumers` as const;
const ECS_VERSION = 'ecs.version' as const;
const EVENT_ACTION = 'event.action' as const;
const RULE_UUID = 'rule.uuid' as const;
const EVENT_KIND = 'event.kind' as const;
const RULE_CATEGORY = 'rule.category' as const;
const RULE_CONSUMERS = 'rule.consumers' as const;
const RULE_ID = 'rule.id' as const;
const RULE_NAME = 'rule.name' as const;
const RULE_CATEGORY = 'rule.category' as const;
const RULE_UUID = 'rule.uuid' as const;
const SPACE_IDS = `${KIBANA_NAMESPACE}.space_ids` as const;
const TAGS = 'tags' as const;
const PRODUCER = `${ALERT_NAMESPACE}.producer` as const;
const OWNER = `${ALERT_NAMESPACE}.owner` as const;
const ALERT_ID = `${ALERT_NAMESPACE}.id` as const;
const ALERT_UUID = `${ALERT_NAMESPACE}.uuid` as const;
const ALERT_START = `${ALERT_NAMESPACE}.start` as const;
const ALERT_END = `${ALERT_NAMESPACE}.end` as const;
const TIMESTAMP = '@timestamp' as const;
const VERSION = `${KIBANA_NAMESPACE}.version` as const;

const ALERT_ACTION_GROUP = `${ALERT_NAMESPACE}.action_group` as const;
const ALERT_DURATION = `${ALERT_NAMESPACE}.duration.us` as const;
const ALERT_SEVERITY_LEVEL = `${ALERT_NAMESPACE}.severity.level` as const;
const ALERT_SEVERITY_VALUE = `${ALERT_NAMESPACE}.severity.value` as const;
const ALERT_STATUS = `${ALERT_NAMESPACE}.status` as const;
const SPACE_IDS = 'kibana.space_ids' as const;
const ALERT_END = `${ALERT_NAMESPACE}.end` as const;
const ALERT_EVALUATION_THRESHOLD = `${ALERT_NAMESPACE}.evaluation.threshold` as const;
const ALERT_EVALUATION_VALUE = `${ALERT_NAMESPACE}.evaluation.value` as const;
const ALERT_ID = `${ALERT_NAMESPACE}.id` as const;
const ALERT_OWNER = `${ALERT_NAMESPACE}.owner` as const;
const ALERT_PRODUCER = `${ALERT_NAMESPACE}.producer` as const;
const ALERT_REASON = `${ALERT_NAMESPACE}.reason` as const;
const ALERT_RISK_SCORE = `${ALERT_NAMESPACE}.risk_score` as const;
const ALERT_SEVERITY = `${ALERT_NAMESPACE}.severity` as const;
const ALERT_SEVERITY_LEVEL = `${ALERT_NAMESPACE}.severity.level` as const;
const ALERT_SEVERITY_VALUE = `${ALERT_NAMESPACE}.severity.value` as const;
const ALERT_START = `${ALERT_NAMESPACE}.start` as const;
const ALERT_STATUS = `${ALERT_NAMESPACE}.status` as const;
const ALERT_SYSTEM_STATUS = `${ALERT_NAMESPACE}.system_status` as const;
const ALERT_UUID = `${ALERT_NAMESPACE}.uuid` as const;
const ALERT_WORKFLOW_REASON = `${ALERT_NAMESPACE}.workflow_reason` as const;
const ALERT_WORKFLOW_STATUS = `${ALERT_NAMESPACE}.workflow_status` as const;
const ALERT_WORKFLOW_USER = `${ALERT_NAMESPACE}.workflow_user` as const;

const ALERT_RULE_AUTHOR = `${ALERT_RULE_NAMESPACE}.author` as const;
const ALERT_RULE_CONSUMERS = `${ALERT_RULE_NAMESPACE}.consumers` as const;
const ALERT_RULE_CREATED_AT = `${ALERT_RULE_NAMESPACE}.created_at` as const;
const ALERT_RULE_CREATED_BY = `${ALERT_RULE_NAMESPACE}.created_by` as const;
const ALERT_RULE_DESCRIPTION = `${ALERT_RULE_NAMESPACE}.description` as const;
const ALERT_RULE_ENABLED = `${ALERT_RULE_NAMESPACE}.enabled` as const;
const ALERT_RULE_FROM = `${ALERT_RULE_NAMESPACE}.from` as const;
const ALERT_RULE_ID = `${ALERT_RULE_NAMESPACE}.id` as const;
const ALERT_RULE_INTERVAL = `${ALERT_RULE_NAMESPACE}.interval` as const;
const ALERT_RULE_LICENSE = `${ALERT_RULE_NAMESPACE}.license` as const;
const ALERT_RULE_NAME = `${ALERT_RULE_NAMESPACE}.name` as const;
const ALERT_RULE_NOTE = `${ALERT_RULE_NAMESPACE}.note` as const;
const ALERT_RULE_REFERENCES = `${ALERT_RULE_NAMESPACE}.references` as const;
const ALERT_RULE_RISK_SCORE = `${ALERT_RULE_NAMESPACE}.risk_score` as const;
const ALERT_RULE_RISK_SCORE_MAPPING = `${ALERT_RULE_NAMESPACE}.risk_score_mapping` as const;
const ALERT_RULE_RULE_ID = `${ALERT_RULE_NAMESPACE}.rule_id` as const;
const ALERT_RULE_RULE_NAME_OVERRIDE = `${ALERT_RULE_NAMESPACE}.rule_name_override` as const;
const ALERT_RULE_SEVERITY = `${ALERT_RULE_NAMESPACE}.severity` as const;
const ALERT_RULE_SEVERITY_MAPPING = `${ALERT_RULE_NAMESPACE}.severity_mapping` as const;
const ALERT_RULE_TAGS = `${ALERT_RULE_NAMESPACE}.tags` as const;
const ALERT_RULE_TO = `${ALERT_RULE_NAMESPACE}.to` as const;
const ALERT_RULE_TYPE = `${ALERT_RULE_NAMESPACE}.type` as const;
const ALERT_RULE_UPDATED_AT = `${ALERT_RULE_NAMESPACE}.updated_at` as const;
const ALERT_RULE_UPDATED_BY = `${ALERT_RULE_NAMESPACE}.updated_by` as const;
const ALERT_RULE_VERSION = `${ALERT_RULE_NAMESPACE}.version` as const;

const fields = {
TIMESTAMP,
CONSUMERS,
ECS_VERSION,
EVENT_KIND,
EVENT_ACTION,
RULE_UUID,
RULE_CATEGORY,
RULE_CONSUMERS,
RULE_ID,
RULE_NAME,
RULE_CATEGORY,
RULE_UUID,
TAGS,
PRODUCER,
OWNER,
TIMESTAMP,
ALERT_ACTION_GROUP,
ALERT_DURATION,
ALERT_END,
ALERT_EVALUATION_THRESHOLD,
ALERT_EVALUATION_VALUE,
ALERT_ID,
ALERT_UUID,
ALERT_OWNER,
ALERT_PRODUCER,
ALERT_REASON,
ALERT_RISK_SCORE,
ALERT_RULE_AUTHOR,
ALERT_RULE_CONSUMERS,
ALERT_RULE_CREATED_AT,
ALERT_RULE_CREATED_BY,
ALERT_RULE_DESCRIPTION,
ALERT_RULE_ENABLED,
ALERT_RULE_FROM,
ALERT_RULE_ID,
ALERT_RULE_INTERVAL,
ALERT_RULE_LICENSE,
ALERT_RULE_NAME,
ALERT_RULE_NOTE,
ALERT_RULE_REFERENCES,
ALERT_RULE_RISK_SCORE,
ALERT_RULE_RISK_SCORE_MAPPING,
ALERT_RULE_RULE_ID,
ALERT_RULE_RULE_NAME_OVERRIDE,
ALERT_RULE_SEVERITY,
ALERT_RULE_SEVERITY_MAPPING,
ALERT_RULE_TAGS,
ALERT_RULE_TO,
ALERT_RULE_TYPE,
ALERT_RULE_UPDATED_AT,
ALERT_RULE_UPDATED_BY,
ALERT_RULE_VERSION,
ALERT_START,
ALERT_END,
ALERT_DURATION,
ALERT_SEVERITY,
ALERT_SEVERITY_LEVEL,
ALERT_SEVERITY_VALUE,
ALERT_STATUS,
ALERT_EVALUATION_THRESHOLD,
ALERT_EVALUATION_VALUE,
ALERT_REASON,
ALERT_SYSTEM_STATUS,
ALERT_UUID,
ALERT_WORKFLOW_REASON,
ALERT_WORKFLOW_STATUS,
ALERT_WORKFLOW_USER,
SPACE_IDS,
VERSION,
};

export {
TIMESTAMP,
EVENT_KIND,
EVENT_ACTION,
RULE_UUID,
RULE_ID,
RULE_NAME,
RULE_CATEGORY,
TAGS,
PRODUCER,
OWNER,
ALERT_ID,
ALERT_UUID,
ALERT_START,
ALERT_END,
ALERT_ACTION_GROUP,
ALERT_DURATION,
ALERT_SEVERITY_LEVEL,
ALERT_SEVERITY_VALUE,
ALERT_STATUS,
ALERT_END,
ALERT_EVALUATION_THRESHOLD,
ALERT_EVALUATION_VALUE,
ALERT_ID,
ALERT_OWNER,
ALERT_PRODUCER,
ALERT_REASON,
ALERT_RISK_SCORE,
ALERT_STATUS,
ALERT_WORKFLOW_REASON,
ALERT_WORKFLOW_STATUS,
ALERT_WORKFLOW_USER,
ALERT_RULE_AUTHOR,
ALERT_RULE_CONSUMERS,
ALERT_RULE_CREATED_AT,
ALERT_RULE_CREATED_BY,
ALERT_RULE_DESCRIPTION,
ALERT_RULE_ENABLED,
ALERT_RULE_FROM,
ALERT_RULE_ID,
ALERT_RULE_INTERVAL,
ALERT_RULE_LICENSE,
ALERT_RULE_NAME,
ALERT_RULE_NOTE,
ALERT_RULE_REFERENCES,
ALERT_RULE_RISK_SCORE,
ALERT_RULE_RISK_SCORE_MAPPING,
ALERT_RULE_RULE_ID,
ALERT_RULE_RULE_NAME_OVERRIDE,
ALERT_RULE_SEVERITY_MAPPING,
ALERT_RULE_TAGS,
ALERT_RULE_TO,
ALERT_RULE_TYPE,
ALERT_RULE_UPDATED_AT,
ALERT_RULE_UPDATED_BY,
ALERT_RULE_VERSION,
ALERT_RULE_SEVERITY,
ALERT_SEVERITY,
ALERT_SEVERITY_LEVEL,
ALERT_SEVERITY_VALUE,
ALERT_START,
ALERT_SYSTEM_STATUS,
ALERT_UUID,
CONSUMERS,
ECS_VERSION,
EVENT_ACTION,
EVENT_KIND,
RULE_CATEGORY,
RULE_CONSUMERS,
RULE_ID,
RULE_NAME,
RULE_UUID,
TAGS,
TIMESTAMP,
SPACE_IDS,
VERSION,
};

export type TechnicalRuleDataFieldName = ValuesType<typeof fields>;
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,17 @@
* 2.0.
*/

import { ALERT_SEVERITY_LEVEL } from '@kbn/rule-data-utils/target/technical_field_names';
import {
ALERT_DURATION,
ALERT_EVALUATION_THRESHOLD,
ALERT_EVALUATION_VALUE,
ALERT_ID,
ALERT_PRODUCER,
ALERT_SEVERITY_LEVEL,
ALERT_START,
ALERT_STATUS,
ALERT_UUID,
} from '@kbn/rule-data-utils';
import { ValuesType } from 'utility-types';
import { EuiTheme } from '../../../../../../../../src/plugins/kibana_react/common';
import { ObservabilityRuleTypeRegistry } from '../../../../../../observability/public';
Expand All @@ -23,28 +33,26 @@ const theme = ({
} as unknown) as EuiTheme;
const alert: Alert = {
'rule.id': ['apm.transaction_duration'],
'kibana.rac.alert.evaluation.value': [2057657.39],
[ALERT_EVALUATION_VALUE]: [2057657.39],
ecezalp marked this conversation as resolved.
Show resolved Hide resolved
'service.name': ['frontend-rum'],
'rule.name': ['Latency threshold | frontend-rum'],
'kibana.rac.alert.duration.us': [62879000],
'kibana.rac.alert.status': ['open'],
[ALERT_DURATION]: [62879000],
[ALERT_STATUS]: ['open'],
tags: ['apm', 'service.name:frontend-rum'],
'transaction.type': ['page-load'],
'kibana.rac.alert.producer': ['apm'],
'kibana.rac.alert.uuid': ['af2ae371-df79-4fca-b0eb-a2dbd9478180'],
[ALERT_PRODUCER]: ['apm'],
[ALERT_UUID]: ['af2ae371-df79-4fca-b0eb-a2dbd9478180'],
'rule.uuid': ['82e0ee40-c2f4-11eb-9a42-a9da66a1722f'],
'event.action': ['active'],
'@timestamp': ['2021-06-01T16:16:05.183Z'],
'kibana.rac.alert.id': ['apm.transaction_duration_All'],
[ALERT_ID]: ['apm.transaction_duration_All'],
'processor.event': ['transaction'],
'kibana.rac.alert.evaluation.threshold': [500000],
'kibana.rac.alert.start': ['2021-06-01T16:15:02.304Z'],
[ALERT_EVALUATION_THRESHOLD]: [500000],
[ALERT_START]: ['2021-06-01T16:15:02.304Z'],
'event.kind': ['state'],
'rule.category': ['Latency threshold'],
};
const chartStartTime = new Date(
alert['kibana.rac.alert.start']![0] as string
).getTime();
const chartStartTime = new Date(alert[ALERT_START]![0] as string).getTime();
ecezalp marked this conversation as resolved.
Show resolved Hide resolved
const getFormatter: ObservabilityRuleTypeRegistry['getFormatter'] = () => () => ({
link: '/',
reason: 'a good reason',
Expand Down
Loading