Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution][Detections][Threshold Rules] Threshold multiple aggregations with cardinality #90826

Merged
merged 50 commits into from
Feb 18, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
a4139ce
Remove unnecessary spreads
madirey Jan 14, 2021
b73b34a
Layout, round 1
madirey Jan 14, 2021
026a6d7
Merge branch 'master' of github.com:elastic/kibana into threshold-mul…
madirey Jan 25, 2021
8c5c889
Merge branch 'master' of github.com:elastic/kibana into threshold-mul…
madirey Jan 27, 2021
ed5f9b0
Revert "Layout, round 1"
madirey Jan 28, 2021
b2f31ba
Make threshold field an array
madirey Jan 28, 2021
0842e84
Add cardinality fields
madirey Jan 28, 2021
02c3b2c
Fix validation schema
madirey Feb 1, 2021
48f6545
Query for multi-aggs
madirey Feb 1, 2021
e435241
Merge branch 'master' of github.com:elastic/kibana into threshold-mul…
madirey Feb 1, 2021
70e4f50
Finish multi-agg aggregation
madirey Feb 1, 2021
cf7ef93
Translate to multi-agg buckets
madirey Feb 7, 2021
1ebfed0
Fix existing tests and add new test skeletons
madirey Feb 7, 2021
0786757
merge master, fix conflicts
madirey Feb 7, 2021
12a98bb
clean up
madirey Feb 9, 2021
6cba63a
Fix types
marshallmain Feb 11, 2021
78e77bc
Fix threshold_result data structure
madirey Feb 11, 2021
465c5a4
previous signals filter
madirey Feb 11, 2021
18d5363
Fix previous signal detection
madirey Feb 12, 2021
41a5ddb
Finish previous signal parsing
madirey Feb 12, 2021
19ed253
tying up loose ends
madirey Feb 14, 2021
733347c
merge master, fix conflicts
madirey Feb 15, 2021
c7eea31
Merge branch 'master' of github.com:elastic/kibana into threshold-mul…
madirey Feb 15, 2021
319e9db
Fix timeline view for multi-agg threshold signals
madirey Feb 15, 2021
e2a7d40
Fix build_bulk_body tests
madirey Feb 15, 2021
c6abdf5
test fixes
madirey Feb 16, 2021
741c75e
Add test for threshold bucket filters
madirey Feb 16, 2021
b277b04
Address comments
madirey Feb 16, 2021
8d1e922
Fixing schema errors
madirey Feb 16, 2021
6b8c8ed
Remove unnecessary comment
madirey Feb 16, 2021
a8dc733
Fix tests
madirey Feb 16, 2021
6fd0836
Fix types
madirey Feb 16, 2021
900ead0
Merge branch 'master' of github.com:elastic/kibana into threshold-mul…
madirey Feb 16, 2021
37956ca
linting
madirey Feb 17, 2021
7956953
linting
madirey Feb 17, 2021
af5ed84
Fixes
madirey Feb 17, 2021
ee103c1
Handle pre-7.12 threshold format in timeline view
madirey Feb 17, 2021
2094d58
missing null check
madirey Feb 17, 2021
bed1faf
adding in follow-up pr
madirey Feb 17, 2021
e979d12
Handle pre-7.12 filters
madirey Feb 17, 2021
9dab01e
Merge branch 'master' of github.com:elastic/kibana into threshold-mul…
madirey Feb 17, 2021
3edc7f2
unnecessary change
madirey Feb 17, 2021
13821bf
Revert "unnecessary change"
madirey Feb 17, 2021
f88cf66
linting
madirey Feb 17, 2021
3e09b24
Fix rule schemas
madirey Feb 17, 2021
6eafa8d
Merge branch 'master' of github.com:elastic/kibana into threshold-mul…
madirey Feb 17, 2021
db6dfa5
Fix tests
madirey Feb 17, 2021
9ba09b6
Merge branch 'master' of github.com:elastic/kibana into threshold-mul…
madirey Feb 18, 2021
b6fd98b
merge master, fix conflicts
madirey Feb 18, 2021
5c503fc
more fixing conflicts
madirey Feb 18, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion x-pack/plugins/osquery/common/ecs/rule/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ export interface RuleEcs {
tags?: string[];
threat?: unknown;
threshold?: {
field: string;
field: string | string[];
value: number;
};
type?: string[];
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -459,12 +459,21 @@ export type Threats = t.TypeOf<typeof threats>;
export const threatsOrUndefined = t.union([threats, t.undefined]);
export type ThreatsOrUndefined = t.TypeOf<typeof threatsOrUndefined>;

export const threshold = t.exact(
t.type({
field: t.string,
value: PositiveIntegerGreaterThanZero,
})
);
export const threshold = t.intersection([
t.exact(
t.type({
field: t.union([t.string, t.array(t.string)]),
value: PositiveIntegerGreaterThanZero,
})
),
t.exact(
t.partial({
cardinality_field: t.union([t.string, t.array(t.string), t.undefined, t.null]),
cardinality_value: t.union([PositiveInteger, t.undefined, t.null]), // TODO: cardinality_value should be set if cardinality_field is set
})
),
]);
// TODO: codec to transform threshold field string to string[] ?
export type Threshold = t.TypeOf<typeof threshold>;

export const thresholdOrUndefined = t.union([threshold, t.undefined]);
Expand Down
5 changes: 1 addition & 4 deletions x-pack/plugins/security_solution/common/ecs/rule/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,7 @@ export interface RuleEcs {
severity?: string[];
tags?: string[];
threat?: unknown;
threshold?: {
field: string;
value: number;
};
threshold?: unknown;
type?: string[];
size?: string[];
to?: string[];
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,5 @@ export interface SignalEcs {
group?: {
id?: string[];
};
threshold_result?: unknown;
}
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,14 @@ export interface MatrixHistogramRequestOptions extends RequestBasicOptions {
timerange: TimerangeInput;
histogramType: MatrixHistogramType;
stackByField: string;
threshold?: { field: string | undefined; value: number } | undefined;
threshold?:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like a few changes similar to this one were made in this PR, would it be worth pulling this out into its own type shared between the updated code? not a blocker, but might be a nice follow up.

| {
field: string | string[] | undefined;
value: number;
cardinality_field?: string | undefined;
cardinality_value?: number | undefined;
}
| undefined;
inspect?: Maybe<Inspect>;
isPtrIncluded?: boolean;
}
Expand Down
Loading