Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Resolver] Refactoring panel view #77928

Merged
merged 32 commits into from
Sep 23, 2020
Merged
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
fc9577e
remove dead code
Sep 18, 2020
62deec4
move panel breadcrumbs to a module
Sep 18, 2020
3841c7b
rename breadcrumb to make room for the new one
Sep 18, 2020
ef56595
move descriptiveName into the view
Sep 18, 2020
3da7c42
rename breadcrumbs again
Sep 18, 2020
29da625
fix intl issue
Sep 18, 2020
28606f2
remove unused translations
Sep 18, 2020
6c6387f
remove copy paste code
Sep 18, 2020
3c66a9d
Remove duplicate code
Sep 18, 2020
9547a4d
remove duplicate styles for LimitWarnings components
Sep 18, 2020
d229fb2
refactor node list
Sep 18, 2020
e7a995c
Add test to make sure Analyzed Event shows up on the graph
Sep 18, 2020
9c0fbfc
cleaning up event detail
Sep 18, 2020
bc06056
refactoring event detail
Sep 18, 2020
76cd15b
fixed the links on the node events panel
Sep 18, 2020
fcafdfe
remove dead translations
Sep 21, 2020
0af4790
add failing test to check that we can naviagte to nodeEvents view. fi…
Sep 21, 2020
83e3f0c
WIP
Sep 21, 2020
e5e8a9f
remove some unsafe resolver event methods
Sep 21, 2020
23434f8
how to fix???
Sep 21, 2020
9558676
upcoming changes to REVERT
Sep 21, 2020
55caf5b
Revert "upcoming changes to REVERT"
Sep 21, 2020
7e349d5
mostly working
Sep 21, 2020
76addca
mostly good
Sep 21, 2020
41711b7
tests for the panel. fix data state type. refactor node events. fetch…
Sep 22, 2020
c0ab9ed
remove dead code
Sep 22, 2020
5960e4d
went too far
Sep 22, 2020
244e5e9
Revert "went too far"
Sep 22, 2020
29392ee
is it working?
Sep 22, 2020
ad81c43
fix type check
Sep 22, 2020
2681c0f
remove TODOs. test deep object entries
Sep 22, 2020
3fbc1b0
fix the mock used in the test plugin. remove dead code. refactor the …
Sep 22, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -4,59 +4,15 @@
* you may not use this file except in compliance with the Elastic License.
*/
import { EndpointDocGenerator } from '../generate_data';
import { descriptiveName, isProcessRunning } from './event';
import { ResolverEvent, SafeResolverEvent } from '../types';
import { isProcessRunning } from './event';
import { SafeResolverEvent } from '../types';

describe('Generated documents', () => {
let generator: EndpointDocGenerator;
beforeEach(() => {
generator = new EndpointDocGenerator('seed');
});

describe('Event descriptive names', () => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved these tests to the view layer

it('returns the right name for a registry event', () => {
const extensions = { registry: { key: `HKLM/Windows/Software/abc` } };
const event = generator.generateEvent({ eventCategory: 'registry', extensions });
// casting to ResolverEvent here because the `descriptiveName` function is used by the frontend is still relies
// on the unsafe ResolverEvent type. Once it's switched over to the safe version we can remove this cast.
expect(descriptiveName(event as ResolverEvent)).toEqual({
subject: `HKLM/Windows/Software/abc`,
});
});

it('returns the right name for a network event', () => {
const randomIP = `${generator.randomIP()}`;
const extensions = { network: { direction: 'outbound', forwarded_ip: randomIP } };
const event = generator.generateEvent({ eventCategory: 'network', extensions });
// casting to ResolverEvent here because the `descriptiveName` function is used by the frontend is still relies
// on the unsafe ResolverEvent type. Once it's switched over to the safe version we can remove this cast.
expect(descriptiveName(event as ResolverEvent)).toEqual({
subject: `${randomIP}`,
descriptor: 'outbound',
});
});

it('returns the right name for a file event', () => {
const extensions = { file: { path: 'C:\\My Documents\\business\\January\\processName' } };
const event = generator.generateEvent({ eventCategory: 'file', extensions });
// casting to ResolverEvent here because the `descriptiveName` function is used by the frontend is still relies
// on the unsafe ResolverEvent type. Once it's switched over to the safe version we can remove this cast.
expect(descriptiveName(event as ResolverEvent)).toEqual({
subject: 'C:\\My Documents\\business\\January\\processName',
});
});

it('returns the right name for a dns event', () => {
const extensions = { dns: { question: { name: `${generator.randomIP()}` } } };
const event = generator.generateEvent({ eventCategory: 'dns', extensions });
// casting to ResolverEvent here because the `descriptiveName` function is used by the frontend is still relies
// on the unsafe ResolverEvent type. Once it's switched over to the safe version we can remove this cast.
expect(descriptiveName(event as ResolverEvent)).toEqual({
subject: extensions.dns.question.name,
});
});
});

describe('Process running events', () => {
it('is a running event when event.type is a string', () => {
const event: SafeResolverEvent = generator.generateEvent({
180 changes: 78 additions & 102 deletions x-pack/plugins/security_solution/common/endpoint/models/event.ts
Original file line number Diff line number Diff line change
@@ -104,18 +104,73 @@ export function timestampAsDateSafeVersion(event: TimestampFields): Date | undef
}
}

export function eventTimestamp(event: ResolverEvent): string | undefined | number {
return event['@timestamp'];
export function eventTimestamp(event: SafeResolverEvent): string | undefined | number {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Converted this (among many things) to use safe types.

return firstNonNullValue(event['@timestamp']);
}

export function eventName(event: ResolverEvent): string {
/**
* Find the name of the related process.
*/
export function processName(event: ResolverEvent): string {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Named this to processName since I think that is more accurate.

if (isLegacyEvent(event)) {
return event.endgame.process_name ? event.endgame.process_name : '';
} else {
return event.process.name;
}
}

/**
* First non-null value in the `user.name` field.
*/
export function userName(event: SafeResolverEvent): string | undefined {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The view was reading this directly.

if (isLegacyEventSafeVersion(event)) {
return undefined;
} else {
return firstNonNullValue(event.user?.name);
}
}

/**
* Returns the process event's parent PID
*/
export function parentPID(event: SafeResolverEvent): number | undefined {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The view was reading this directly.

return firstNonNullValue(
isLegacyEventSafeVersion(event) ? event.endgame.ppid : event.process?.parent?.pid
);
}

/**
* First non-null value for the `process.hash.md5` field.
*/
export function md5HashForProcess(event: SafeResolverEvent): string | undefined {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The view was reading this directly.

return firstNonNullValue(isLegacyEventSafeVersion(event) ? undefined : event.process?.hash?.md5);
}

/**
* First non-null value for the `event.process.args` field.
*/
export function argsForProcess(event: SafeResolverEvent): string | undefined {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The view was reading this directly.

if (isLegacyEventSafeVersion(event)) {
// There is not currently a key for this on Legacy event types
return undefined;
}
return firstNonNullValue(event.process?.args);
}

/**
* First non-null value in the `user.name` field.
*/
export function userDomain(event: SafeResolverEvent): string | undefined {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The view was reading this directly.

if (isLegacyEventSafeVersion(event)) {
return undefined;
} else {
return firstNonNullValue(event.user?.domain);
}
}

/**
* Find the name of the related process.
*/
export function processNameSafeVersion(event: SafeResolverEvent): string | undefined {
if (isLegacyEventSafeVersion(event)) {
return firstNonNullValue(event.endgame.process_name);
@@ -124,11 +179,10 @@ export function processNameSafeVersion(event: SafeResolverEvent): string | undef
}
}

export function eventId(event: ResolverEvent): number | undefined | string {
if (isLegacyEvent(event)) {
return event.endgame.serial_event_id;
}
return event.event.id;
export function eventID(event: SafeResolverEvent): number | undefined | string {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spelling and type safety

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❔ needs a doc comment?

return firstNonNullValue(
isLegacyEventSafeVersion(event) ? event.endgame.serial_event_id : event.event?.id
);
}

/**
@@ -275,18 +329,14 @@ export function ancestryArray(event: AncestryArrayFields): string[] | undefined
/**
* Minimum fields needed from the `SafeResolverEvent` type for the function below to operate correctly.
*/
type GetAncestryArrayFields = AncestryArrayFields & ParentEntityIDFields;
type AncestryFields = AncestryArrayFields & ParentEntityIDFields;

/**
* Returns an array of strings representing the ancestry for a process.
*
* @param event an ES document
*/
export function getAncestryAsArray(event: GetAncestryArrayFields | undefined): string[] {
if (!event) {
return [];
}

export function ancestry(event: AncestryFields): string[] {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed name to match convention. no longer accepts undefined.

const ancestors = ancestryArray(event);
if (ancestors) {
return ancestors;
@@ -300,107 +350,33 @@ export function getAncestryAsArray(event: GetAncestryArrayFields | undefined): s
return [];
}

/**
* @param event The event to get the category for
*/
export function primaryEventCategory(event: ResolverEvent): string | undefined {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed this concept entirely

if (isLegacyEvent(event)) {
const legacyFullType = event.endgame.event_type_full;
if (legacyFullType) {
return legacyFullType;
}
} else {
const eventCategories = event.event.category;
const category = typeof eventCategories === 'string' ? eventCategories : eventCategories[0];

return category;
}
}

/**
* @param event The event to get the full ECS category for
*/
export function allEventCategories(event: ResolverEvent): string | string[] | undefined {
if (isLegacyEvent(event)) {
const legacyFullType = event.endgame.event_type_full;
if (legacyFullType) {
return legacyFullType;
}
} else {
return event.event.category;
}
export function eventCategory(event: SafeResolverEvent): string[] {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This returns all event event categories now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❔ Should we call it event categor_ies_ (plural)? Because it could be many or stick with categor_y_ b/c ECS?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure. Any opinion on that?

Copy link
Contributor

@michaelolo24 michaelolo24 Sep 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plural makes sense, but following the ECS paradigm is probably best imo. Just makes it easier to tie it back

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regardless of the name, the TS type indicates that it returns an array. I think we can defer this decision.

return values(
isLegacyEventSafeVersion(event) ? event.endgame.event_type_full : event.event?.category
);
}

/**
* ECS event type will be things like 'creation', 'deletion', 'access', etc.
* see: https://www.elastic.co/guide/en/ecs/current/ecs-event.html
* @param event The ResolverEvent to get the ecs type for
*/
export function ecsEventType(event: ResolverEvent): Array<string | undefined> {
if (isLegacyEvent(event)) {
return [event.endgame.event_subtype_full];
}
return typeof event.event.type === 'string' ? [event.event.type] : event.event.type;
export function eventType(event: SafeResolverEvent): string[] {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renamed from ecsEventType. made it use safe types.

return values(
isLegacyEventSafeVersion(event) ? event.endgame.event_subtype_full : event.event?.type
);
}

/**
* #Descriptive Names For Related Events:
*
* The following section provides facilities for deriving **Descriptive Names** for ECS-compliant event data.
* There are drawbacks to trying to do this: It *will* require ongoing maintenance. It presents temptations to overarticulate.
* On balance, however, it seems that the benefit of giving the user some form of information they can recognize & scan outweighs the drawbacks.
* event.kind as an array.
*/
type DeepPartial<T> = T extends object ? { [K in keyof T]?: DeepPartial<T[K]> } : T;
/**
* Based on the ECS category of the event, attempt to provide a more descriptive name
* (e.g. the `event.registry.key` for `registry` or the `dns.question.name` for `dns`, etc.).
* This function returns the data in the form of `{subject, descriptor}` where `subject` will
* tend to be the more distinctive term (e.g. 137.213.212.7 for a network event) and the
* `descriptor` can be used to present more useful/meaningful view (e.g. `inbound 137.213.212.7`
* in the example above).
* see: https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html
* @param event The ResolverEvent to get the descriptive name for
* @returns { descriptiveName } An attempt at providing a readable name to the user
*/
export function descriptiveName(event: ResolverEvent): { subject: string; descriptor?: string } {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to a component.

if (isLegacyEvent(event)) {
return { subject: eventName(event) };
}

// To be somewhat defensive, we'll check for the presence of these.
const partialEvent: DeepPartial<ResolverEvent> = event;

/**
* This list of attempts can be expanded/adjusted as the underlying model changes over time:
*/

// Stable, per ECS 1.5: https://www.elastic.co/guide/en/ecs/current/ecs-allowed-values-event-category.html

if (partialEvent.network?.forwarded_ip) {
return {
subject: String(partialEvent.network?.forwarded_ip),
descriptor: String(partialEvent.network?.direction),
};
}

if (partialEvent.file?.path) {
return {
subject: String(partialEvent.file?.path),
};
}

// Extended categories (per ECS 1.5):
const pathOrKey = partialEvent.registry?.path || partialEvent.registry?.key;
if (pathOrKey) {
return {
subject: String(pathOrKey),
};
}

if (partialEvent.dns?.question?.name) {
return { subject: String(partialEvent.dns?.question?.name) };
export function eventKind(event: SafeResolverEvent): string[] {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this was exposed before.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❔ Can kind really be an array? It sorta goes against the grain of what ECS says it's supposed to be used for ( They may warrant different retention, different access control... ). I guess it's safer to treat it that way, but I'm just wondering if there's some kind of misalignment we'd be allowing by that...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My reasoning: ES will accept and possibly return an array and the SIEM is permissive about it.

I can bring up this topic with some people around the org, but I think it's something we should be consistent on within all of Elastic.

if (isLegacyEventSafeVersion(event)) {
return [];
} else {
return values(event.event?.kind);
}

// Fall back on entityId if we can't fish a more descriptive name out.
return { subject: entityId(event) };
}
Original file line number Diff line number Diff line change
@@ -183,7 +183,7 @@ export interface ResolverTree {
relatedEvents: Omit<ResolverRelatedEvents, 'entityID'>;
relatedAlerts: Omit<ResolverRelatedAlerts, 'entityID'>;
ancestry: ResolverAncestry;
lifecycle: ResolverEvent[];
lifecycle: SafeResolverEvent[];
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll see a lot of this. We're using the safe version of this type in most places now. In a follow up PR we could probably finish the task (just in time to introduce an even safer version.)

stats: ResolverNodeStats;
}

@@ -209,7 +209,7 @@ export interface SafeResolverTree {
*/
export interface ResolverLifecycleNode {
entityID: string;
lifecycle: ResolverEvent[];
lifecycle: SafeResolverEvent[];
/**
* stats are only set when the entire tree is being fetched
*/
@@ -263,7 +263,7 @@ export interface SafeResolverAncestry {
*/
export interface ResolverRelatedEvents {
entityID: string;
events: ResolverEvent[];
events: SafeResolverEvent[];
nextEvent: string | null;
}

Original file line number Diff line number Diff line change
@@ -9,7 +9,6 @@ import {
ResolverTree,
ResolverEntityIndex,
} from '../../../../common/endpoint/types';
import { mockEndpointEvent } from '../../mocks/endpoint_event';
import { mockTreeWithNoAncestorsAnd2Children } from '../../mocks/resolver_tree';
import { DataAccessLayer } from '../../types';

@@ -54,13 +53,7 @@ export function noAncestorsTwoChildren(): { dataAccessLayer: DataAccessLayer; me
relatedEvents(entityID: string): Promise<ResolverRelatedEvents> {
return Promise.resolve({
entityID,
events: [
mockEndpointEvent({
entityID,
name: 'event',
timestamp: 0,
}),
],
events: [],
nextEvent: null,
});
},
Original file line number Diff line number Diff line change
@@ -61,7 +61,7 @@ export function noAncestorsTwoChildenInIndexCalledAwesomeIndex(): {
events: [
mockEndpointEvent({
entityID,
name: 'event',
processName: 'event',
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this field was renamed. mockEndpointEvent is no longer process event specific.

timestamp: 0,
}),
],
Original file line number Diff line number Diff line change
@@ -66,7 +66,7 @@ export function noAncestorsTwoChildrenWithRelatedEventsOnOrigin(): {
entityID,
events,
nextEvent: null,
} as ResolverRelatedEvents);
});
},

/**
4 changes: 2 additions & 2 deletions x-pack/plugins/security_solution/public/resolver/index.ts
Original file line number Diff line number Diff line change
@@ -7,7 +7,7 @@ import { Provider } from 'react-redux';
import { ResolverPluginSetup } from './types';
import { resolverStoreFactory } from './store/index';
import { ResolverWithoutProviders } from './view/resolver_without_providers';
import { noAncestorsTwoChildren } from './data_access_layer/mocks/no_ancestors_two_children';
import { noAncestorsTwoChildrenWithRelatedEventsOnOrigin } from './data_access_layer/mocks/no_ancestors_two_children_with_related_events_on_origin';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍


/**
* These exports are used by the plugin 'resolverTest' defined in x-pack's plugin_functional suite.
@@ -23,7 +23,7 @@ export function resolverPluginSetup(): ResolverPluginSetup {
ResolverWithoutProviders,
mocks: {
dataAccessLayer: {
noAncestorsTwoChildren,
noAncestorsTwoChildrenWithRelatedEventsOnOrigin,
},
},
};
Loading