Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Endpoint] resolver v1 events #59233

Merged
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
a9fcd64
Unifying the test index name for resolver and alerts
jonathan-buttner Mar 2, 2020
4223a42
Endpoint isn't sending the agent field so check for it
jonathan-buttner Mar 2, 2020
8562b20
Merge remote-tracking branch 'buttner/feature/single-event-index' int…
kqualters-elastic Mar 3, 2020
940959e
Update resolver to use either legacy or ecs events
kqualters-elastic Mar 3, 2020
692a945
Use correct format for child events api
kqualters-elastic Mar 3, 2020
fa4c139
Poorly handle empty child events response
kqualters-elastic Mar 6, 2020
479f813
Adding string or array for category and type
jonathan-buttner Mar 6, 2020
d9db350
Merge pull request #1 from jonathan-buttner/add-array
kqualters-elastic Mar 6, 2020
ca85aa4
Add return types to process event models
kqualters-elastic Mar 6, 2020
c160677
Merge remote-tracking branch 'upstream/master' into task/resolver-v1-…
kqualters-elastic Mar 9, 2020
084ac79
Merge branch 'task/resolver-v1-events' of https://github.com/kqualter…
kqualters-elastic Mar 10, 2020
5e06e8d
Create a common/models.ts for common event logic
kqualters-elastic Mar 10, 2020
971866a
Merge remote-tracking branch 'upstream/master' into task/resolver-v1-…
kqualters-elastic Mar 10, 2020
e5c0a1d
Decrease resolver min height
kqualters-elastic Mar 10, 2020
ed6c269
Merge remote-tracking branch 'upstream/master' into task/resolver-v1-…
kqualters-elastic Mar 11, 2020
aa3912f
Merge branch 'master' into task/resolver-v1-events
kqualters-elastic Mar 12, 2020
3eea057
Merge branch 'master' into task/resolver-v1-events
kqualters-elastic Mar 13, 2020
074bba0
Merge branch 'master' into task/resolver-v1-events
kqualters-elastic Mar 16, 2020
7b13c14
Update types to match cli tool
kqualters-elastic Mar 16, 2020
7b13e0e
Merge branch 'master' into task/resolver-v1-events
kqualters-elastic Mar 17, 2020
9e4a829
Add a smoke test for resolver rendering nodes, remove unused selector
kqualters-elastic Mar 17, 2020
840911c
Make parent process name optional
kqualters-elastic Mar 17, 2020
f38635c
Remove parent.parent from endpoint event
kqualters-elastic Mar 17, 2020
ef7b568
Add common/models/event
kqualters-elastic Mar 18, 2020
bcf1035
Internationalize some strings, address pr comments
kqualters-elastic Mar 18, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 14 additions & 3 deletions x-pack/plugins/endpoint/common/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,9 @@ export enum Direction {

export class EndpointAppConstants {
static BASE_API_URL = '/api/endpoint';
static ALERT_INDEX_NAME = 'my-index';
static ENDPOINT_INDEX_NAME = 'endpoint-agent*';
static EVENT_INDEX_NAME = 'endpoint-events-*';
static ALERT_INDEX_NAME = 'events-endpoint-1';
static EVENT_INDEX_NAME = 'events-endpoint-*';
static DEFAULT_TOTAL_HITS = 10000;
/**
* Legacy events are stored in indices with endgame-* prefix
Expand Down Expand Up @@ -315,23 +315,34 @@ export interface EndpointEvent {
category: string;
type: string;
id: string;
kind: string;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❔ At a glance, we have three things (category, type, and now kind) that are basically perfect synonyms. Can we do something to make this type more readable or useful? Either changing from string to the things that can actually be there, or documenting the differences?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bkimmel I can link you to the docs that describe the differences. We could use enums I suppose if we wanted but we'd have to update them as ecs changes. I think it'd probably go better the document the differences route.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bkimmel long term we need a way to keep these types in sync w/ ECS. Part of that should be getting descriptions of each field. In the meantime, getting comments on each field here is on my radar.

};
endpoint: {
process: {
entity_id: string;
parent: {
parent?: {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need process.parent.parent?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't, not sure how this was added. bad merge or something. removing.

entity_id: string;
};
name: string;
};
};
agent: {
id: string;
type: string;
};
process: {
name: string;
};
}

export type ResolverEvent = EndpointEvent | LegacyEndpointEvent;

export function isLegacyEvent(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you plz move this out of types

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❔ If its sole purpose seems to be as a type discriminant, where should it go? A selector?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think 'model' would be good. I'd say 'no' to selector as it doesn't take global state as its only param (doesn't take it at all).

event: EndpointEvent | LegacyEndpointEvent
): event is LegacyEndpointEvent {
return (event as LegacyEndpointEvent).endgame !== undefined;
}

/**
* The PageId type is used for the payload when firing userNavigatedToPage actions
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,91 +4,95 @@
* you may not use this file except in compliance with the Elastic License.
*/
import React, { memo, useMemo } from 'react';
import styled from 'styled-components';
import { i18n } from '@kbn/i18n';
import { FormattedMessage } from '@kbn/i18n/react';
import { EuiSpacer, EuiTitle, EuiText, EuiHealth, EuiTabbedContent } from '@elastic/eui';
import { useAlertListSelector } from '../../hooks/use_alerts_selector';
import * as selectors from '../../../../store/alerts/selectors';
import { ResolverEvent } from '../../../../../../../common/types';
import { MetadataPanel } from './metadata_panel';
import { FormattedDate } from '../../formatted_date';
import { AlertDetailResolver } from '../../resolver';

export const AlertDetailsOverview = memo(() => {
const alertDetailsData = useAlertListSelector(selectors.selectedAlertDetailsData);
if (alertDetailsData === undefined) {
return null;
}
const selectedAlertIsLegacyEndpointEvent = useAlertListSelector(
selectors.selectedAlertIsLegacyEndpointEvent
);
export const AlertDetailsOverview = styled(
memo(({ className }: { className?: string }) => {
const alertDetailsData = useAlertListSelector(selectors.selectedAlertDetailsData);
if (alertDetailsData === undefined) {
return null;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please remove the selectedAlertIsLegacyEndpointEvent selector if it's no longer used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


const tabs = useMemo(() => {
return [
{
id: 'overviewMetadata',
name: i18n.translate(
'xpack.endpoint.application.endpoint.alertDetails.overview.tabs.overview',
{
defaultMessage: 'Overview',
}
),
content: (
<>
<EuiSpacer />
<MetadataPanel />
</>
),
},
{
id: 'overviewResolver',
name: i18n.translate(
'xpack.endpoint.application.endpoint.alertDetails.overview.tabs.resolver',
{
defaultMessage: 'Resolver',
}
),
content: (
<>
<EuiSpacer />
{selectedAlertIsLegacyEndpointEvent && <AlertDetailResolver />}
</>
),
},
];
}, [selectedAlertIsLegacyEndpointEvent]);
const tabs = useMemo(() => {
return [
{
id: 'overviewMetadata',
name: i18n.translate(
'xpack.endpoint.application.endpoint.alertDetails.overview.tabs.overview',
{
defaultMessage: 'Overview',
}
),
content: (
<>
<EuiSpacer />
<MetadataPanel />
</>
),
},
{
id: 'overviewResolver',
name: i18n.translate(
'xpack.endpoint.application.endpoint.alertDetails.overview.tabs.resolver',
{
defaultMessage: 'Resolver',
}
),
content: (
<>
<EuiSpacer />
<AlertDetailResolver selectedEvent={(alertDetailsData as unknown) as ResolverEvent} />
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this unsafe cast needed? can we work out a way to remove it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imo we should get rid of the AlertData type, as it's really just a variation of an EndpointEvent

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does removing the cast change anything then?

Copy link
Contributor

@peluja1012 peluja1012 Mar 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oatkiller @kqualters-elastic @marshallmain ResolverEvent is defined as:

ResolverEvent = EndpointEvent | LegacyEndpointEvent;

AlertData, which alertDetailsData is a type of, is defined as:

export type AlertData = AlertEvent & AlertMetadata;

We have malware_event, network, process, and registry ECS schemas defined here: https://github.com/elastic/endpoint-app-team/tree/master/schemas/v1

Does it make sense to model our types after these schemas? If so, I think our AlertEvent type should follow the malware_event ECS and our EndpointEvent type should probably be a union of a new ProcessEvent type and our existing AlertEvent.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes agreed, wouldn't hurt to support LegacyEndpointEvent as well

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that adds up. That being said, if malware_event, network, process, and registry are in the same index, I don't see why they shouldn't be the same schema. In fact, I don't see why ECS shouldn't just be a single schema. Its not Elastic Common Schemas is it?

</>
),
},
];
}, [alertDetailsData]);

return (
<>
<section className="details-overview-summary">
<EuiTitle size="s">
<h3>
<FormattedMessage
id="xpack.endpoint.application.endpoint.alertDetails.overview.title"
defaultMessage="Detected Malicious File"
/>
</h3>
</EuiTitle>
<EuiSpacer />
<EuiText>
<p>
<FormattedMessage
id="xpack.endpoint.application.endpoint.alertDetails.overview.summary"
defaultMessage="MalwareScore detected the opening of a document on {hostname} on {date}"
values={{
hostname: alertDetailsData.host.hostname,
date: <FormattedDate timestamp={alertDetailsData['@timestamp']} />,
}}
/>
</p>
</EuiText>
<EuiSpacer />
<EuiText>
Endpoint Status: <EuiHealth color="success">Online</EuiHealth>
</EuiText>
<EuiText>Alert Status: Open</EuiText>
<EuiSpacer />
</section>
<EuiTabbedContent tabs={tabs} initialSelectedTab={tabs[0]} />
</>
);
});
return (
<>
<section className="details-overview-summary">
<EuiTitle size="s">
<h3>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❔ I think the h3 might be ( ❔ ) redundant but maybe you have to use the size="l" variant of EUITitle

<FormattedMessage
id="xpack.endpoint.application.endpoint.alertDetails.overview.title"
defaultMessage="Detected Malicious File"
/>
</h3>
</EuiTitle>
<EuiSpacer />
<EuiText>
<p>
<FormattedMessage
id="xpack.endpoint.application.endpoint.alertDetails.overview.summary"
defaultMessage="MalwareScore detected the opening of a document on {hostname} on {date}"
values={{
hostname: alertDetailsData.host.hostname,
date: <FormattedDate timestamp={alertDetailsData['@timestamp']} />,
}}
/>
</p>
</EuiText>
<EuiSpacer />
<EuiText>
Endpoint Status: <EuiHealth color="success">Online</EuiHealth>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you please i18n this?

</EuiText>
<EuiText>Alert Status: Open</EuiText>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you please i18n this?

<EuiSpacer />
</section>
<EuiTabbedContent tabs={tabs} initialSelectedTab={tabs[0]} className={className} />
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆗 Nice use of EuiTabbedContent

</>
);
})
)`
height: 100%;
width: 100%;
`;
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ import { Provider } from 'react-redux';
import { useKibana } from '../../../../../../../../src/plugins/kibana_react/public';
import { Resolver } from '../../../../embeddables/resolver/view';
import { EndpointPluginServices } from '../../../../plugin';
import { LegacyEndpointEvent } from '../../../../../common/types';
import { ResolverEvent } from '../../../../../common/types';
import { storeFactory } from '../../../../embeddables/resolver/store';

export const AlertDetailResolver = styled(
React.memo(
({ className, selectedEvent }: { className?: string; selectedEvent?: LegacyEndpointEvent }) => {
({ className, selectedEvent }: { className?: string; selectedEvent?: ResolverEvent }) => {
const context = useKibana<EndpointPluginServices>();
const { store } = storeFactory(context);

Expand All @@ -33,4 +33,6 @@ export const AlertDetailResolver = styled(
width: 100%;
display: flex;
flex-grow: 1;
// ugly demo hack until can play nice with eui
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is still here by the time we merge, let's remove the comment and add a github issue instead

min-height: 1000px;
`;
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,15 @@

import { uniquePidForProcess, uniqueParentPidForProcess } from './process_event';
import { IndexedProcessTree } from '../types';
import { LegacyEndpointEvent } from '../../../../common/types';
import { ResolverEvent } from '../../../../common/types';
import { levelOrder as baseLevelOrder } from '../lib/tree_sequencers';

/**
* Create a new IndexedProcessTree from an array of ProcessEvents
*/
export function factory(processes: LegacyEndpointEvent[]): IndexedProcessTree {
const idToChildren = new Map<number | undefined, LegacyEndpointEvent[]>();
const idToValue = new Map<number, LegacyEndpointEvent>();
export function factory(processes: ResolverEvent[]): IndexedProcessTree {
const idToChildren = new Map<string | undefined, ResolverEvent[]>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🆗 Cool, I didn't know you could use Type directions in constructors that way.

const idToValue = new Map<string, ResolverEvent>();

for (const process of processes) {
idToValue.set(uniquePidForProcess(process), process);
Expand All @@ -36,10 +36,7 @@ export function factory(processes: LegacyEndpointEvent[]): IndexedProcessTree {
/**
* Returns an array with any children `ProcessEvent`s of the passed in `process`
*/
export function children(
tree: IndexedProcessTree,
process: LegacyEndpointEvent
): LegacyEndpointEvent[] {
export function children(tree: IndexedProcessTree, process: ResolverEvent): ResolverEvent[] {
const id = uniquePidForProcess(process);
const processChildren = tree.idToChildren.get(id);
return processChildren === undefined ? [] : processChildren;
Expand All @@ -50,8 +47,8 @@ export function children(
*/
export function parent(
tree: IndexedProcessTree,
childProcess: LegacyEndpointEvent
): LegacyEndpointEvent | undefined {
childProcess: ResolverEvent
): ResolverEvent | undefined {
const uniqueParentPid = uniqueParentPidForProcess(childProcess);
if (uniqueParentPid === undefined) {
return undefined;
Expand All @@ -74,7 +71,7 @@ export function root(tree: IndexedProcessTree) {
if (size(tree) === 0) {
return null;
}
let current: LegacyEndpointEvent = tree.idToProcess.values().next().value;
let current: ResolverEvent = tree.idToProcess.values().next().value;
while (parent(tree, current) !== undefined) {
current = parent(tree, current)!;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,50 +4,77 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { LegacyEndpointEvent } from '../../../../common/types';
import { isLegacyEvent, ResolverEvent } from '../../../../common/types';

/**
* Returns true if the process's eventType is either 'processCreated' or 'processRan'.
* Resolver will only render 'graphable' process events.
*/
export function isGraphableProcess(passedEvent: LegacyEndpointEvent) {
export function isGraphableProcess(passedEvent: ResolverEvent) {
return eventType(passedEvent) === 'processCreated' || eventType(passedEvent) === 'processRan';
}

/**
* Returns a custom event type for a process event based on the event's metadata.
*/
export function eventType(passedEvent: LegacyEndpointEvent) {
const {
endgame: { event_type_full: type, event_subtype_full: subType },
} = passedEvent;
export function eventType(passedEvent: ResolverEvent) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you add a return type here?

'processCreated' | 'processRan' | ... etc

if (isLegacyEvent(passedEvent)) {
const {
endgame: { event_type_full: type, event_subtype_full: subType },
} = passedEvent;

if (type === 'process_event') {
if (subType === 'creation_event' || subType === 'fork_event' || subType === 'exec_event') {
return 'processCreated';
} else if (subType === 'already_running') {
return 'processRan';
} else if (subType === 'termination_event') {
return 'processTerminated';
} else {
return 'unknownProcessEvent';
if (type === 'process_event') {
if (subType === 'creation_event' || subType === 'fork_event' || subType === 'exec_event') {
return 'processCreated';
} else if (subType === 'already_running') {
return 'processRan';
} else if (subType === 'termination_event') {
return 'processTerminated';
} else {
return 'unknownProcessEvent';
}
} else if (type === 'alert_event') {
return 'processCausedAlert';
}
return 'unknownEvent';
} else {
const {
event: { type, category, kind },
} = passedEvent;
if (category === 'process') {
if (type === 'start' || type === 'change') {
return 'processCreated';
} else if (type === 'info') {
return 'processRan';
} else if (type === 'end') {
return 'processTerminated';
} else {
return 'unknownProcessEvent';
}
} else if (kind === 'alert') {
return 'processCausedAlert';
}
} else if (type === 'alert_event') {
return 'processCausedAlert';
}
return 'unknownEvent';
}

/**
* Returns the process event's pid
*/
export function uniquePidForProcess(event: LegacyEndpointEvent) {
return event.endgame.unique_pid;
export function uniquePidForProcess(event: ResolverEvent) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add an explicit return type here

if (isLegacyEvent(event)) {
return String(event.endgame.unique_pid);
} else {
return event.endpoint.process.entity_id;
}
}

/**
* Returns the process event's parent pid
*/
export function uniqueParentPidForProcess(event: LegacyEndpointEvent) {
return event.endgame.unique_ppid;
export function uniqueParentPidForProcess(event: ResolverEvent) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add an explicit return type here

if (isLegacyEvent(event)) {
return String(event.endgame.unique_ppid);
} else {
return event.endpoint.process.parent?.entity_id;
}
}
Loading