Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security solution][Endpoint] Add Host Isolation related data to the endpoint generator and test data loader #100727

Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ import seedrandom from 'seedrandom';
import uuid from 'uuid';

const OS_FAMILY = ['windows', 'macos', 'linux'];
const BOOLEANS = [true, false];
paul-tavares marked this conversation as resolved.
Show resolved Hide resolved
/** Array of 14 day offsets */
const DAY_OFFSETS = Array.from({ length: 14 }, (_, i) => 8.64e7 * (i + 1));

/**
* A generic base class to assist in creating domain specific data generators. It includes
Expand All @@ -33,6 +36,23 @@ export class BaseDataGenerator<GeneratedDoc extends {} = {}> {
throw new Error('method not implemented!');
}

/** Returns a future ISO date string */
protected randomFutureDate(from?: Date): string {
const now = from ? from.getTime() : Date.now();
return new Date(now + this.randomChoice(DAY_OFFSETS)).toISOString();
}

/** Returns a past ISO date string */
protected randomPastDate(from?: Date): string {
const now = from ? from.getTime() : Date.now();
return new Date(now - this.randomChoice(DAY_OFFSETS)).toISOString();
}

/** Generate either `true` or `false` */
protected randomBoolean(): boolean {
return this.randomChoice(BOOLEANS);
paul-tavares marked this conversation as resolved.
Show resolved Hide resolved
}

/** generate random OS family value */
protected randomOSFamily(): string {
return this.randomChoice(OS_FAMILY);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { DeepPartial } from 'utility-types';
import { merge } from 'lodash';
import { BaseDataGenerator } from './base_data_generator';
import { EndpointAction, EndpointActionResponse, ISOLATION_ACTIONS } from '../types';

const ISOLATION_COMMANDS: ISOLATION_ACTIONS[] = ['isolate', 'unisolate'];

export class FleetActionGenerator extends BaseDataGenerator {
/** Generate an Action */
generate(overrides: DeepPartial<EndpointAction> = {}): EndpointAction {
const timeStamp = new Date(this.randomPastDate());

return merge(
{
action_id: this.randomUUID(),
'@timestamp': timeStamp.toISOString(),
expiration: this.randomFutureDate(timeStamp),
type: 'INPUT_ACTION',
input_type: 'endpoint',
agents: [this.randomUUID()],
user_id: 'elastic',
data: {
command: this.randomIsolateCommand(),
comment: this.randomString(15),
},
},
overrides
);
}

/** Generates an action response */
generateResponse(overrides: DeepPartial<EndpointActionResponse> = {}): EndpointActionResponse {
const timeStamp = new Date();

return merge(
{
action_data: {
command: this.randomIsolateCommand(),
comment: '',
},
action_id: this.randomUUID(),
agent_id: this.randomUUID(),
started_at: this.randomPastDate(),
completed_at: timeStamp.toISOString(),
error: 'some error happen',
'@timestamp': timeStamp.toISOString(),
},
overrides
);
}

protected randomIsolateCommand() {
return this.randomChoice(ISOLATION_COMMANDS);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -465,10 +465,10 @@ export class EndpointDocGenerator extends BaseDataGenerator {
applied: this.randomChoice(APPLIED_POLICIES),
},
configuration: {
isolation: false,
isolation: this.randomBoolean(),
paul-tavares marked this conversation as resolved.
Show resolved Hide resolved
},
state: {
isolation: false,
isolation: this.randomBoolean(),
},
},
};
Expand Down
48 changes: 48 additions & 0 deletions x-pack/plugins/security_solution/common/endpoint/index_data.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,10 @@ import { policyFactory as policyConfigFactory } from './models/policy_config';
import { HostMetadata } from './types';
import { KbnClientWithApiKeySupport } from '../../scripts/endpoint/kbn_client_with_api_key_support';
import { FleetAgentGenerator } from './data_generators/fleet_agent_generator';
import { FleetActionGenerator } from './data_generators/fleet_action_generator';

const fleetAgentGenerator = new FleetAgentGenerator();
const fleetActionGenerator = new FleetActionGenerator();

export async function indexHostsAndAlerts(
client: Client,
Expand Down Expand Up @@ -175,6 +177,9 @@ async function indexHostDocs({
},
},
};

// Create some actions for this Host
await indexFleetActionsForHost(client, hostMetadata);
}

await client.index({
Expand Down Expand Up @@ -397,3 +402,46 @@ const indexFleetAgentForHost = async (

return agentDoc;
};

const indexFleetActionsForHost = async (
esClient: Client,
endpointHost: HostMetadata
): Promise<void> => {
const ES_INDEX_OPTIONS = { headers: { 'X-elastic-product-origin': 'fleet' } };
const agentId = endpointHost.elastic.agent.id;

for (let i = 0; i < 5; i++) {
// create an action
const isolateAction = fleetActionGenerator.generate({
data: { comment: 'data generator: this host is bad' },
});

isolateAction.agents = [agentId];

await esClient.index(
{
index: '.fleet-actions',
body: isolateAction,
},
ES_INDEX_OPTIONS
);

// Create an action response for the above
const unIsolateAction = fleetActionGenerator.generateResponse({
action_id: isolateAction.action_id,
agent_id: agentId,
action_data: {
command: isolateAction.data.command,
comment: 'data generator: bad host isolated',
paul-tavares marked this conversation as resolved.
Show resolved Hide resolved
},
});

await esClient.index(
{
index: '.fleet-actions-results',
body: unIsolateAction,
},
ES_INDEX_OPTIONS
);
}
};
15 changes: 15 additions & 0 deletions x-pack/plugins/security_solution/common/endpoint/types/actions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,21 @@ export interface EndpointAction {
};
}

export interface EndpointActionResponse {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this already exist anywhere else? @ashokaditya maybe in your PR?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might exist in Ash's draft. Does not exist anywhere else yet, since we don't really know what the success structure looks like (or even failure when it comes from the endpoint).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I was wondering that as well. Feels like the Agent should know what an error/success looks like since they are the ones writing this to es. (just by looking at the schema, I thought that if error was populated, then it "failed", else, it was "success" 🤷‍♂️ )

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agent knows about the fields it writes. It does not know about the fields Endpoint writes (and vice versa).

AFAIK it treats the response given via endpoint as a total unparsed black box, wraps it with its own fields, and then passes to fleet server (where maybe more fields are written? no? who knows?).

that error field is coming from either fleet-server or agent who have a failure in their system before being able to deliver to endpoint.

'@timestamp': string;
/** The id of the action for which this response is associated with */
action_id: string;
/** The agent id that sent this action response */
agent_id: string;
started_at: string;
completed_at: string;
error: string;
action_data: {
command: ISOLATION_ACTIONS;
comment?: string;
};
}

export type HostIsolationRequestBody = TypeOf<typeof HostIsolationRequestSchema.body>;

export interface HostIsolationResponse {
Expand Down