Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SIEM][CASE] Refactor Connectors - Jira Connector #63450

Merged
merged 35 commits into from
Apr 30, 2020
Merged
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
e42a338
Refactor connectors
cnasikas Apr 14, 2020
812b4b2
Add tests
cnasikas Apr 14, 2020
f6708ac
Delete servicenow
cnasikas Apr 15, 2020
04cc044
Change UI
cnasikas Apr 15, 2020
71324a7
Presonalized configuration
cnasikas Apr 16, 2020
f4ab071
Refactor validators
cnasikas Apr 16, 2020
1273434
Refactor get error message
cnasikas Apr 16, 2020
c629c13
Init IBM Resilient
cnasikas Apr 16, 2020
cb21062
Sequential comments
cnasikas Apr 16, 2020
4e256ff
Create resilient flyout
cnasikas Apr 16, 2020
aadf8bf
Refactor connectors flyout
cnasikas Apr 21, 2020
3352ca8
Adopt Jira
cnasikas Apr 22, 2020
4beadfe
Common API
cnasikas Apr 22, 2020
e1484c8
Fix tests
cnasikas Apr 23, 2020
c9b1267
Update README
cnasikas Apr 23, 2020
6b481c9
Fix i18n
cnasikas Apr 23, 2020
84b4d13
Fix lint errors
cnasikas Apr 27, 2020
4af3820
Test Jira service
cnasikas Apr 27, 2020
eb166e2
Test Jira api
cnasikas Apr 27, 2020
0ed84b1
Add integration tests
cnasikas Apr 27, 2020
af483b9
Refactor structure
cnasikas Apr 27, 2020
57d6107
Fix circular dependencies
cnasikas Apr 28, 2020
aa6fa7c
Rename folder
cnasikas Apr 29, 2020
aee3e47
Change translation's keys
cnasikas Apr 29, 2020
329d083
Change naming for sub actions
cnasikas Apr 29, 2020
ce9db2a
Improve typing
cnasikas Apr 30, 2020
ed41eb6
Better translation keys
cnasikas Apr 30, 2020
66b09b7
Improve comments creation
cnasikas Apr 30, 2020
9245748
Improve transformers
cnasikas Apr 30, 2020
9c54ea4
Improve variable destruction and types
cnasikas Apr 30, 2020
1a64bf6
Improve API helper functions
cnasikas Apr 30, 2020
3603141
Fix integration tests
cnasikas Apr 30, 2020
269d164
Merge branch 'master' into refactor_connectors
elasticmachine Apr 30, 2020
42fbd47
Merge branch 'master' into refactor_connectors
elasticmachine Apr 30, 2020
102826e
Change undefined values to nullable
cnasikas Apr 30, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 56 additions & 4 deletions x-pack/plugins/actions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Table of Contents
- [RESTful API](#restful-api)
- [`POST /api/action`: Create action](#post-apiaction-create-action)
- [`DELETE /api/action/{id}`: Delete action](#delete-apiactionid-delete-action)
- [`GET /api/action/_getAll`: Get all actions](#get-apiaction-get-all-actions)
- [`GET /api/action/_getAll`: Get all actions](#get-apiactiongetall-get-all-actions)
- [`GET /api/action/{id}`: Get action](#get-apiactionid-get-action)
- [`GET /api/action/types`: List action types](#get-apiactiontypes-list-action-types)
- [`PUT /api/action/{id}`: Update action](#put-apiactionid-update-action)
Expand Down Expand Up @@ -64,6 +64,12 @@ Table of Contents
- [`config`](#config-6)
- [`secrets`](#secrets-6)
- [`params`](#params-6)
- [`subActionParams (pushToService)`](#subactionparams-pushtoservice)
- [Jira](#jira)
- [`config`](#config-7)
- [`secrets`](#secrets-7)
- [`params`](#params-7)
- [`subActionParams (pushToService)`](#subactionparams-pushtoservice-1)
- [Command Line Utility](#command-line-utility)

## Terminology
Expand Down Expand Up @@ -143,8 +149,8 @@ This is the primary function for an action type. Whenever the action needs to ex
| actionId | The action saved object id that the action type is executing for. |
| config | The decrypted configuration given to an action. This comes from the action saved object that is partially or fully encrypted within the data store. If you would like to validate the config before being passed to the executor, define `validate.config` within the action type. |
| params | Parameters for the execution. These will be given at execution time by either an alert or manually provided when calling the plugin provided execute function. |
| services.callCluster(path, opts) | Use this to do Elasticsearch queries on the cluster Kibana connects to. This function is the same as any other `callCluster` in Kibana but runs in the context of the user who is calling the action when security is enabled.|
| services.getScopedCallCluster | This function scopes an instance of CallCluster by returning a `callCluster(path, opts)` function that runs in the context of the user who is calling the action when security is enabled. This must only be called with instances of CallCluster provided by core.|
| services.callCluster(path, opts) | Use this to do Elasticsearch queries on the cluster Kibana connects to. This function is the same as any other `callCluster` in Kibana but runs in the context of the user who is calling the action when security is enabled. |
| services.getScopedCallCluster | This function scopes an instance of CallCluster by returning a `callCluster(path, opts)` function that runs in the context of the user who is calling the action when security is enabled. This must only be called with instances of CallCluster provided by core. |
| services.savedObjectsClient | This is an instance of the saved objects client. This provides the ability to do CRUD on any saved objects within the same space the alert lives in.<br><br>The scope of the saved objects client is tied to the user in context calling the execute API or the API key provided to the execute plugin function (only when security isenabled). |
| services.log(tags, [data], [timestamp]) | Use this to create server logs. (This is the same function as server.log) |

Expand Down Expand Up @@ -483,13 +489,59 @@ The ServiceNow action uses the [V2 Table API](https://developer.servicenow.com/a

### `params`

| Property | Description | Type |
| --------------- | ------------------------------------------------------------------------------------ | ------ |
| subAction | The sub action to perform. It can be `pushToService`, `handshake`, and `getIncident` | string |
| subActionParams | The parameters of the sub action | object |

#### `subActionParams (pushToService)`

| Property | Description | Type |
| ----------- | -------------------------------------------------------------------------------------------------------------------------- | --------------------- |
| caseId | The case id | string |
| title | The title of the case | string _(optional)_ |
| description | The description of the case | string _(optional)_ |
| comments | The comments of the case. A comment is of the form `{ commentId: string, version: string, comment: string }` | object[] _(optional)_ |
| incidentID | The id of the incident in ServiceNow . If presented the incident will be update. Otherwise a new incident will be created. | string _(optional)_ |
| externalId | The id of the incident in ServiceNow . If presented the incident will be update. Otherwise a new incident will be created. | string _(optional)_ |

---

## Jira

ID: `.jira`

The Jira action uses the [V2 API](https://developer.atlassian.com/cloud/jira/platform/rest/v2/) to create and update Jira incidents.

### `config`

| Property | Description | Type |
| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ |
| apiUrl | ServiceNow instance URL. | string |
| casesConfiguration | Case configuration object. The object should contain an attribute called `mapping`. A `mapping` is an array of objects. Each mapping object should be of the form `{ source: string, target: string, actionType: string }`. `source` is the Case field. `target` is the Jira field where `source` will be mapped to. `actionType` can be one of `nothing`, `overwrite` or `append`. For example the `{ source: 'title', target: 'summary', actionType: 'overwrite' }` record, inside mapping array, means that the title of a case will be mapped to the short description of an incident in ServiceNow and will be overwrite on each update. | object |

### `secrets`

| Property | Description | Type |
| -------- | --------------------------------------- | ------ |
| email | email for HTTP Basic authentication | string |
| apiToken | API token for HTTP Basic authentication | string |

### `params`

| Property | Description | Type |
| --------------- | ------------------------------------------------------------------------------------ | ------ |
| subAction | The sub action to perform. It can be `pushToService`, `handshake`, and `getIncident` | string |
| subActionParams | The parameters of the sub action | object |

#### `subActionParams (pushToService)`

| Property | Description | Type |
| ----------- | ------------------------------------------------------------------------------------------------------------------- | --------------------- |
| caseId | The case id | string |
| title | The title of the case | string _(optional)_ |
| description | The description of the case | string _(optional)_ |
| comments | The comments of the case. A comment is of the form `{ commentId: string, version: string, comment: string }` | object[] _(optional)_ |
| externalId | The id of the incident in Jira. If presented the incident will be update. Otherwise a new incident will be created. | string _(optional)_ |

# Command Line Utility

Expand Down
93 changes: 93 additions & 0 deletions x-pack/plugins/actions/server/builtin_action_types/case/api.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import {
ExternalServiceApi,
ExternalServiceParams,
PushToServiceResponse,
GetIncidentApiHandlerArgs,
HandshakeApiHandlerArgs,
PushToServiceApiHandlerArgs,
} from './types';
import { prepareFieldsForTransformation, transformFields, transformComments } from './utils';

const handshakeHandler = async ({
externalService,
mapping,
params,
}: HandshakeApiHandlerArgs) => {};
const getIncidentHandler = async ({
externalService,
mapping,
params,
}: GetIncidentApiHandlerArgs) => {};

const pushToServiceHandler = async ({
externalService,
mapping,
params,
}: PushToServiceApiHandlerArgs): Promise<PushToServiceResponse> => {
const { externalId, comments } = params;
const updateIncident = externalId ? true : false;
const defaultPipes = updateIncident ? ['informationUpdated'] : ['informationCreated'];
let currentIncident: ExternalServiceParams | undefined;
let res: PushToServiceResponse;

if (externalId) {
currentIncident = await externalService.getIncident(externalId);
}

const fields = prepareFieldsForTransformation({
params,
mapping,
defaultPipes,
});

const incident = transformFields({
params,
fields,
currentIncident,
});

if (updateIncident) {
res = await externalService.updateIncident({ incidentId: externalId, incident });
} else {
res = await externalService.createIncident({ incident });
}

if (
comments &&
Array.isArray(comments) &&
comments.length > 0 &&
mapping.get('comments')?.actionType !== 'nothing'
) {
const commentsTransformed = transformComments(comments, ['informationAdded']);

res.comments = [];
for (const currentComment of commentsTransformed) {
const comment = await externalService.createComment({
incidentId: res.id,
comment: currentComment,
field: mapping.get('comments')?.target ?? 'comments',
});
res.comments = [
...(res.comments ?? []),
{
commentId: comment.commentId,
pushedDate: comment.pushedDate,
},
];
}
}

return res;
};

export const api: ExternalServiceApi = {
handshake: handshakeHandler,
pushToService: pushToServiceHandler,
getIncident: getIncidentHandler,
};
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,4 @@
* you may not use this file except in compliance with the Elastic License.
*/

export const ACTION_TYPE_ID = '.servicenow';
export const SUPPORTED_SOURCE_FIELDS = ['title', 'comments', 'description'];
99 changes: 99 additions & 0 deletions x-pack/plugins/actions/server/builtin_action_types/case/schema.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import { schema } from '@kbn/config-schema';

export const MappingActionType = schema.oneOf([
schema.literal('nothing'),
schema.literal('overwrite'),
schema.literal('append'),
]);

export const MapRecordSchema = schema.object({
source: schema.string(),
target: schema.string(),
actionType: MappingActionType,
});

export const CaseConfigurationSchema = schema.object({
mapping: schema.arrayOf(MapRecordSchema),
});

export const ExternalIncidentServiceConfiguration = {
apiUrl: schema.string(),
casesConfiguration: CaseConfigurationSchema,
};

export const ExternalIncidentServiceConfigurationSchema = schema.object(
ExternalIncidentServiceConfiguration
);

export const ExternalIncidentServiceSecretConfiguration = {
password: schema.string(),
username: schema.string(),
};

export const ExternalIncidentServiceSecretConfigurationSchema = schema.object(
ExternalIncidentServiceSecretConfiguration
);

export const UserSchema = schema.object({
fullName: schema.oneOf([schema.nullable(schema.string()), schema.maybe(schema.string())]),
username: schema.oneOf([schema.nullable(schema.string()), schema.maybe(schema.string())]),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made a note about the use of schema.maybe() in another comment, but this one seems ... more interesting. I think technically you can just use fullName: schema.nullable(schema.string()), but am curious if you had to do it this way for some other reason. Even for cases like MappingActionType above, I've instead sometimes just set the schema to a string, and validated the fixed set of literals in a custom validator, to produce a better error message for validation.

Probably worthwhile noting that the validation error messages produced from schema.oneOf() are quite verbose, and can be a little confusing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a good reason for that either. Probably I did it to match fullname: string | undefined | null. You are right, no need for that.

About schema.oneOf you are right. It's quite verbose and difficult to catch. I tried to use the validate function but there was a bug (#64906). I would follow your advice and create a custom validator to produce better error messages for our schema in another PR.

});

const EntityInformation = {
createdAt: schema.string(),
createdBy: UserSchema,
updatedAt: schema.nullable(schema.string()),
updatedBy: schema.nullable(UserSchema),
};

export const EntityInformationSchema = schema.object(EntityInformation);

export const CommentSchema = schema.object({
commentId: schema.string(),
comment: schema.string(),
version: schema.maybe(schema.string()),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should be using schema.nullable() here instead of schema.maybe(). Otherwise, if the field is set at some point, and then you'd like to "null it out", there's no way to do that, as the updates we make to the SO are partial updates, so the effect would be that the field is unchanged.

There are a couple of other references to schema.maybe() in here as well.

I'll note that it's unwieldy to use the TS types generated from schema.nullable() to generate "writable" data structures in code, as TS will require the fields to be set to undefined or null - you can't simply just not set the field. But I believe that's true for schema.maybe() as well, and in general, using the the TS types generated from schema to use with programmatically built data structures is ... can be a bit painful.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right about nullable. I do not know why we started with undefined. The UI sent null for non-used values. Thanks a lot for catching this.

...EntityInformation,
});

export const ExecutorSubActionSchema = schema.oneOf([
schema.literal('getIncident'),
schema.literal('pushToService'),
schema.literal('handshake'),
]);

export const ExecutorSubActionPushParamsSchema = schema.object({
caseId: schema.string(),
title: schema.string(),
description: schema.maybe(schema.string()),
comments: schema.maybe(schema.arrayOf(CommentSchema)),
externalId: schema.nullable(schema.string()),
...EntityInformation,
});

export const ExecutorSubActionGetIncidentParamsSchema = schema.object({
externalId: schema.string(),
});

// Reserved for future implementation
export const ExecutorSubActionHandshakeParamsSchema = schema.object({});

export const ExecutorParamsSchema = schema.oneOf([
cnasikas marked this conversation as resolved.
Show resolved Hide resolved
schema.object({
subAction: schema.literal('getIncident'),
subActionParams: ExecutorSubActionGetIncidentParamsSchema,
}),
schema.object({
subAction: schema.literal('handshake'),
subActionParams: ExecutorSubActionHandshakeParamsSchema,
}),
schema.object({
subAction: schema.literal('pushToService'),
subActionParams: ExecutorSubActionPushParamsSchema,
}),
]);
Loading