[Cases][Connectors] Migrate to using Saved Object References array to reference action saved objects from Cases

[ymao1]

Currently, there are several cases saved object types that directly store the associated actions SO ID instead of utilizing the references array. With alerting planning to make actions share-capable in 8.0.0, the SO IDs for these actions will be re-generated, which will break the association in these cases saved objects.

Cases SO

Stores the associated action SO ID in the cases.connector.id field

Example `cases` saved object

{
	"_index": ".kibana_8.0.0_001",
	"_id": "cases:554e2720-e8b2-11eb-bc58-13b6ae6d011a",
	"_score": 0.0,
	"_source": {
		"cases": {
			"type": "individual",
			"title": "test case",
			"tags": [],
			"description": "hi",
			"connector": {
				"id": "7b0cf000-e8a7-11eb-bc58-13b6ae6d011a", <---- action SO ID
				"name": "test servicenow",
				"type": ".servicenow",
				"fields": [{
						"key": "urgency",
						"value": null
					},
					{
						"key": "severity",
						"value": null
					},
					{
						"key": "impact",
						"value": null
					},
					{
						"key": "category",
						"value": null
					},
					{
						"key": "subcategory",
						"value": null
					}
				]
			},
			"settings": {
				"syncAlerts": true
			},
			"owner": "securitySolution",
			"closed_at": null,
			"closed_by": null,
			"created_at": "2021-07-19T16:56:59.796Z",
			"created_by": {
				"email": null,
				"full_name": null,
				"username": "elastic"
			},
			"external_service": null,
			"status": "open",
			"updated_at": "2021-07-19T17:04:19.545Z",
			"updated_by": {
				"full_name": null,
				"email": null,
				"username": "elastic"
			}
		},
		"type": "cases",
		"references": [],
		"migrationVersion": {
			"cases": "7.14.0"
		},
		"coreMigrationVersion": "8.0.0",
		"updated_at": "2021-07-19T17:04:19.548Z"
	}
}

When the action SO is made share-capable, this error will show in the Cases UI because the ID has changed.

Screenshot of error

Cases User Actions SO

Stores the associated action SO ID when the user changes the connector associated with a case. The old and new action SO IDs are stored as part of a stringified JSON.

Example `cases-user-actions` saved object

{
	"_index": ".kibana_8.0.0_001",
	"_id": "cases-user-actions:99836640-e8a7-11eb-bc58-13b6ae6d011a",
	"_score": 0.0,
	"_source": {
		"cases-user-actions": {
			"action_field": [
				"connector"
			],
			"action": "update",
			"action_at": "2021-07-19T15:40:09.374Z",
			"action_by": {
				"email": null,
				"full_name": null,
				"username": "elastic"
			},
			"new_value": """{"id ":"7b0cf000- e8a7- 11eb-bc58-13b6ae6d011a ","name ":"test servicenow ","type ":".servicenow ","fields ":null}""",
			"old_value": """{"id ":"6e67ec46-28af-4c19-8f19-683d6223ae26","name ":"service now personal developer instance ","type ":".servicenow ","fields ":{"urgency ":null,"severity ":null,"impact ":null,"category ":null,"subcategory ":null}}""",
			"owner": "securitySolution"
		},
		"type": "cases-user-actions",
		"references": [{
			"type": "cases",
			"name": "associated-cases",
			"id": "270e4080-e8a7-11eb-bc58-13b6ae6d011a"
		}],
		"migrationVersion": {
			"cases-user-actions": "7.14.0"
		},
		"coreMigrationVersion": "8.0.0",
		"updated_at": "2021-07-19T15:40:09.764Z"
	}
}

When the action SO is made share-capable, the Cases UI will display a user action message of ${user} removed external incident management system instead of the correct user action.

Cases Configure SO

Stores the associated action SO ID in the connector.id field.

Example `cases-configure` saved object

{
	"_index": ".kibana_8.0.0_001",
	"_id": "cases-configure:2d5dbcc0-e8b3-11eb-bc58-13b6ae6d011a",
	"_score": 0.0,
	"_source": {
		"cases-configure": {
			"connector": {
				"id": "7b0cf000-e8a7-11eb-bc58-13b6ae6d011a",
				"name": "test servicenow",
				"type": ".servicenow",
				"fields": []
			},
			"closure_type": "close-by-user",
			"owner": "securitySolution",
			"created_at": "2021-07-19T17:03:02.286Z",
			"created_by": {
				"username": "elastic",
				"email": null,
				"full_name": null
			},
			"updated_at": null,
			"updated_by": null
		},
		"type": "cases-configure",
		"references": [],
		"migrationVersion": {
			"cases-configure": "7.14.0"
		},
		"coreMigrationVersion": "8.0.0",
		"updated_at": "2021-07-19T17:03:02.571Z"
	}
}

When the action SO is made share-capable, the Configure Cases UI will incorrect state that the associated connector has been deleted.

Screenshot of message

Note that the cases-connector-mappings SO correctly uses the references array to store the associated action SO.

Recommended approach

The recommended approach is to update the cases, cases-user-actions and cases-configure saved objects so that associated action SO IDs are stored inside the references array. This requires a code update to extracted the associated action SO ID on create/update from the document into the references array and to inject the ID on find/get. This would also require a migration to update existing SOs of this type to use the references array. Once this is done, the saved object service will takes care of updating the references when action SOs are made share-capable.

Alternative approach

Alternatively, the cases team could explore using the .resolve() functionality provided by the saved object service in order to resolve the outdated action SO ID to their re-generated ID. This would require tracking down all the places where the cases, cases-user-actions and cases-configure saved objects are used to retrieve the associated actiohn SO and injecting the resolve layer to ensure the correct SO is loaded. This seems like a more brittle approach.

When does this need to be done?

These changes should be completed by 7.16 FF as the saved object type conversions will be done in 8.0

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

[cnasikas]

This is done in #104221 and #108272