Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sharing saved objects, phase 2 #80945

Merged
merged 41 commits into from
Jan 20, 2021
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
d47d578
Small refactor, tweak unit tests
jportner Oct 16, 2020
1b0165b
Add serializer unit tests
jportner Oct 19, 2020
e27e6be
Tweak migration loop
jportner Oct 19, 2020
02db847
Do not allow migrations higher than the current Kibana version
jportner Oct 20, 2020
c7b6d15
Add "conversion" transforms for object types
jportner Oct 20, 2020
394188b
Add "reference" transforms for object types
jportner Nov 20, 2020
7d3e99d
Add LegacyUrlAlias and SavedObjectsClient.resolve()
jportner Nov 30, 2020
5db1d2c
Create LegacyUrlAlias objects during "conversion" transforms
jportner Dec 1, 2020
0b5ab24
Add integration tests
jportner Dec 2, 2020
dd9ecd6
Update docs
jportner Dec 2, 2020
6404b5c
Merge branch 'master' into pr/jportner/80945
jportner Dec 3, 2020
04812c7
Merge branch 'master' into pr/jportner/80945
jportner Dec 4, 2020
42cc694
PR review feedback
jportner Dec 4, 2020
8fbcca7
More PR review feedback
jportner Dec 4, 2020
357ecbe
Merge branch 'master' into pr/jportner/80945
jportner Dec 4, 2020
5a52f54
Merge branch 'master' into issue-54837-ssobjects-phase-2
kibanamachine Dec 4, 2020
3ca6b23
Merge branch 'master' into pr/jportner/80945
jportner Dec 7, 2020
66b850d
Fix 500 error
jportner Dec 7, 2020
729d71c
Merge branch 'master' into issue-54837-ssobjects-phase-2
kibanamachine Dec 12, 2020
6891bcf
Merge branch 'master' into pr/jportner/80945
jportner Dec 15, 2020
5589d31
Fix merge problems
jportner Dec 15, 2020
e465648
Merge branch 'master' into pr/jportner/80945
jportner Dec 16, 2020
f7b8d40
Add usage data collection for saved objects `resolve` API
jportner Dec 16, 2020
abd7d2b
Rename things based on review feedback
jportner Dec 16, 2020
4937925
Merge branch 'master' into pr/jportner/80945
jportner Dec 21, 2020
3b15046
Merge branch 'master' into pr/jportner/80945
jportner Jan 6, 2021
56b38be
Merge branch 'master' into pr/jportner/80945
jportner Jan 11, 2021
0b45201
More PR review feedback
jportner Jan 11, 2021
4a7e0d9
Merge branch 'master' into pr/jportner/80945
jportner Jan 12, 2021
c65048a
Merge branch 'master' into pr/jportner/80945
jportner Jan 14, 2021
ca96af1
Merge branch 'master' into pr/jportner/80945
jportner Jan 15, 2021
b88b376
Small refactor of document migrator
jportner Jan 14, 2021
c486690
Update validation in the document migrator
jportner Jan 15, 2021
24a90b3
Expose `coreMigrationVersion` saved object field to consumers
jportner Jan 15, 2021
bc3b7e0
Merge branch 'master' into pr/jportner/80945
jportner Jan 19, 2021
7796bc1
Add minimum version for saved object namespace type conversion
jportner Jan 19, 2021
9ebec6c
Merge branch 'master' into pr/jportner/80945
jportner Jan 19, 2021
873e1a6
Merge branch 'master' into pr/jportner/80945
jportner Jan 20, 2021
717efcf
Update license headers
jportner Jan 20, 2021
9f88c39
Merge branch 'master' into pr/jportner/80945
jportner Jan 20, 2021
0b934ca
Merge branch 'master' into pr/jportner/80945
jportner Jan 20, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/api/saved-objects.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ The following saved objects APIs are available:

* <<saved-objects-api-get, Get object API>> to retrieve a single {kib} saved object by ID

* <<saved-objects-api-resolve, Resolve object API>> to retrieve a single {kib} saved object by ID, using any legacy URL alias if it exists
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We mention here (and elsewhere) about legacy URLs, but we don't ever explain what they are or why they're important. We can leave this as a followup item if you'd like.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this blurb to the top of the resolve API docs page:

Under certain circumstances, when Kibana is upgraded, saved object migrations may necessitate regenerating some object IDs to enable new
features. When an object's ID is regenerated, a legacy URL alias is created for that object, preserving its old ID. In such a scenario, that
object can be retrieved via the Resolve API using either its new ID or its old ID.

I figure that's a good starting point.


* <<saved-objects-api-bulk-get, Bulk get objects API>> to retrieve multiple {kib} saved objects by ID

* <<saved-objects-api-find, Find objects API>> to retrieve a paginated set of {kib} saved objects by various conditions
Expand Down Expand Up @@ -40,4 +42,5 @@ include::saved-objects/delete.asciidoc[]
include::saved-objects/export.asciidoc[]
include::saved-objects/import.asciidoc[]
include::saved-objects/resolve_import_errors.asciidoc[]
include::saved-objects/resolve.asciidoc[]
include::saved-objects/rotate_encryption_key.asciidoc[]
2 changes: 1 addition & 1 deletion docs/api/saved-objects/get.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ The API returns the following:
"title": "[Flights] Global Flight Dashboard",
"hits": 0,
"description": "Analyze mock flight data for ES-Air, Logstash Airways, Kibana Airlines and JetBeats",
"panelsJSON": "[{\"panelIndex\":\"1\",\"gridData\":{\"x\":0,\"y\":0,\"w\":32,\"h\":7,\"i\":\"1\"},\"embeddableConfig\":{},\"version\":\"6.3.0\",\"panelRefName\":\"panel_0\"},{\"panelIndex\":\"3\",\"gridData\":{\"x\":17,\"y\":7,\"w\":23,\"h\":12,\"i\":\"3\"},\"embeddableConfig\":{\"vis\":{\"colors\":{\"Average Ticket Price\":\"#0A50A1\",\"Flight Count\":\"#82B5D8\"},\"legendOpen\":false}},\"version\":\"6.3.0\",\"panelRefName\":\"panel_1\"},{\"panelIndex\":\"4\",\"gridData\":{\"x\":0,\"y\":85,\"w\":48,\"h\":15,\"i\":\"4\"},\"embeddableConfig\":{},\"version\":\"6.3.0\",\"panelRefName\":\"panel_2\"},{\"panelIndex\":\"5\",\"gridData\":{\"x\":0,\"y\":7,\"w\":17,\"h\":12,\"i\":\"5\"},\"embeddableConfig\":{\"vis\":{\"colors\":{\"ES-Air\":\"#447EBC\",\"JetBeats\":\"#65C5DB\",\"Kibana Airlines\":\"#BA43A9\",\"Logstash Airways\":\"#E5AC0E\"},\"legendOpen\":false}},\"version\":\"6.3.0\",\"panelRefName\":\"panel_3\"},{\"panelIndex\":\"6\",\"gridData\":{\"x\":24,\"y\":33,\"w\":24,\"h\":14,\"i\":\"6\"},\"embeddableConfig\":{\"vis\":{\"colors\":{\"Carrier Delay\":\"#5195CE\",\"Late Aircraft Delay\":\"#1F78C1\",\"NAS Delay\":\"#70DBED\",\"No Delay\":\"#BADFF4\",\"Security Delay\":\"#052B51\",\"Weather Delay\":\"#6ED0E0\"}}},\"version\":\"6.3.0\",\"panelRefName\":\"panel_4\"},{\"panelIndex\":\"7\",\"gridData\":{\"x\":24,\"y\":19,\"w\":24,\"h\":14,\"i\":\"7\"},\"embeddableConfig\":{},\"version\":\"6.3.0\",\"panelRefName\":\"panel_5\"},{\"panelIndex\":\"10\",\"gridData\":{\"x\":0,\"y\":35,\"w\":24,\"h\":12,\"i\":\"10\"},\"embeddableConfig\":{\"vis\":{\"colors\":{\"Count\":\"#1F78C1\"},\"legendOpen\":false}},\"version\":\"6.3.0\",\"panelRefName\":\"panel_6\"},{\"panelIndex\":\"13\",\"gridData\":{\"x\":10,\"y\":19,\"w\":14,\"h\":8,\"i\":\"13\"},\"embeddableConfig\":{\"vis\":{\"colors\":{\"Count\":\"#1F78C1\"},\"legendOpen\":false}},\"version\":\"6.3.0\",\"panelRefName\":\"panel_7\"},{\"panelIndex\":\"14\",\"gridData\":{\"x\":10,\"y\":27,\"w\":14,\"h\":8,\"i\":\"14\"},\"embeddableConfig\":{\"vis\":{\"colors\":{\"Count\":\"#1F78C1\"},\"legendOpen\":false}},\"version\":\"6.3.0\",\"panelRefName\":\"panel_8\"},{\"panelIndex\":\"18\",\"gridData\":{\"x\":24,\"y\":70,\"w\":24,\"h\":15,\"i\":\"18\"},\"embeddableConfig\":{\"mapCenter\":[27.421687059550266,15.371002131141724],\"mapZoom\":1},\"version\":\"6.3.0\",\"panelRefName\":\"panel_9\"},{\"panelIndex\":\"21\",\"gridData\":{\"x\":0,\"y\":62,\"w\":48,\"h\":8,\"i\":\"21\"},\"embeddableConfig\":{},\"version\":\"6.3.0\",\"panelRefName\":\"panel_10\"},{\"panelIndex\":\"22\",\"gridData\":{\"x\":32,\"y\":0,\"w\":16,\"h\":7,\"i\":\"22\"},\"embeddableConfig\":{},\"version\":\"6.3.0\",\"panelRefName\":\"panel_11\"},{\"panelIndex\":\"23\",\"gridData\":{\"x\":0,\"y\":70,\"w\":24,\"h\":15,\"i\":\"23\"},\"embeddableConfig\":{\"mapCenter\":[42.19556096274418,9.536742995308601e-7],\"mapZoom\":1},\"version\":\"6.3.0\",\"panelRefName\":\"panel_12\"},{\"panelIndex\":\"25\",\"gridData\":{\"x\":0,\"y\":19,\"w\":10,\"h\":8,\"i\":\"25\"},\"embeddableConfig\":{\"vis\":{\"defaultColors\":{\"0 - 50\":\"rgb(247,251,255)\",\"100 - 150\":\"rgb(107,174,214)\",\"150 - 200\":\"rgb(33,113,181)\",\"200 - 250\":\"rgb(8,48,107)\",\"50 - 100\":\"rgb(198,219,239)\"},\"legendOpen\":false}},\"version\":\"6.3.0\",\"panelRefName\":\"panel_13\"},{\"panelIndex\":\"27\",\"gridData\":{\"x\":0,\"y\":27,\"w\":10,\"h\":8,\"i\":\"27\"},\"embeddableConfig\":{\"vis\":{\"defaultColors\":{\"0 - 50\":\"rgb(247,251,255)\",\"100 - 150\":\"rgb(107,174,214)\",\"150 - 200\":\"rgb(33,113,181)\",\"200 - 250\":\"rgb(8,48,107)\",\"50 - 100\":\"rgb(198,219,239)\"},\"legendOpen\":false}},\"version\":\"6.3.0\",\"panelRefName\":\"panel_14\"},{\"panelIndex\":\"28\",\"gridData\":{\"x\":0,\"y\":47,\"w\":24,\"h\":15,\"i\":\"28\"},\"embeddableConfig\":{\"vis\":{\"defaultColors\":{\"0 -* Connection #0 to host 69c72adb58fa46c69a01afdf4a6cbfd3.us-west1.gcp.cloud.es.io left intact\n 11\":\"rgb(247,251,255)\",\"11 - 22\":\"rgb(208,225,242)\",\"22 - 33\":\"rgb(148,196,223)\",\"33 - 44\":\"rgb(74,152,201)\",\"44 - 55\":\"rgb(23,100,171)\"},\"legendOpen\":false}},\"version\":\"6.3.0\",\"panelRefName\":\"panel_15\"},{\"panelIndex\":\"29\",\"gridData\":{\"x\":40,\"y\":7,\"w\":8,\"h\":6,\"i\":\"29\"},\"embeddableConfig\":{},\"version\":\"6.3.0\",\"panelRefName\":\"panel_16\"},{\"panelIndex\":\"30\",\"gridData\":{\"x\":40,\"y\":13,\"w\":8,\"h\":6,\"i\":\"30\"},\"embeddableConfig\":{},\"version\":\"6.3.0\",\"panelRefName\":\"panel_17\"},{\"panelIndex\":\"31\",\"gridData\":{\"x\":24,\"y\":47,\"w\":24,\"h\":15,\"i\":\"31\"},\"embeddableConfig\":{},\"version\":\"6.3.0\",\"panelRefName\":\"panel_18\"}]",
"panelsJSON": "[ . . . ]",
"optionsJSON": "{\"hidePanelTitles\":false,\"useMargins\":true}",
"version": 1,
"timeRestore": true,
Expand Down
130 changes: 130 additions & 0 deletions docs/api/saved-objects/resolve.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
[[saved-objects-api-resolve]]
=== Resolve object API
++++
<titleabbrev>Resolve object</titleabbrev>
++++

experimental[] Retrieve a single {kib} saved object by ID, using any legacy URL alias if it exists.

Under certain circumstances, when Kibana is upgraded, saved object migrations may necessitate regenerating some object IDs to enable new
features. When an object's ID is regenerated, a legacy URL alias is created for that object, preserving its old ID. In such a scenario, that
object can be retrieved via the Resolve API using either its new ID or its old ID.

[[saved-objects-api-resolve-request]]
==== Request

`GET <kibana host>:<port>/api/saved_objects/resolve/<type>/<id>`

`GET <kibana host>:<port>/s/<space_id>/api/saved_objects/resolve/<type>/<id>`

[[saved-objects-api-resolve-params]]
==== Path parameters

`space_id`::
(Optional, string) An identifier for the space. If `space_id` is not provided in the URL, the default space is used.


`type`::
(Required, string) Valid options include `visualization`, `dashboard`, `search`, `index-pattern`, `config`, and `timelion-sheet`.

`id`::
(Required, string) The ID of the object to retrieve.

[[saved-objects-api-resolve-codes]]
==== Response code

`200`::
Indicates a successful call.

[[saved-objects-api-resolve-example]]
==== Example

Retrieve the index pattern object with the `my-pattern` ID:

[source,sh]
--------------------------------------------------
$ curl -X GET api/saved_objects/resolve/index-pattern/my-pattern
--------------------------------------------------
// KIBANA

The API returns the following:

[source,sh]
--------------------------------------------------
{
"saved_object": {
"id": "my-pattern",
"type": "index-pattern",
"version": 1,
"attributes": {
"title": "my-pattern-*"
}
},
"outcome": "exactMatch"
}
--------------------------------------------------

The `outcome` field may be any of the following:

* `"exactMatch"` -- One document exactly matched the given ID.
* `"aliasMatch"` -- One document with a legacy URL alias matched the given ID; in this case the `saved_object.id` field is different than the given ID.
* `"conflict"` -- Two documents matched the given ID, one was an exact match and another with a legacy URL alias; in this case the `saved_object` object is the exact match, and the `saved_object.id` field is the same as the given ID.

Retrieve a dashboard object in the `testspace` by ID:
jportner marked this conversation as resolved.
Show resolved Hide resolved

[source,sh]
--------------------------------------------------
$ curl -X GET s/testspace/api/saved_objects/resolve/dashboard/7adfa750-4c81-11e8-b3d7-01146121b73d
--------------------------------------------------
// KIBANA

The API returns the following:

[source,sh]
--------------------------------------------------
{
"saved_object": {
"id": "7adfa750-4c81-11e8-b3d7-01146121b73d",
"type": "dashboard",
"updated_at": "2019-07-23T00:11:07.059Z",
"version": "WzQ0LDFd",
"attributes": {
"title": "[Flights] Global Flight Dashboard",
"hits": 0,
"description": "Analyze mock flight data for ES-Air, Logstash Airways, Kibana Airlines and JetBeats",
"panelsJSON": "[ . . . ]",
"optionsJSON": "{\"hidePanelTitles\":false,\"useMargins\":true}",
"version": 1,
"timeRestore": true,
"timeTo": "now",
"timeFrom": "now-24h",
"refreshInterval": {
"display": "15 minutes",
"pause": false,
"section": 2,
"value": 900000
},
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\"query\":{\"language\":\"kuery\",\"query\":\"\"},\"filter\":[],\"highlightAll\":true,\"version\":true}"
}
},
"references": [
{
"name": "panel_0",
"type": "visualization",
"id": "aeb212e0-4c84-11e8-b3d7-01146121b73d"
},
. . .
{
"name": "panel_18",
"type": "visualization",
"id": "ed78a660-53a0-11e8-acbd-0be0ad9d822b"
}
],
"migrationVersion": {
"dashboard": "7.0.0"
}
},
"outcome": "conflict"
}
--------------------------------------------------
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-public](./kibana-plugin-core-public.md) &gt; [SavedObject](./kibana-plugin-core-public.savedobject.md) &gt; [coreMigrationVersion](./kibana-plugin-core-public.savedobject.coremigrationversion.md)

## SavedObject.coreMigrationVersion property

A semver value that is used when upgrading objects between Kibana versions.

<b>Signature:</b>

```typescript
coreMigrationVersion?: string;
```
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ export interface SavedObject<T = unknown>
| Property | Type | Description |
| --- | --- | --- |
| [attributes](./kibana-plugin-core-public.savedobject.attributes.md) | <code>T</code> | The data for a Saved Object is stored as an object in the <code>attributes</code> property. |
| [coreMigrationVersion](./kibana-plugin-core-public.savedobject.coremigrationversion.md) | <code>string</code> | A semver value that is used when upgrading objects between Kibana versions. |
| [error](./kibana-plugin-core-public.savedobject.error.md) | <code>SavedObjectError</code> | |
| [id](./kibana-plugin-core-public.savedobject.id.md) | <code>string</code> | The ID of this Saved Object, guaranteed to be unique for all objects of the same <code>type</code> |
| [migrationVersion](./kibana-plugin-core-public.savedobject.migrationversion.md) | <code>SavedObjectsMigrationVersion</code> | Information about the migrations that have been applied to this SavedObject. When Kibana starts up, KibanaMigrator detects outdated documents and migrates them based on this value. For each migration that has been applied, the plugin's name is used as a key and the latest migration version as the value. |
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-public](./kibana-plugin-core-public.md) &gt; [SavedObjectsCreateOptions](./kibana-plugin-core-public.savedobjectscreateoptions.md) &gt; [coreMigrationVersion](./kibana-plugin-core-public.savedobjectscreateoptions.coremigrationversion.md)

## SavedObjectsCreateOptions.coreMigrationVersion property

A semver value that is used when upgrading objects between Kibana versions.

<b>Signature:</b>

```typescript
coreMigrationVersion?: string;
```
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ export interface SavedObjectsCreateOptions

| Property | Type | Description |
| --- | --- | --- |
| [coreMigrationVersion](./kibana-plugin-core-public.savedobjectscreateoptions.coremigrationversion.md) | <code>string</code> | A semver value that is used when upgrading objects between Kibana versions. |
| [id](./kibana-plugin-core-public.savedobjectscreateoptions.id.md) | <code>string</code> | (Not recommended) Specify an id instead of having the saved objects service generate one for you. |
| [migrationVersion](./kibana-plugin-core-public.savedobjectscreateoptions.migrationversion.md) | <code>SavedObjectsMigrationVersion</code> | Information about the migrations that have been applied to this SavedObject. When Kibana starts up, KibanaMigrator detects outdated documents and migrates them based on this value. For each migration that has been applied, the plugin's name is used as a key and the latest migration version as the value. |
| [overwrite](./kibana-plugin-core-public.savedobjectscreateoptions.overwrite.md) | <code>boolean</code> | If a document with the given <code>id</code> already exists, overwrite it's contents (default=false). |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,13 @@ Constructs a new instance of the `SimpleSavedObject` class
<b>Signature:</b>

```typescript
constructor(client: SavedObjectsClientContract, { id, type, version, attributes, error, references, migrationVersion }: SavedObjectType<T>);
constructor(client: SavedObjectsClientContract, { id, type, version, attributes, error, references, migrationVersion, coreMigrationVersion, }: SavedObjectType<T>);
```

## Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| client | <code>SavedObjectsClientContract</code> | |
| { id, type, version, attributes, error, references, migrationVersion } | <code>SavedObjectType&lt;T&gt;</code> | |
| { id, type, version, attributes, error, references, migrationVersion, coreMigrationVersion, } | <code>SavedObjectType&lt;T&gt;</code> | |

Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-public](./kibana-plugin-core-public.md) &gt; [SimpleSavedObject](./kibana-plugin-core-public.simplesavedobject.md) &gt; [coreMigrationVersion](./kibana-plugin-core-public.simplesavedobject.coremigrationversion.md)

## SimpleSavedObject.coreMigrationVersion property

<b>Signature:</b>

```typescript
coreMigrationVersion: SavedObjectType<T>['coreMigrationVersion'];
```
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,15 @@ export declare class SimpleSavedObject<T = unknown>

| Constructor | Modifiers | Description |
| --- | --- | --- |
| [(constructor)(client, { id, type, version, attributes, error, references, migrationVersion })](./kibana-plugin-core-public.simplesavedobject._constructor_.md) | | Constructs a new instance of the <code>SimpleSavedObject</code> class |
| [(constructor)(client, { id, type, version, attributes, error, references, migrationVersion, coreMigrationVersion, })](./kibana-plugin-core-public.simplesavedobject._constructor_.md) | | Constructs a new instance of the <code>SimpleSavedObject</code> class |

## Properties

| Property | Modifiers | Type | Description |
| --- | --- | --- | --- |
| [\_version](./kibana-plugin-core-public.simplesavedobject._version.md) | | <code>SavedObjectType&lt;T&gt;['version']</code> | |
| [attributes](./kibana-plugin-core-public.simplesavedobject.attributes.md) | | <code>T</code> | |
| [coreMigrationVersion](./kibana-plugin-core-public.simplesavedobject.coremigrationversion.md) | | <code>SavedObjectType&lt;T&gt;['coreMigrationVersion']</code> | |
| [error](./kibana-plugin-core-public.simplesavedobject.error.md) | | <code>SavedObjectType&lt;T&gt;['error']</code> | |
| [id](./kibana-plugin-core-public.simplesavedobject.id.md) | | <code>SavedObjectType&lt;T&gt;['id']</code> | |
| [migrationVersion](./kibana-plugin-core-public.simplesavedobject.migrationversion.md) | | <code>SavedObjectType&lt;T&gt;['migrationVersion']</code> | |
Expand Down
Loading