Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic server side import API for saved objects #32158

Merged
merged 48 commits into from
Mar 8, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
c2512a4
Initial work
mikecote Feb 25, 2019
720699b
Add overwrite and skip support
mikecote Feb 25, 2019
2308bc5
Cleanup and add tests
mikecote Feb 26, 2019
4386021
Move code into separate files
mikecote Feb 27, 2019
20a1329
Remove reduce
mikecote Feb 27, 2019
8dfb656
New API parameters
mikecote Feb 27, 2019
d04ceb9
Add support to replace references
mikecote Feb 27, 2019
c6d3dc1
Add better error handling
mikecote Feb 27, 2019
96233a8
Add spaces tests
mikecote Feb 27, 2019
82f7090
Fix return type in collectSavedObjects
mikecote Feb 27, 2019
25d2065
Merge branch 'master' of github.com:elastic/kibana into saved-object/…
mikecote Feb 28, 2019
3c3623f
Apply PR feedback
mikecote Feb 28, 2019
e4f7a41
Update jest tests due to jest version upgrade
mikecote Feb 28, 2019
ee87cbb
Add docs
mikecote Feb 28, 2019
324f6c1
WIP
mikecote Feb 28, 2019
3043d41
Merge branch 'master' of github.com:elastic/kibana into saved-object/…
mikecote Mar 1, 2019
b51ec73
Split import routes pt1
mikecote Mar 1, 2019
ab1797a
Merge
mikecote Mar 4, 2019
b70d7de
Add tests
mikecote Mar 4, 2019
81b7fa9
Fix broken tests
mikecote Mar 4, 2019
dc94039
Update docs and fix broken test
mikecote Mar 4, 2019
1216289
Add successCount to _import endpoint
mikecote Mar 4, 2019
23ef287
Make skip by default in resolution API
mikecote Mar 4, 2019
f3aa068
Update tests for removal of skips
mikecote Mar 5, 2019
0d1f669
Add back support for skips
mikecote Mar 5, 2019
3540a2a
Add success count
mikecote Mar 5, 2019
7c47e48
Merge with master
mikecote Mar 5, 2019
d01192b
Add back resolve import conflicts x-pack tests
mikecote Mar 6, 2019
4e35a5e
Remove writev from filter stream
mikecote Mar 6, 2019
78cf906
Delete _mock_server.d.ts file
mikecote Mar 6, 2019
bfc92c5
Rename lib/import_saved_objects to lib/import
mikecote Mar 6, 2019
6476456
Filter records at stream level for conflict resolution
mikecote Mar 6, 2019
4e04b07
Update docs
mikecote Mar 6, 2019
37f2f04
Merge branch 'master' of github.com:elastic/kibana into saved-object/…
mikecote Mar 6, 2019
da6ad6e
Add tests to validate documentation
mikecote Mar 6, 2019
51c5249
Return 200 instead of other code for errors, include errors array
mikecote Mar 6, 2019
38886ce
Change [] to {}
mikecote Mar 6, 2019
e3047f1
Merge branch 'master' of github.com:elastic/kibana into saved-object/…
mikecote Mar 7, 2019
61abed5
Apply PR feedback
mikecote Mar 7, 2019
5f0c0b6
Fix import object limit to not return 500
mikecote Mar 7, 2019
ebd39d9
Change some wording in the docs
mikecote Mar 7, 2019
72c421d
Fix status code
mikecote Mar 7, 2019
7cd2b50
Apply PR feedback pt2
mikecote Mar 7, 2019
6f51a75
Merge branch 'master' of github.com:elastic/kibana into saved-object/…
mikecote Mar 8, 2019
a511dca
Lower maxImportPayloadBytes to 10MB
mikecote Mar 8, 2019
0f8c3ae
Add unknown type tests for import
mikecote Mar 8, 2019
d7f8956
Add unknown type tests for resolve_import_conflicts
mikecote Mar 8, 2019
162c595
Fix tslint issues
mikecote Mar 8, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/api/saved-objects.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ NOTE: You cannot access these endpoints via the Console in Kibana.
* <<saved-objects-api-update>>
* <<saved-objects-api-delete>>
* <<saved-objects-api-export>>
* <<saved-objects-api-import>>
* <<saved-objects-api-resolve-import-conflicts>>

include::saved-objects/get.asciidoc[]
include::saved-objects/bulk_get.asciidoc[]
Expand All @@ -28,3 +30,5 @@ include::saved-objects/bulk_create.asciidoc[]
include::saved-objects/update.asciidoc[]
include::saved-objects/delete.asciidoc[]
include::saved-objects/export.asciidoc[]
include::saved-objects/import.asciidoc[]
include::saved-objects/resolve_import_conflicts.asciidoc[]
96 changes: 96 additions & 0 deletions docs/api/saved-objects/import.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
[[saved-objects-api-import]]
=== Import Objects

experimental[This functionality is *experimental* and may be changed or removed completely in a future release.]

The import saved objects API enables you to create a set of Kibana saved objects from a file created by the export API.

Note: You cannot access this endpoint via the Console in Kibana.

==== Request

`POST /api/saved_objects/_import`

==== Query Parameters

`overwrite` (optional)::
(boolean) Overwrite saved objects if they exist already

==== Request body

The request body must be of type multipart/form-data.

`file`::
A file exported using the export API.

==== Response body

The response body will have a top level `success` property that indicates
if the import was successful or not as well as a `successCount` indicating how many records are successfully imported.
In the scenario the import wasn't successful a top level `errors` array will contain the objects that failed to import.

==== Examples

The following example imports an index pattern and dashboard.

[source,js]
--------------------------------------------------
POST api/saved_objects/_import
Content-Type: multipart/form-data; boundary=EXAMPLE
--EXAMPLE
Content-Disposition: form-data; name="file"; filename="export.ndjson"
Content-Type: application/ndjson

{"type":"index-pattern","id":"my-pattern","attributes":{"title":"my-pattern-*"}}
{"type":"dashboard","id":"my-dashboard","attributes":{"title":"Look at my dashboard"}}
--EXAMPLE--
--------------------------------------------------
// KIBANA

A successful call returns a response code of `200` and a response body
containing a JSON structure similar to the following example:

[source,js]
--------------------------------------------------
{
"success": true,
"successCount": 2
}
--------------------------------------------------

The following example imports an index pattern and dashboard but has a conflict on the index pattern.

[source,js]
--------------------------------------------------
POST api/saved_objects/_import
Content-Type: multipart/form-data; boundary=EXAMPLE
--EXAMPLE
Content-Disposition: form-data; name="file"; filename="export.ndjson"
Content-Type: application/ndjson

{"type":"index-pattern","id":"my-pattern","attributes":{"title":"my-pattern-*"}}
{"type":"dashboard","id":"my-dashboard","attributes":{"title":"Look at my dashboard"}}
--EXAMPLE--
--------------------------------------------------
// KIBANA

The call returns a response code of `200` and a response body
containing a JSON structure similar to the following example:

[source,js]
--------------------------------------------------
{
"success": false,
"successCount": 1,
"errors": [
{
"id": "my-pattern",
"type": "index-pattern",
"error": {
"statusCode": 409,
"message": "version conflict, document already exists",
},
},
],
}
--------------------------------------------------
104 changes: 104 additions & 0 deletions docs/api/saved-objects/resolve_import_conflicts.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
[[saved-objects-api-resolve-import-conflicts]]
=== Resolve Import Conflicts

experimental[This functionality is *experimental* and may be changed or removed completely in a future release.]

The resolve import conflicts API enables you to resolve conflicts given by the import API by either overwriting specific saved objects or changing references to a newly created object.

Note: You cannot access this endpoint via the Console in Kibana.

==== Request

`POST /api/saved_objects/_resolve_import_conflicts`

==== Request body

The request body must be of type multipart/form-data.

`file`::
(ndjson) The same new line delimited JSON objects given to the import API.

`overwrites` (optional)::
(array) A list of `type` and `id` objects allowed to be overwritten on import.

`replaceReferences` (optional)::
(array) A list of `type`, `from` and `to` used to change imported saved object references to.

`skips` (optional)::
(array) A list of `type` and `id` objects to skip importing.

==== Response body

The response body will have a top level `success` property that indicates
if the import was successful or not as well as a `successCount` indicating how many records are successfully resolved.
In the scenario the import wasn't successful a top level `errors` array will contain the objects that failed to import.

==== Examples

The following example resolves conflicts for an index pattern and dashboard but indicates to skip the index pattern.
This will cause the index pattern to not be in the system and the dashboard to overwrite the existing saved object.

[source,js]
--------------------------------------------------
POST api/saved_objects/_resolve_import_conflicts
Content-Type: multipart/form-data; boundary=EXAMPLE
--EXAMPLE
Content-Disposition: form-data; name="file"; filename="export.ndjson"
Content-Type: application/ndjson

{"type":"index-pattern","id":"my-pattern","attributes":{"title":"my-pattern-*"}}
{"type":"dashboard","id":"my-dashboard","attributes":{"title":"Look at my dashboard"}}
--EXAMPLE
Content-Disposition: form-data; name="skips"

[{"type":"index-pattern","id":"my-pattern"}]
--EXAMPLE
Content-Disposition: form-data; name="overwrites"

[{"type":"dashboard","id":"my-dashboard"}]
--EXAMPLE--
--------------------------------------------------
// KIBANA

A successful call returns a response code of `200` and a response body
containing a JSON structure similar to the following example:

[source,js]
--------------------------------------------------
{
"success": true,
"successCount": 1
}
--------------------------------------------------

The following example resolves conflicts for a visualization and dashboard but indicates
to replace the dashboard references to another visualization.

[source,js]
--------------------------------------------------
POST api/saved_objects/_resolve_import_conflicts
Content-Type: multipart/form-data; boundary=EXAMPLE
--EXAMPLE
Content-Disposition: form-data; name="file"; filename="export.ndjson"
Content-Type: application/ndjson

{"type":"visualization","id":"my-vis","attributes":{"title":"Look at my visualization"}}
{"type":"dashboard","id":"my-dashboard","attributes":{"title":"Look at my dashboard"},"references":[{"name":"panel_0","type":"visualization","id":"my-vis"}]}
--EXAMPLE
Content-Disposition: form-data; name="replaceReferences"

[{"type":"visualization","from":"my-vis","to":"my-vis-2"}]
--EXAMPLE--
--------------------------------------------------
// KIBANA

A successful call returns a response code of `200` and a response body
containing a JSON structure similar to the following example:

[source,js]
--------------------------------------------------
{
"success": true,
"successCount": 1
}
--------------------------------------------------
1 change: 1 addition & 0 deletions src/legacy/server/config/schema.js
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,7 @@ export default () => Joi.object({
}).default(),

savedObjects: Joi.object({
maxImportPayloadBytes: Joi.number().default(10485760),
maxImportExportSize: Joi.number().default(10000),
}).default(),

Expand Down
Loading