Skip to content

Commit

Permalink
Logstash Pipeline REST api spec and documentation (elastic#67788)
Browse files Browse the repository at this point in the history
This commit adds REST api specifications for the Logstash Pipeline APIs
and documentation of the APIs.

Closes elastic#67360
  • Loading branch information
jaymode committed Jan 28, 2021
1 parent 25272e1 commit 74c5540
Show file tree
Hide file tree
Showing 8 changed files with 336 additions and 0 deletions.
2 changes: 2 additions & 0 deletions docs/reference/rest-api/index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ not be included yet.
* <<ingest-apis,Ingest APIs>>
* <<info-api,Info API>>
* <<licensing-apis,Licensing APIs>>
* <<logstash-apis,Logstash APIs>>
* <<ml-apis,{ml-cap} {anomaly-detect} APIs>>
* <<ml-df-analytics-apis,{ml-cap} {dfanalytics} APIs>>
* <<migration-api,Migration APIs>>
Expand Down Expand Up @@ -56,6 +57,7 @@ include::{es-repo-dir}/ilm/apis/ilm-api.asciidoc[]
include::{es-repo-dir}/ingest/apis/index.asciidoc[]
include::info.asciidoc[]
include::{es-repo-dir}/licensing/index.asciidoc[]
include::{xes-repo-dir}/rest-api/logstash/index.asciidoc[]
include::{es-repo-dir}/ml/anomaly-detection/apis/index.asciidoc[]
include::{es-repo-dir}/ml/df-analytics/apis/index.asciidoc[]
include::{es-repo-dir}/migration/migration.asciidoc[]
Expand Down
71 changes: 71 additions & 0 deletions x-pack/docs/en/rest-api/logstash/delete-pipeline.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
[role="xpack"]
[testenv="basic"]
[[logstash-api-delete-pipeline]]
=== Delete Pipeline API

This API deletes Pipelines used for Logstash Central Management.

[[logstash-api-delete-pipeline-request]]
==== {api-request-title}

`DELETE _logstash/pipeline/<pipeline_id>`

[[logstash-api-delete-pipeline-prereqs]]
==== {api-prereq-title}

* If the {es} {security-features} are enabled, you must have the
`manage_logstash_pipelines` cluster privilege to use this API.

[[logstash-api-delete-pipeline-desc]]
==== {api-description-title}

Delete a Logstash Pipelines.

[[logstash-api-delete-pipeline-params]]
==== {api-path-parms-title}

`<pipeline_id>`::
(Required, string) Identifier for the Pipeline.

[[logstash-api-delete-pipeline-example]]
==== {api-examples-title}

The following example deletes the Pipeline named `my_pipeline`:

//////////////////////////
[source,console]
--------------------------------------------------
PUT _logstash/pipeline/my_pipeline
{
"description": "Sample Pipeline for illustration purposes",
"last_modified": "2021-01-02T02:50:51.250Z",
"pipeline_metadata": {
"type": "logstash_pipeline",
"version": "1"
},
"username": "elastic",
"pipeline": "input {}\n filter { grok {} }\n output {}",
"pipeline_settings": {
"pipeline.workers": 1,
"pipeline.batch.size": 125,
"pipeline.batch.delay": 50,
"queue.type": "memory",
"queue.max_bytes.number": 1,
"queue.max_bytes.units": "gb",
"queue.checkpoint.writes": 1024
}
}
--------------------------------------------------
//////////////////////////

[source,console]
--------------------------------------------------
DELETE _logstash/pipeline/my_pipeline
--------------------------------------------------
// TEST[continued]


If the request succeeds, you receive an empty response with an appropriate
status code.
98 changes: 98 additions & 0 deletions x-pack/docs/en/rest-api/logstash/get-pipeline.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
[role="xpack"]
[testenv="basic"]
[[logstash-api-get-pipeline]]
=== Get Pipeline API

This API retrieves Pipelines used for Logstash Central Management.

[[logstash-api-get-pipeline-request]]
==== {api-request-title}

`GET _logstash/pipeline`

`GET _logstash/pipeline/<pipeline_id>`

[[logstash-api-get-pipeline-prereqs]]
==== {api-prereq-title}

* If the {es} {security-features} are enabled, you must have the
`manage_logstash_pipelines` cluster privilege to use this API.

[[logstash-api-get-pipeline-desc]]
==== {api-description-title}

Retrieve one or more Logstash Pipelines.

[[logstash-api-get-pipeline-params]]
==== {api-path-parms-title}

`<pipeline_id>`::
(Optional, string) Comma-separated list of Pipeline identifiers.

[[logstash-api-get-pipeline-example]]
==== {api-examples-title}

The following example retrieves the Pipeline named `my_pipeline`:

//////////////////////////
[source,console]
--------------------------------------------------
PUT _logstash/pipeline/my_pipeline
{
"description": "Sample Pipeline for illustration purposes",
"last_modified": "2021-01-02T02:50:51.250Z",
"pipeline_metadata": {
"type": "logstash_pipeline",
"version": "1"
},
"username": "elastic",
"pipeline": "input {}\n filter { grok {} }\n output {}",
"pipeline_settings": {
"pipeline.workers": 1,
"pipeline.batch.size": 125,
"pipeline.batch.delay": 50,
"queue.type": "memory",
"queue.max_bytes.number": 1,
"queue.max_bytes.units": "gb",
"queue.checkpoint.writes": 1024
}
}
--------------------------------------------------
//////////////////////////

[source,console]
--------------------------------------------------
GET _logstash/pipeline/my_pipeline
--------------------------------------------------
// TEST[continued]


If the request succeeds, the body of the response contains the Pipeline definition:

[source,console-result]
--------------------------------------------------
{
"my_pipeline": {
"description": "Sample Pipeline for illustration purposes",
"last_modified": "2021-01-02T02:50:51.250Z",
"pipeline_metadata": {
"type": "logstash_pipeline",
"version": "1"
},
"username": "elastic",
"pipeline": "input {}\n filter { grok {} }\n output {}",
"pipeline_settings": {
"pipeline.workers": 1,
"pipeline.batch.size": 125,
"pipeline.batch.delay": 50,
"queue.type": "memory",
"queue.max_bytes.number": 1,
"queue.max_bytes.units": "gb",
"queue.checkpoint.writes": 1024
}
}
}
--------------------------------------------------
// TESTRESPONSE
14 changes: 14 additions & 0 deletions x-pack/docs/en/rest-api/logstash/index.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
[role="xpack"]
[[logstash-apis]]
== Logstash APIs

The following APIs are used to manage Pipelines used by Logstash Central
Management:

* <<logstash-api-put-pipeline>>
* <<logstash-api-get-pipeline>>
* <<logstash-api-delete-pipeline>>

include::delete-pipeline.asciidoc[]
include::get-pipeline.asciidoc[]
include::put-pipeline.asciidoc[]
61 changes: 61 additions & 0 deletions x-pack/docs/en/rest-api/logstash/put-pipeline.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
[role="xpack"]
[testenv="basic"]
[[logstash-api-put-pipeline]]
=== Put Pipeline API

This API creates or updates Pipelines used for Logstash Central Management.

[[logstash-api-put-pipeline-request]]
==== {api-request-title}

`PUT _logstash/pipeline/<pipeline_id>`

[[logstash-api-put-pipeline-prereqs]]
==== {api-prereq-title}

* If the {es} {security-features} are enabled, you must have the
`manage_logstash_pipelines` cluster privilege to use this API.

[[logstash-api-put-pipeline-desc]]
==== {api-description-title}

Creates a Logstash Pipeline. If the specified Pipeline exists, the pipeline is
replaced.

[[logstash-api-put-pipeline-params]]
==== {api-path-parms-title}

`<pipeline_id>`::
(Required, string) Identifier for the Pipeline.

[[logstash-api-put-pipeline-example]]
==== {api-examples-title}

The following example creates a new Pipeline named `my_pipeline`:

[source,console]
--------------------------------------------------
PUT _logstash/pipeline/my_pipeline
{
"description": "Sample Pipeline for illustration purposes",
"last_modified": "2021-01-02T02:50:51.250Z",
"pipeline_metadata": {
"type": "logstash_pipeline",
"version": "1"
},
"username": "elastic",
"pipeline": "input {}\n filter { grok {} }\n output {}",
"pipeline_settings": {
"pipeline.workers": 1,
"pipeline.batch.size": 125,
"pipeline.batch.delay": 50,
"queue.type": "memory",
"queue.max_bytes.number": 1,
"queue.max_bytes.units": "gb",
"queue.checkpoint.writes": 1024
}
}
--------------------------------------------------

If the request succeeds, you receive an empty response with an appropriate
status code.
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"logstash.delete_pipeline":{
"documentation":{
"url":"https://www.elastic.co/guide/en/elasticsearch/reference/current/logstash-api-delete-pipeline.html",
"description":"Deletes Logstash Pipelines used by Central Management"
},
"stability":"stable",
"visibility":"public",
"headers":{
"accept": [ "application/json"]
},
"url":{
"paths":[
{
"path":"/_logstash/pipeline/{id}",
"methods":[ "DELETE" ],
"parts":{
"id":{
"type":"string",
"description":"The ID of the Pipeline"
}
}
}
]
},
"params":{}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"logstash.get_pipeline":{
"documentation":{
"url":"https://www.elastic.co/guide/en/elasticsearch/reference/current/logstash-api-get-pipeline.html",
"description":"Retrieves Logstash Pipelines used by Central Management"
},
"stability":"stable",
"visibility":"public",
"headers":{
"accept": [ "application/json"]
},
"url":{
"paths":[
{
"path":"/_logstash/pipeline/{id}",
"methods":[ "GET" ],
"parts":{
"id":{
"type":"string",
"description":"A comma-separated list of Pipeline IDs"
}
}
}
]
},
"params":{}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
{
"logstash.put_pipeline":{
"documentation":{
"url":"https://www.elastic.co/guide/en/elasticsearch/reference/current/logstash-api-put-pipeline.html",
"description":"Adds and updates Logstash Pipelines used for Central Management"
},
"stability":"stable",
"visibility":"public",
"headers":{
"accept": [ "application/json"],
"content_type": ["application/json"]
},
"url":{
"paths":[
{
"path":"/_logstash/pipeline/{id}",
"methods":[ "PUT" ],
"parts":{
"id":{
"type":"string",
"description":"The ID of the Pipeline"
}
}
}
]
},
"params":{
},
"body":{
"description":"The Pipeline to add or update",
"required":true
}
}
}

0 comments on commit 74c5540

Please sign in to comment.