diff --git a/docs/reference/api-conventions.asciidoc b/docs/reference/api-conventions.asciidoc index a0e52f8656693..7b95abf6c6cf4 100644 --- a/docs/reference/api-conventions.asciidoc +++ b/docs/reference/api-conventions.asciidoc @@ -52,14 +52,19 @@ NOTE: Single index APIs, such as the <> and syntax. [[hidden-indices]] -==== Hidden indices +==== Hidden data streams and indices -Indices that are configured to be hidden with the <> setting are -excluded from mult-target queries by default. -To include hidden indices, you must specify the `expand_wildcards` parameter. +For most APIs, wildcard expressions do not match hidden data streams and indices +by default. To match hidden data streams and indices using a wildcard +expression, you must specify the `expand_wildcards` query parameter. -The backing indices for data streams are hidden indices, -and some features like {ml} store information in hidden indices. +You can create hidden data streams by setting +<> to `true` in the stream's matching +index template. You can hide indices using the <> +index setting. + +The backing indices for data streams are hidden automatically. Some features, +such as {ml}, store information in hidden indices. Global index templates that match all indices are not applied to hidden indices. diff --git a/docs/reference/data-streams/set-up-a-data-stream.asciidoc b/docs/reference/data-streams/set-up-a-data-stream.asciidoc index 9ad3b957643f6..609602f68489a 100644 --- a/docs/reference/data-streams/set-up-a-data-stream.asciidoc +++ b/docs/reference/data-streams/set-up-a-data-stream.asciidoc @@ -66,7 +66,8 @@ PUT /_ilm/policy/my-data-stream-policy template is used for data streams. . Use the wizard to finish defining your template. Specify: -* One or more index patterns that match the data stream's name. +* One or more index patterns that match the data stream's name. + +include::{es-repo-dir}/indices/create-data-stream.asciidoc[tag=data-stream-name] * Mappings and settings for the stream's backing indices. @@ -94,7 +95,6 @@ maps `@timestamp` as a `date` field with default options. If using {ilm-init}, specify your lifecycle policy in the `index.lifecycle.name` setting. - TIP: Carefully consider your template's mappings and settings. Later changes may require reindexing. See <>. @@ -105,7 +105,7 @@ image::images/data-streams/create-index-template.png[Create template page] .API example ==== Use the <> to create an index -template. The template must include an empty `data_stream` object, indicating +template. The template must include a `data_stream` object, indicating it's used for data streams. [source,console] diff --git a/docs/reference/indices/create-data-stream.asciidoc b/docs/reference/indices/create-data-stream.asciidoc index d4de016686b0a..6705afccdfc99 100644 --- a/docs/reference/indices/create-data-stream.asciidoc +++ b/docs/reference/indices/create-data-stream.asciidoc @@ -49,14 +49,16 @@ DELETE /_index_template/template -- (Required, string) Name of the data stream to create. +// tag::data-stream-name[] Data stream names must meet the following criteria: - Lowercase only -- Cannot include `\`, `/`, `*`, `?`, `"`, `<`, `>`, `|`, ` ` (space character), -`,`, `#`, `:` -- Cannot start with `-`, `_`, `+`, `.` +- Cannot include `\`, `/`, `*`, `?`, `"`, `<`, `>`, `|`, `,`, `#`, `:`, or a +space character +- Cannot start with `-`, `_`, `+`, or `.ds-` - Cannot be `.` or `..` -- Cannot be longer than 255 bytes (note it is bytes, so multi-byte characters -will count towards the 255 limit faster) +- Cannot be longer than 255 bytes. Multi-byte characters +count towards this limit faster. +// end::data-stream-name[] -- diff --git a/docs/reference/indices/data-stream-stats.asciidoc b/docs/reference/indices/data-stream-stats.asciidoc index a3773079a28e8..c00f006526d35 100644 --- a/docs/reference/indices/data-stream-stats.asciidoc +++ b/docs/reference/indices/data-stream-stats.asciidoc @@ -72,6 +72,10 @@ To target all data streams in a cluster, omit this parameter or use `*`. [[data-stream-stats-api-query-params]] ==== {api-query-parms-title} +include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=ds-expand-wildcards] ++ +Defaults to `open`. + `human`:: (Optional, Boolean) If `true`, the response includes statistics in human-readable <>. + +`hidden`:: +(Boolean) +If `true`, the data stream is <>. ==== [[get-data-stream-api-example]] diff --git a/docs/reference/indices/put-index-template.asciidoc b/docs/reference/indices/put-index-template.asciidoc index a0b8ee5956793..5a8ff0a1521f1 100644 --- a/docs/reference/indices/put-index-template.asciidoc +++ b/docs/reference/indices/put-index-template.asciidoc @@ -103,12 +103,20 @@ your template is applied instead of the built-in template for `logs-*-*`. [xpack]#`data_stream`#:: (Optional, object) -Indicates whether the template is used to create data streams and their backing -indices. If so, use an empty object as the argument: + -`data_stream: { }`. +If this object is included, the template is used to create data streams and +their backing indices. Supports an empty object: `data_stream: { }` + Data streams require a matching index template with a `data_stream` object. See <>. ++ +.Properties of `data_stream` +[%collapsible%open] +==== +[[data-stream-hidden]] +`hidden`:: +(Optional, Boolean) +If `true`, the data stream is <>. Defaults to `false`. +==== `template`:: (Optional, object) diff --git a/docs/reference/rest-api/common-parms.asciidoc b/docs/reference/rest-api/common-parms.asciidoc index 61d4ef4eaa10a..282ed5471aa52 100644 --- a/docs/reference/rest-api/common-parms.asciidoc +++ b/docs/reference/rest-api/common-parms.asciidoc @@ -215,26 +215,48 @@ Enrich policy name used to limit the request. end::enrich-policy[] +tag::ds-expand-wildcards[] +`expand_wildcards`:: ++ +-- +(Optional, string) +Type of data stream that wildcard expressions can match. Supports +comma-separated values, such as `open,hidden`. Valid values are: + +`all`, `hidden`:: +Match any data stream, including <> ones. + +`open`, `closed`:: +Matches any non-hidden data stream. Data streams cannot be closed. + +`none`:: +Wildcard expressions are not accepted. +-- +end::ds-expand-wildcards[] + tag::expand-wildcards[] `expand_wildcards`:: + -- -(Optional, string) Controls what kind of indices that wildcard expressions can -expand to. Multiple values are accepted when separated by a comma, as in -`open,hidden`. Valid values are: +(Optional, string) +Type of index that wildcard expressions can match. If the request can target +data streams, this argument determines whether wildcard expressions match +hidden data streams. Supports comma-separated values, such as `open,hidden`. +Valid values are: `all`:: -Expand to open and closed indices, including <>. +Match any data stream or index, including <> ones. `open`:: -Expand only to open indices. +Match open, non-hidden indices. Also matches any non-hidden data stream. `closed`:: -Expand only to closed indices. +Match closed, non-hidden indices. Also matches any non-hidden data stream. Data +streams cannot be closed. `hidden`:: -Expansion of wildcards will include hidden indices. -Must be combined with `open`, `closed`, or both. +Match hidden data streams and hidden indices. Must be combined with `open`, +`closed`, or both. `none`:: Wildcard expressions are not accepted.