Skip to content

Commit

Permalink
ILM: restrict customization of alias to suffix. (#3905)
Browse files Browse the repository at this point in the history 
Change how rollover aliases can be configured to only allow
customizing a suffix. This change intends to make the ILM
setup less error prone and to make the transition to the
new index_templates and data streams easier.

Follow up on #3826
closes #3895

Co-authored-by: Brandon Morelli <[email protected]>
  • Loading branch information
@simitt @bmorelli25
simitt and bmorelli25 authored Jun 23, 2020
1 parent ef162f1 commit fc3b62a
Show file tree
Hide file tree
Showing 11 changed files with 124 additions and 115 deletions.
28 changes: 14 additions & 14 deletions _meta/beat.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -382,25 +382,27 @@ apm-server:
# Default value is `true`.
#require_policy: true

# The configured event types, policies and rollover_aliases will be merged with
# the default setup. Only configure the mappings that should be customized.
# NOTICE: when customizing `rollover_alias`, it is strongly recommended to keep
# the prefix "apm-%{[observer.version]}" unchanged;
# in case the prefix is changed, `setup.template.name` and `setup.template.pattern`
# need to be changed accordingly to match the created indices
# Customized mappings will be merged with the default setup, so you only need to configure mappings for the
# event types, policies, and index suffixes that you want to customize.
# Indices are named in this way: `apm-%{[observer.version]}-%{[event.type]}-{index_suffix}`,
# e.g., apm-7.9.0-span-custom*. The `index_suffix` is optional.
# NOTE: When configuring an `index_suffix`, ensure that no previously set up templates conflict with the
# newly configured ones. If an index matches multiple templates with the same order, the settings of
# the templates will override each other. Any conflicts need to be cleaned up manually.
# NOTE: When customizing `setup.template.name` and `setup.template.pattern`, ensure they still match the indices.
#mapping:
#- event_type: "error"
# policy_name: "apm-rollover-30-days"
# rollover_alias: "apm-%{[observer.version]}-error"
# index_suffix: ""
#- event_type: "span"
# policy_name: "apm-rollover-30-days"
# rollover_alias: "apm-%{[observer.version]}-span"
# index_suffix: ""
#- event_type: "transaction"
# policy_name: "apm-rollover-30-days"
# rollover_alias: "apm-%{[observer.version]}-transaction"
# index_suffix: ""
#- event_type: "metric"
# policy_name: "apm-rollover-30-days"
# rollover_alias: "apm-%{[observer.version]}-metric"
# index_suffix: ""

# Configured policies are added to pre-defined default policies.
# If a policy with the same name as a default policy is configured, the configured policy overwrites the default policy.
Expand Down Expand Up @@ -493,14 +495,12 @@ apm-server:
#setup.template.enabled: true

# Template name. By default the template name is "apm-%{[observer.version]}"
# The template name and pattern has to be set in case the elasticsearch index pattern is modified,
# or the `apm-server.ilm.setup.mapping` rollover_aliases are changed.
# The template name and pattern has to be set in case the elasticsearch index pattern is modified.
#setup.template.name: "apm-%{[observer.version]}"

# Template pattern. By default the template pattern is "apm-%{[observer.version]}-*" to apply to the default index settings.
# The first part is the version of apm-server and then -* is used to match all daily indices.
# The template name and pattern has to be set in case the elasticsearch index pattern is modified,
# or the `apm-server.ilm.setup.mapping` rollover_aliases are changed.
# The template name and pattern has to be set in case the elasticsearch index pattern is modified.
#setup.template.pattern: "apm-%{[observer.version]}-*"

# Path to fields.yml file to generate the template.
Expand Down
28 changes: 14 additions & 14 deletions apm-server.docker.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -382,25 +382,27 @@ apm-server:
# Default value is `true`.
#require_policy: true

# The configured event types, policies and rollover_aliases will be merged with
# the default setup. Only configure the mappings that should be customized.
# NOTICE: when customizing `rollover_alias`, it is strongly recommended to keep
# the prefix "apm-%{[observer.version]}" unchanged;
# in case the prefix is changed, `setup.template.name` and `setup.template.pattern`
# need to be changed accordingly to match the created indices
# Customized mappings will be merged with the default setup, so you only need to configure mappings for the
# event types, policies, and index suffixes that you want to customize.
# Indices are named in this way: `apm-%{[observer.version]}-%{[event.type]}-{index_suffix}`,
# e.g., apm-7.9.0-span-custom*. The `index_suffix` is optional.
# NOTE: When configuring an `index_suffix`, ensure that no previously set up templates conflict with the
# newly configured ones. If an index matches multiple templates with the same order, the settings of
# the templates will override each other. Any conflicts need to be cleaned up manually.
# NOTE: When customizing `setup.template.name` and `setup.template.pattern`, ensure they still match the indices.
#mapping:
#- event_type: "error"
# policy_name: "apm-rollover-30-days"
# rollover_alias: "apm-%{[observer.version]}-error"
# index_suffix: ""
#- event_type: "span"
# policy_name: "apm-rollover-30-days"
# rollover_alias: "apm-%{[observer.version]}-span"
# index_suffix: ""
#- event_type: "transaction"
# policy_name: "apm-rollover-30-days"
# rollover_alias: "apm-%{[observer.version]}-transaction"
# index_suffix: ""
#- event_type: "metric"
# policy_name: "apm-rollover-30-days"
# rollover_alias: "apm-%{[observer.version]}-metric"
# index_suffix: ""

# Configured policies are added to pre-defined default policies.
# If a policy with the same name as a default policy is configured, the configured policy overwrites the default policy.
Expand Down Expand Up @@ -493,14 +495,12 @@ apm-server:
#setup.template.enabled: true

# Template name. By default the template name is "apm-%{[observer.version]}"
# The template name and pattern has to be set in case the elasticsearch index pattern is modified,
# or the `apm-server.ilm.setup.mapping` rollover_aliases are changed.
# The template name and pattern has to be set in case the elasticsearch index pattern is modified.
#setup.template.name: "apm-%{[observer.version]}"

# Template pattern. By default the template pattern is "apm-%{[observer.version]}-*" to apply to the default index settings.
# The first part is the version of apm-server and then -* is used to match all daily indices.
# The template name and pattern has to be set in case the elasticsearch index pattern is modified,
# or the `apm-server.ilm.setup.mapping` rollover_aliases are changed.
# The template name and pattern has to be set in case the elasticsearch index pattern is modified.
#setup.template.pattern: "apm-%{[observer.version]}-*"

# Path to fields.yml file to generate the template.
Expand Down
28 changes: 14 additions & 14 deletions apm-server.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -382,25 +382,27 @@ apm-server:
# Default value is `true`.
#require_policy: true

# The configured event types, policies and rollover_aliases will be merged with
# the default setup. Only configure the mappings that should be customized.
# NOTICE: when customizing `rollover_alias`, it is strongly recommended to keep
# the prefix "apm-%{[observer.version]}" unchanged;
# in case the prefix is changed, `setup.template.name` and `setup.template.pattern`
# need to be changed accordingly to match the created indices
# Customized mappings will be merged with the default setup, so you only need to configure mappings for the
# event types, policies, and index suffixes that you want to customize.
# Indices are named in this way: `apm-%{[observer.version]}-%{[event.type]}-{index_suffix}`,
# e.g., apm-7.9.0-span-custom*. The `index_suffix` is optional.
# NOTE: When configuring an `index_suffix`, ensure that no previously set up templates conflict with the
# newly configured ones. If an index matches multiple templates with the same order, the settings of
# the templates will override each other. Any conflicts need to be cleaned up manually.
# NOTE: When customizing `setup.template.name` and `setup.template.pattern`, ensure they still match the indices.
#mapping:
#- event_type: "error"
# policy_name: "apm-rollover-30-days"
# rollover_alias: "apm-%{[observer.version]}-error"
# index_suffix: ""
#- event_type: "span"
# policy_name: "apm-rollover-30-days"
# rollover_alias: "apm-%{[observer.version]}-span"
# index_suffix: ""
#- event_type: "transaction"
# policy_name: "apm-rollover-30-days"
# rollover_alias: "apm-%{[observer.version]}-transaction"
# index_suffix: ""
#- event_type: "metric"
# policy_name: "apm-rollover-30-days"
# rollover_alias: "apm-%{[observer.version]}-metric"
# index_suffix: ""

# Configured policies are added to pre-defined default policies.
# If a policy with the same name as a default policy is configured, the configured policy overwrites the default policy.
Expand Down Expand Up @@ -493,14 +495,12 @@ apm-server:
#setup.template.enabled: true

# Template name. By default the template name is "apm-%{[observer.version]}"
# The template name and pattern has to be set in case the elasticsearch index pattern is modified,
# or the `apm-server.ilm.setup.mapping` rollover_aliases are changed.
# The template name and pattern has to be set in case the elasticsearch index pattern is modified.
#setup.template.name: "apm-%{[observer.version]}"

# Template pattern. By default the template pattern is "apm-%{[observer.version]}-*" to apply to the default index settings.
# The first part is the version of apm-server and then -* is used to match all daily indices.
# The template name and pattern has to be set in case the elasticsearch index pattern is modified,
# or the `apm-server.ilm.setup.mapping` rollover_aliases are changed.
# The template name and pattern has to be set in case the elasticsearch index pattern is modified.
#setup.template.pattern: "apm-%{[observer.version]}-*"

# Path to fields.yml file to generate the template.
Expand Down
2 changes: 1 addition & 1 deletion changelogs/head.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ https://github.com/elastic/apm-server/compare/7.8\...master[View commits]
[float]
==== Added
* Support configurable response headers for the RUM endpoints {pull}3820[3820]
* Support custom ILM rollover aliases {pull}3826[3826]
* Support custom ILM index suffix {pull}3826[3826],{pull}3905[3905]
* Jaeger traceIds/spanIds are formatted without leading zeros {pull}3849[3849]
* Index Page URL and referer as ECS fields {pull}3857[3857]
* Map the Jaeger attribute message_bus.destination to the Elastic APM type messaging {pull}3884[3884]
Expand Down
17 changes: 9 additions & 8 deletions docs/ilm-reference.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -51,16 +51,16 @@ apm-server:
mapping:
- event_type: "error"
policy_name: "apm-rollover-30-days"
rollover_alias: "apm-%{[observer.version]}-error"
index_suffix: ""
- event_type: "span"
policy_name: "apm-rollover-30-days"
rollover_alias: "apm-%{[observer.version]}-span"
index_suffix: ""
- event_type: "transaction"
policy_name: "apm-rollover-30-days"
rollover_alias: "apm-%{[observer.version]}-transaction"
index_suffix: ""
- event_type: "metric"
policy_name: "apm-rollover-30-days"
rollover_alias: "apm-%{[observer.version]}-metric"
index_suffix: ""
policies:
- name: "apm-rollover-30-days"
policy:
Expand Down Expand Up @@ -213,11 +213,12 @@ APM Server will still make use of ILM and connect your template with the defined
[[ilm-setup-mapping-config]]
===== `apm-server.ilm.setup.mapping`

Maps each event type to a corresponding `policy_name` and `rollover_alias`.
Maps each event type to a corresponding `policy_name` and `index_suffix`.
APM event types can only be `error`, `span`, `transaction`, and `metric`.
If you attempt to map an index lifecycle policy to a different event type, APM Server will not start.
If you only map a subset of APM event types, the default values will be used for omitted event types.
If a policy is defined, it must be mapped to an event type to take effect.
When customizing `rollover_alias`, it is strongly recommended to keep the prefix `apm-%{[observer.version]}` unchanged;
if the prefix must be changed, `setup.template.name` and `setup.template.pattern` need to be changed accordingly,
to match the created indices.

By default, the APM Server creates a template without a custom index suffix per event type. When defining custom
index suffixes, always ensure that templates, that might have been set up previously, are removed or do not conflict.
See <<custom-ilm-index-suffix,customizing an index suffix>> for more information.
72 changes: 41 additions & 31 deletions docs/ilm.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ All of this can be done directly from the `apm-server.yml` file.

* <<ilm-enable>>
* <<ilm-setup>>
* <<custom-ilm-alias>>
* <<custom-ilm-index-suffix>>
* <<custom-ilm-policy>>
* <<example-ilm-config>>
* <<map-ilm-policy>>
Expand Down Expand Up @@ -61,29 +61,42 @@ apm-server:
ILM can now be customized via the remaining `apm-server.ilm.setup.*` configuration options.

[float]
[[custom-ilm-alias]]
==== Create a custom ILM `rollover_alias`

You can define a custom rollover alias per event type. The rollover alias can only contain a
restricted set of variables, limited to variables concerning `observer.*`. In case other
variables are configured, the server will refuse to start.
The related ILM policy can specify a rollover action, e.g. when an index reaches a size of 50GB
it should roll over to create a new index. The `rollover_alias` abstracts away the concrete write index, while
[[custom-ilm-index-suffix]]
==== Create a custom ILM `index_suffix`

You can define a custom index suffix for each event type. The index suffix is limited to variables
concerning `observer.*`; if other variables are configured, the server will refuse to start.
The resulting rollover alias and index name will be of type `apm-{version}-{event_type}-{custom_index_suffix},
e.g., `apm-7.9.0-span-foo`.
The mapped ILM policy can specify a rollover action, e.g., when an index reaches a size of 50GB,
it should roll over to create a new index. The rollover alias abstracts away the concrete write index, while
automatically being updated and keeping track of the current write index.
This allows to use the `rollover_alias` instead of specific indices in queries.
This allows you to use the rollover alias instead of specific indices in queries.

For example when defining a rollover alias with name `apm-7.8.0-transaction`, an alias pointing to a write index
`apm-7.8.0-transaction-000001` is created. When the index reaches the defined size or age it will roll over to
a new index `apm-7.8.0-transaction-000002`. The rollover alias `apm-7.8.0-transaction` keeps
track of which index is the write index when ingesting data.
For example, the default rollover alias for event type `transaction` would be `apm-7.8.0-transaction`. This alias points
to a write index named `apm-7.8.0-transaction-000001`. When this index reaches its defined size or age, it will roll over to
a new index named `apm-7.8.0-transaction-000002`. The rollover alias of `apm-7.8.0-transaction` keeps
track of which index is the current write index while ingesting data.

NOTE: Ensure that the configured `rollover_alias` for all event types start with the same prefix,
and that this prefix is also configured for `setup.template.name` and `setup.template.pattern`.
If the template configuration is not accordingly changed, the Elasticsearch index template with
the predefined mappings will not match against created indices, leading to indexing issues.
NOTE: Manually ensure that templates containing customized index information do not conflict with each other
or the default templates.

The example below shows how to change the `rollover_alias` to a custom value,
and set the template configuration to a matching name and pattern.
By default, the APM Server creates a template without a custom index suffix per event type. When defining custom
index suffixes, always ensure that templates, that might have been set up previously, are removed or do not conflict.
A conflicting behavior could occur when an index matches multiple templates with the same `order`.
For example, the APM Server was started without any customization, leading to a default index setup. Afterward, the
Server configuration is customized to add the index suffix `production` for the event type `span`.
When the Server restarts, it will set up a new index template based on the new custom index suffix.
A newly created index named `apm-server-7.9.0-span-production` would now match the default template with the index pattern of
`apm-server-7.9.0-span*`, but also the new template with the index pattern `apm-server-7.9.0-span-production*`.
In this case, the old template needs to be manually deleted,
or the `index_pattern` or `order` need to be changed to avoid conflicts.

NOTE: If you customize `setup.template.pattern`, ensure that the configured pattern
still matches the rollover aliases. If it doesn't, the Elasticsearch index template with the predefined mappings will
not match against created indices, leading to indexing issues.

The example below shows how to change the `index_suffix` to a custom value.

[source,yml]
----
Expand All @@ -93,16 +106,13 @@ apm-server:
setup:
mapping:
- event_type: "error"
rollover_alias: "apm-dev-error"
index_suffix: "dev"
- event_type: "span"
rollover_alias: "apm-dev-span"
index_suffix: "dev"
- event_type: "transaction"
rollover_alias: "apm-dev-transaction"
index_suffix: "dev"
- event_type: "metric"
rollover_alias: "apm-dev-metric"
setup.template.name: "apm-dev"
setup.template.pattern: "apm-dev*"
index_suffix: "dev"
----

[float]
Expand Down Expand Up @@ -253,16 +263,16 @@ Additionally this example shows how to set custom rollover aliases.
mapping:
- event_type: "error"
policy_name: "apm-error-span-policy"
rollover_alias: "apm-%{[observer.version]}-development-error"
index_suffix: "development"
- event_type: "span"
policy_name: "apm-error-span-policy"
rollover_alias: "apm-%{[observer.version]}-development-span"
index_suffix: "development"
- event_type: "transaction"
policy_name: "apm-transaction-metric-policy"
rollover_alias: "apm-%{[observer.version]}-development-transaction"
index_suffix: "development"
- event_type: "metric"
policy_name: "apm-transaction-metric-policy"
rollover_alias: "apm-%{[observer.version]}-development-metric"
index_suffix: "development"
enabled: true
policies:
- name: "apm-error-span-policy"
Expand Down
Loading

0 comments on commit fc3b62a

Please sign in to comment.