Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a setting to disable stack installed templates #62835

Closed
dakrone opened this issue Sep 23, 2020 · 5 comments · Fixed by #63764
Closed

Add a setting to disable stack installed templates #62835

dakrone opened this issue Sep 23, 2020 · 5 comments · Fixed by #63764
Assignees
Labels
>enhancement Team:Data Management Meta label for data/management team

Comments

@dakrone
Copy link
Member

dakrone commented Sep 23, 2020

From #57629 onward we have started installing logs, metrics, and synthetics templates that match the logs-*-*, metrics-*-*, and synthetics-*-* index patterns.

An issue arises when a user upgrades their cluster to a version that includes these templates while also using an index pattern that would coincide with these (such as a user having daily indices named logs-data-things-2020-09-23 for example).

In order to mitigate these issues, we should add a node level setting that disables putting these templates in place automatically. The setting should also be dynamic (so as not to require restarting all the nodes if a user needs to get rid of the templates), and we should also better document the stack plugin, the built in templates, and potential issues that may arrive when a user upgrades.

@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features (:Core/Features/Features)

@elasticmachine elasticmachine added the Team:Data Management Meta label for data/management team label Sep 23, 2020
@kobelb
Copy link
Contributor

kobelb commented Sep 30, 2020

Should we be installing these templates by default? This feels like a breaking change, where users that were previously using Elasticsearch in a legitimate manner by having indices named logs-data-things-2020-09-23 are now unable to upgrade without changing a setting.

@mbarretta
Copy link

mbarretta commented Oct 1, 2020

I've seen this occur already with a metrics-* index template, and yes, it broke their system and forced a rename of the new metrics-*-* template to metrics-kill-9-* (he had a sense of humor). A rename was required because they cannot be deleted.

As opposed to a node-level setting, can something first prompt that a collision will occur and/or force a positive confirmation to proceed? Otherwise, it puts the onus on users to read through the release/upgrade notes, notice the potential collision, and make the setting change (++ dynamic).

Failing that, the change is made and their own index templates will silently fail to apply.

@dakrone dakrone self-assigned this Oct 15, 2020
dakrone added a commit to dakrone/elasticsearch that referenced this issue Oct 15, 2020
This change allows the setting for disabling the automatically installed stack templates (the
`logs-*-*`, `metrics-*-*`, and `synthetics-*-*` templates) to be changed dynamically.

As a byproduct, it also moves thes `IndexTemplateRegistry` to use an `initialize()` method so that
constructors are not tempted to use a `this` reference in the constructor (see elastic#37861 for more
information about why to avoid that).

Resolves elastic#62835
@pugnascotia
Copy link
Contributor

This is awkward for writing deprecation logs to a data stream, since that functionality expects the logs-*-* template to exist and relies on it.

@dakrone
Copy link
Member Author

dakrone commented Oct 16, 2020

@pugnascotia we currently already have the setting (#63764 just makes it dynamic), so we cannot remove it, I would suggest maybe that the deprecation log data stream install its own template (with a more narrow index pattern and higher priority) so that it's independent of the stack templates

dakrone added a commit that referenced this issue Oct 16, 2020
* Make stack.templates.enabled a dynamic setting

This change allows the setting for disabling the automatically installed stack templates (the
`logs-*-*`, `metrics-*-*`, and `synthetics-*-*` templates) to be changed dynamically.

As a byproduct, it also moves thes `IndexTemplateRegistry` to use an `initialize()` method so that
constructors are not tempted to use a `this` reference in the constructor (see #37861 for more
information about why to avoid that).

Resolves #62835
dakrone added a commit to dakrone/elasticsearch that referenced this issue Oct 16, 2020
* Make stack.templates.enabled a dynamic setting

This change allows the setting for disabling the automatically installed stack templates (the
`logs-*-*`, `metrics-*-*`, and `synthetics-*-*` templates) to be changed dynamically.

As a byproduct, it also moves thes `IndexTemplateRegistry` to use an `initialize()` method so that
constructors are not tempted to use a `this` reference in the constructor (see elastic#37861 for more
information about why to avoid that).

Resolves elastic#62835
dakrone added a commit that referenced this issue Oct 16, 2020
* Make stack.templates.enabled a dynamic setting

This change allows the setting for disabling the automatically installed stack templates (the
`logs-*-*`, `metrics-*-*`, and `synthetics-*-*` templates) to be changed dynamically.

As a byproduct, it also moves thes `IndexTemplateRegistry` to use an `initialize()` method so that
constructors are not tempted to use a `this` reference in the constructor (see #37861 for more
information about why to avoid that).

Resolves #62835
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>enhancement Team:Data Management Meta label for data/management team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants