Adjust /_cat/templates not to request all metadata #78832
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Today `GET /_cat/templates` retrieves the whole cluster metadata from the master, which includes all sorts of unnecessary junk and consumes significant resources. This commit reimplements these endpoints using `GetIndexTemplatesAction` and `GetComposableIndexTemplateAction` which are much more efficient. The docs for this API indicate that it accepts a comma-separated list of template names/patterns of the form `GET /_cat/templates/name1,name2` but in fact today it only accepts a single name or pattern. This commit also adds support for multiple names/patterns as the docs claim. Backport of elastic#78829
DaveCTurner
added
>bug
:Data Management/CAT APIs
|
Pinging @elastic/es-data-management (Team:Data Management) |
|
@elasticmachine test this please |
DaveCTurner
deleted the
2021-10-07-cat-templates-without-cluster-state-redux-7x
branch
tvernum
added a commit
to tvernum/elasticsearch
that referenced
this pull request
Dec 23, 2021
All three template types (legacy templates, composable index templates and component templates) are stored in cluster state metadata (in fields "templates", "index_template" and "component_template"). This cluster state is readable (via GET /_cluster/state) for users who have the monitor privilege at the cluster level. However, calling the explicit read endpoints for these templates required the manage_index_templates privilege. This change grants access to the template specific retrieval APIs for all users (or API Keys) with the cluster monitor privilge so that they can make use of these fit-for-purpose APIs instead of parsing data directly from cluster metadata Relates: elastic/beats#29554, elastic#78832
elasticsearchmachine
pushed a commit
that referenced
this pull request
Jan 5, 2022
All three template types (legacy templates, composable index templates and component templates) are stored in cluster state metadata (in fields "templates", "index_template" and "component_template"). This cluster state is readable (via GET /_cluster/state) for users who have the monitor privilege at the cluster level. However, calling the explicit read endpoints for these templates required the manage_index_templates privilege. This change grants access to the template specific retrieval APIs for all users (or API Keys) with the cluster monitor privilege so that they can make use of these fit-for-purpose APIs instead of parsing data directly from cluster metadata Relates: elastic/beats#29554 Relates: #78832
tvernum
added a commit
to tvernum/elasticsearch
that referenced
this pull request
Jan 5, 2022
All three template types (legacy templates, composable index templates and component templates) are stored in cluster state metadata (in fields "templates", "index_template" and "component_template"). This cluster state is readable (via GET /_cluster/state) for users who have the monitor privilege at the cluster level. However, calling the explicit read endpoints for these templates required the manage_index_templates privilege. This change grants access to the template specific retrieval APIs for all users (or API Keys) with the cluster monitor privilege so that they can make use of these fit-for-purpose APIs instead of parsing data directly from cluster metadata Relates: elastic/beats#29554 Relates: elastic#78832
tvernum
added a commit
to tvernum/elasticsearch
that referenced
this pull request
Jan 5, 2022
All three template types (legacy templates, composable index templates and component templates) are stored in cluster state metadata (in fields "templates", "index_template" and "component_template"). This cluster state is readable (via GET /_cluster/state) for users who have the monitor privilege at the cluster level. However, calling the explicit read endpoints for these templates required the manage_index_templates privilege. This change grants access to the template specific retrieval APIs for all users (or API Keys) with the cluster monitor privilge so that they can make use of these fit-for-purpose APIs instead of parsing data directly from cluster metadata Relates: elastic/beats#29554, elastic#78832 Backport of: elastic#82046
elasticsearchmachine
pushed a commit
that referenced
this pull request
Jan 5, 2022
All three template types (legacy templates, composable index templates and component templates) are stored in cluster state metadata (in fields "templates", "index_template" and "component_template"). This cluster state is readable (via GET /_cluster/state) for users who have the monitor privilege at the cluster level. However, calling the explicit read endpoints for these templates required the manage_index_templates privilege. This change grants access to the template specific retrieval APIs for all users (or API Keys) with the cluster monitor privilege so that they can make use of these fit-for-purpose APIs instead of parsing data directly from cluster metadata Relates: elastic/beats#29554 Relates: #78832
tvernum
added a commit
that referenced
this pull request
Jan 5, 2022
All three template types (legacy templates, composable index templates and component templates) are stored in cluster state metadata (in fields "templates", "index_template" and "component_template"). This cluster state is readable (via GET /_cluster/state) for users who have the monitor privilege at the cluster level. However, calling the explicit read endpoints for these templates required the manage_index_templates privilege. This change grants access to the template specific retrieval APIs for all users (or API Keys) with the cluster monitor privilge so that they can make use of these fit-for-purpose APIs instead of parsing data directly from cluster metadata Relates: elastic/beats#29554, #78832 Backport of: #82046
astefan
pushed a commit
to astefan/elasticsearch
that referenced
this pull request
Jan 7, 2022
All three template types (legacy templates, composable index templates and component templates) are stored in cluster state metadata (in fields "templates", "index_template" and "component_template"). This cluster state is readable (via GET /_cluster/state) for users who have the monitor privilege at the cluster level. However, calling the explicit read endpoints for these templates required the manage_index_templates privilege. This change grants access to the template specific retrieval APIs for all users (or API Keys) with the cluster monitor privilege so that they can make use of these fit-for-purpose APIs instead of parsing data directly from cluster metadata Relates: elastic/beats#29554 Relates: elastic#78832
astefan
pushed a commit
to astefan/elasticsearch
that referenced
this pull request
Jan 7, 2022
All three template types (legacy templates, composable index templates and component templates) are stored in cluster state metadata (in fields "templates", "index_template" and "component_template"). This cluster state is readable (via GET /_cluster/state) for users who have the monitor privilege at the cluster level. However, calling the explicit read endpoints for these templates required the manage_index_templates privilege. This change grants access to the template specific retrieval APIs for all users (or API Keys) with the cluster monitor privilege so that they can make use of these fit-for-purpose APIs instead of parsing data directly from cluster metadata Relates: elastic/beats#29554 Relates: elastic#78832
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Today
GET /_cat/templatesretrieves the whole cluster metadata fromthe master, which includes all sorts of unnecessary junk and consumes
significant resources. This commit reimplements these endpoints using
GetIndexTemplatesActionandGetComposableIndexTemplateActionwhichare much more efficient.
The docs for this API indicate that it accepts a comma-separated list of
template names/patterns of the form
GET /_cat/templates/name1,name2but in fact today it only accepts a single name or pattern. This commit
also adds support for multiple names/patterns as the docs claim.
Backport of #78829