Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch add-block-index-to-close, add-index-block and finalize-index-block tasks #81627

Closed
Tracked by #77466
DaveCTurner opened this issue Dec 10, 2021 · 2 comments
Closed
Tracked by #77466
Assignees
Labels
>bug :Data Management/Indices APIs APIs to create and manage indices and templates Team:Data Management Meta label for data/management team

Comments

@DaveCTurner
Copy link
Contributor

The process of closing an index involves multiple cluster state updates to add and finalise its blocks; there are also other reasons for adding blocks to an index. ILM may close an index to adjust its codec prior to force-merging, and may apply a write block for other reasons. Today the associated cluster state update tasks run at priority URGENT and are not batched, so can take a very long time to process and gum up other things.

I encountered a cluster that was processing a few thousand such tasks at a rate of about 20 per minute.

We should batch these tasks together to avoid this causing a problem in future.

Relates #77466

@DaveCTurner DaveCTurner added >bug :Data Management/Indices APIs APIs to create and manage indices and templates labels Dec 10, 2021
@elasticmachine elasticmachine added the Team:Data Management Meta label for data/management team label Dec 10, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

@joegallo
Copy link
Contributor

joegallo commented Mar 8, 2022

Handled by way of #83760, #84259, and #84374.

@joegallo joegallo closed this as completed Mar 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Data Management/Indices APIs APIs to create and manage indices and templates Team:Data Management Meta label for data/management team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants