Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ILM] Mechanism to bypass rollover and move to next action for indices where indexing is complete #35944

Closed
colings86 opened this issue Nov 27, 2018 · 3 comments
Assignees
Labels
:Data Management/ILM+SLM Index and Snapshot lifecycle management >feature

Comments

@colings86
Copy link
Contributor

colings86 commented Nov 27, 2018

At the moment ILM will wait indefinitely for the rollover conditions to be met before allowing the index to progress on its lifecycle. This is exactly what we should expect to happen in most cases but sometimes the user will decommission an index (stop indexing to it entirely). This can be for a few reasons:

  • The user is no longer indexing from that source
  • The user has changed indexing strategy and is indexing to a completely different index pattern with a different alias now

In these cases there is currently no good way for the user to indicate that the index is "complete" and it should continue to move through its lifecycle.

One idea here would be to introduce a new index setting named something like index.lifecycle.index_complete which the rollover steps can look at to determine that a rollover is not needed and they can return complete without checking the rollover conditions or performing rollover. The check could error if the write alias is still present on the index so ILM has some confidence that the index is not being used.

This setting could also be used in the CCR use-case to indicate that the leader index has been rolled over so when replicated to the follower index the follower would know to move passed the hot phase when the following index has caught up with the leader.

@colings86 colings86 added >feature :Data Management/ILM+SLM Index and Snapshot lifecycle management labels Nov 27, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features

@jakelandis
Copy link
Contributor

Blocked by #36620

@gwbrown
Copy link
Contributor

gwbrown commented Dec 28, 2018

As #36235 and #36620 have been merged (+ backported appropriately), we can close this issue now.

@gwbrown gwbrown closed this as completed Dec 28, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Data Management/ILM+SLM Index and Snapshot lifecycle management >feature
Projects
None yet
Development

No branches or pull requests

4 participants