[ILM] Fix downsample to skip already downsampled indices #102250
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This fixes a bug in ILM where if an ILM policy is updated such that the next phase (e.g. warm) contains the downsample action from the previous phase (e.g. hot), whilst the managed index is waiting to enter the next phase (e.g. warm), if the managed index was already downsampled in hot it would get deleted in the warm phase.
andreidan
added
>bug
:Data Management/ILM+SLM
elasticsearchmachine
added
the
Team:Data Management
|
Hi @andreidan, I've created a changelog YAML for you. |
|
Pinging @elastic/es-data-management (Team:Data Management) |
dakrone
reviewed
Nov 15, 2023
Comment on lines
+169
to
+181
| if (downsampleStatus == IndexMetadata.DownsampleTaskStatus.UNKNOWN) { | ||
| // This isn't a downsample index, but it has the name of our target downsample index - very bad, we'll skip the | ||
| // downsample action to avoid blocking the lifecycle of this index - if there | ||
| // is another downsample action configured in the next phase, it'll be able to proceed successfully | ||
| logger.warn( | ||
| "index [{}] as part of policy [{}] cannot be downsampled at interval [{}] in phase [{}] because it has" | ||
| + " the name of the target downsample index and is itself not a downsampled index. Skipping the downsample " | ||
| + "action.", | ||
| index.getName(), | ||
| indexMetadata.getLifecyclePolicyName(), | ||
| fixedInterval, | ||
| phase | ||
| ); |
There was a problem hiding this comment.
I don't understand, why do we only log this warning if the status is unknown, if the status is successful we still skip this downsample, but we don't log about it. Shouldn't we log in all cases then?
There was a problem hiding this comment.
If the status is successful we don't log anything because this action (even as a no-op) has achieve its goal - the index is downsampled at the desired /configured interval.
If the index has the shape of a downsample index, but it is not (i.e. status is UNKNOWN) we're skipping this action without achieving its goal - i.e. the data is not downsampled. We're logging the warning that we haven't executed downsampling.
There was a problem hiding this comment.
That makes sense, thanks for the explanation
dakrone
approved these changes
Nov 15, 2023
Comment on lines
+169
to
+181
| if (downsampleStatus == IndexMetadata.DownsampleTaskStatus.UNKNOWN) { | ||
| // This isn't a downsample index, but it has the name of our target downsample index - very bad, we'll skip the | ||
| // downsample action to avoid blocking the lifecycle of this index - if there | ||
| // is another downsample action configured in the next phase, it'll be able to proceed successfully | ||
| logger.warn( | ||
| "index [{}] as part of policy [{}] cannot be downsampled at interval [{}] in phase [{}] because it has" | ||
| + " the name of the target downsample index and is itself not a downsampled index. Skipping the downsample " | ||
| + "action.", | ||
| index.getName(), | ||
| indexMetadata.getLifecyclePolicyName(), | ||
| fixedInterval, | ||
| phase | ||
| ); |
There was a problem hiding this comment.
That makes sense, thanks for the explanation
andreidan
added
the
auto-merge-without-approval
💚 Backport successful
|
andreidan
added a commit
to andreidan/elasticsearch
that referenced
this pull request
Nov 15, 2023
) This fixes a bug in ILM where if an ILM policy is updated such that the next phase (e.g. warm) contains the downsample action from the previous phase (e.g. hot), whilst the managed index is waiting to enter the next phase (e.g. warm), if the managed index was already downsampled in hot it would get deleted in the warm phase. Fixes elastic#102249
elasticsearchmachine
pushed a commit
that referenced
this pull request
Nov 15, 2023
…102251) This fixes a bug in ILM where if an ILM policy is updated such that the next phase (e.g. warm) contains the downsample action from the previous phase (e.g. hot), whilst the managed index is waiting to enter the next phase (e.g. warm), if the managed index was already downsampled in hot it would get deleted in the warm phase. Fixes #102249
martijnvg
reviewed
Nov 16, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This fixes a bug in ILM where if an ILM policy is updated such that the next phase (e.g. warm) contains the downsample action from the previous phase (e.g. hot), whilst the managed index is waiting to enter the next phase (e.g. warm), if the managed index was already downsampled in hot it would get deleted in the warm phase.
Fixes #102249