Downsample action can attempt to downsample on the same interval and delete data #102249
Labels
>bug
:Data Management/ILM+SLM
Index and Snapshot lifecycle management
Team:Data Management
Meta label for data/management team
v8.11.2
v8.12.0
Elasticsearch Version
8.10, 8.11, 8.12
Installed Plugins
No response
Java Version
bundled
OS Version
Darwin
Problem Description
When an ILM policy is created we don't allow the same donwsample action (same interval) to be configured in subsequent phases.
This can be achieved though by updating the ILM policy by removing it from the hot phase and configuring it with the same
fixed_interval
in the warm phase.This can cause some in-flight indices that were downsampled in hot to now be downsampled in warm for the same interval.
ILM will go through the warm/downsample action, skip the downsample operation (as the source index is donwsampled already) and delete the source index. This leads to data loss.
The exact use case is “update the next phase to contain the downsample action from the previous phase, whilst the managed index is waiting to enter the next phase”.
Steps to Reproduce
Logs (if relevant)
No response
The text was updated successfully, but these errors were encountered: