[ILM] Error moving to the phase complete step when phase was deleted

[andreidan]

Elasticsearch Version

7.17, 8.6, 8.5

Installed Plugins

No response

Java Version

bundled

OS Version

Darwin

Problem Description

Some ILM steps are implicit ie. injected (migrate action) or the policy/phase complete step.

ILM tries to honour the cached ILM phase however, these implicit steps are not seen as cached so might still end up with errors like:

step [{"phase":"warm","action":"complete","name":"complete"}] for index [index] with policy [my-policy] does not exist

or

step [{"phase":"warm","action":"migrate","name":"check-migration"}] for index [index] with policy [my-policy] does not exist

The problem lies here where we only look at the the explicitly declared actions in the cached phase

Steps to Reproduce

public void testValidateTransitionToCachedStepWhenMissingPhaseFromPolicy() {
        // we'll test the case when the warm phase was deleted and the next step is the phase complete one

        LifecycleExecutionState.Builder executionState = LifecycleExecutionState.builder()
            .setPhase("warm")
            .setAction("migrate")
            .setStep("check-migration")
            .setPhaseDefinition("""
                {
                        "policy" : "my-policy",
                        "phase_definition" : {
                          "min_age" : "20m",
                          "actions" : {
                            "set_priority" : {
                              "priority" : 150
                            }
                          }
                        },
                        "version" : 1,
                        "modified_date_in_millis" : 1578521007076
                      }""");

        IndexMetadata meta = buildIndexMetadata("my-policy", executionState);

        try (Client client = new NoOpClient(getTestName())) {
            Step.StepKey currentStepKey = new Step.StepKey("warm", MigrateAction.NAME, DataTierMigrationRoutedStep.NAME);
            Step.StepKey nextStepKey = new Step.StepKey("warm", PhaseCompleteStep.NAME, PhaseCompleteStep.NAME);

            Step.StepKey waitForRolloverStepKey = new Step.StepKey("hot", RolloverAction.NAME, WaitForRolloverReadyStep.NAME);
            Step.StepKey rolloverStepKey = new Step.StepKey("hot", RolloverAction.NAME, RolloverStep.NAME);
            Step waitForRolloverReadyStep = new WaitForRolloverReadyStep(
                waitForRolloverStepKey,
                rolloverStepKey,
                client,
                null,
                null,
                null,
                1L,
                null,
                null,
                null,
                null,
                null,
                null
            );

            try {
                IndexLifecycleTransition.validateTransition(
                    meta,
                    currentStepKey,
                    nextStepKey,
                    createOneStepPolicyStepRegistry("my-policy", waitForRolloverReadyStep)
                );
            } catch (Exception e) {
                logger.error(e.getMessage(), e);
                fail("validateTransition should not throw exception on valid transitions");
            }
        }
    }

    public void testValidateTransitionToInjectedMissingStep() {
        // we'll test the case when the warm phase was deleted and the next step is an injected one

        LifecycleExecutionState.Builder executionState = LifecycleExecutionState.builder()
            .setPhase("warm")
            .setAction("migrate")
            .setStep("migrate")
            .setPhaseDefinition("""
                {
                        "policy" : "my-policy",
                        "phase_definition" : {
                          "min_age" : "20m",
                          "actions" : {
                            "set_priority" : {
                              "priority" : 150
                            }
                          }
                        },
                        "version" : 1,
                        "modified_date_in_millis" : 1578521007076
                      }""");

        IndexMetadata meta = buildIndexMetadata("my-policy", executionState);

        try (Client client = new NoOpClient(getTestName())) {
            Step.StepKey currentStepKey = new Step.StepKey("warm", MigrateAction.NAME, MigrateAction.NAME);
            Step.StepKey nextStepKey = new Step.StepKey("warm", MigrateAction.NAME, DataTierMigrationRoutedStep.NAME);

            Step.StepKey waitForRolloverStepKey = new Step.StepKey("hot", RolloverAction.NAME, WaitForRolloverReadyStep.NAME);
            Step.StepKey rolloverStepKey = new Step.StepKey("hot", RolloverAction.NAME, RolloverStep.NAME);
            Step waitForRolloverReadyStep = new WaitForRolloverReadyStep(
                waitForRolloverStepKey,
                rolloverStepKey,
                client,
                null,
                null,
                null,
                1L,
                null,
                null,
                null,
                null,
                null,
                null
            );

            try {
                IndexLifecycleTransition.validateTransition(
                    meta,
                    currentStepKey,
                    nextStepKey,
                    createOneStepPolicyStepRegistry("my-policy", waitForRolloverReadyStep)
                );
            } catch (Exception e) {
                logger.error(e.getMessage(), e);
                fail("validateTransition should not throw exception on valid transitions");
            }
        }
    }

Logs (if relevant)

No response

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)