Prevent downsample indices to inherit pipeline-related index settings

[salvatore-campagna]

Elasticsearch Version

8.7

Installed Plugins

No response

Java Version

bundled

OS Version

All

Problem Description

When downsampling a source index into a target index, the target index is created copying some settings from the source index. Among the settings we copy there are two of them index.default_pipeline and index.final_pipeline which control execution of ingest pipelines for the (target) index. As a result of the transformation we apply to the target (downsampled) index, including aggregation of metric fields, it is possible that the pipeline script execution fails because some metric fields are converted to aggregate_metric_double fields. These fields are not accessible in the same way as the original ones since they are of different type and because they just include aggregate metrics (sum, min, max and value_count).

We need to skip copying the two settings, index.default_pipeline and index.final_pipeline, from the source index to the target index in order to avoid pipeline processing to be applied to the target (downsampled) index.

NOTE: the two settings might be inherited by a source index that is part of a data stream

Steps to Reproduce

Try to downsample a source index that has the following two settings index.default_pipeline and index.final_pipeline. The target index will include the same settings. That should not be the case.

Logs (if relevant)

No response