Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not copy index.default_pipeline and index.final_pipeline #96494

Conversation

salvatore-campagna
Copy link
Contributor

We need to skip copying the two settings, index.default_pipeline and index.final_pipeline, from the source index to the target index in order to avoid pipeline processing to be applied to the target (downsampled) index.
Some pipeline scripts might fail due to the differences in mappings and settings between the downsampling source and target index.

Resolves #96478

@salvatore-campagna salvatore-campagna added :StorageEngine/Rollup Turn fine-grained time-based data into coarser-grained data >bug auto-backport-and-merge v8.8.1 v8.9.0 labels Jun 1, 2023
@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Jun 1, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo (Team:Analytics)

@elasticsearchmachine
Copy link
Collaborator

Hi @salvatore-campagna, I've created a changelog YAML for you.

Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -584,12 +591,16 @@ private IndexMetadata.Builder copyIndexMetadata(IndexMetadata sourceIndexMetadat
// the same rules with resize apply
continue;
}
if (FORBIDDEN_SETTINGS.contains(key)) {
continue;
}
// Do not override settings that have already been set in the rollup index.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe update this comment too?

@salvatore-campagna
Copy link
Contributor Author

I forgot to push the yaml test... 🤔

@elasticsearchmachine
Copy link
Collaborator

💚 Backport successful

Status Branch Result
8.8

salvatore-campagna added a commit to salvatore-campagna/elasticsearch that referenced this pull request Jun 1, 2023
…96494)

We need to skip copying the two settings, index.default_pipeline and index.final_pipeline,
from the source index to the target index in order to avoid pipeline processing to be applied 
to the target (downsampled) index. Some pipeline scripts might fail due to the differences in
mappings and settings between the downsampling source and target index.
elasticsearchmachine pushed a commit that referenced this pull request Jun 1, 2023
…96500)

We need to skip copying the two settings, index.default_pipeline and index.final_pipeline,
from the source index to the target index in order to avoid pipeline processing to be applied 
to the target (downsampled) index. Some pipeline scripts might fail due to the differences in
mappings and settings between the downsampling source and target index.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :StorageEngine/Rollup Turn fine-grained time-based data into coarser-grained data Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v8.8.1 v8.9.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Prevent downsample indices to inherit pipeline-related index settings
3 participants