Add support for index_filter to open pit #102388
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The open point in time API accepts a list of indices and opens a point in time view against those indices. Like we do already for field caps, this commit allows users to provide an index_filter parameter as part of the request body, that will be used to execute the can match phase and exclude the indices that can't possibly match such filter. Closes elastic#99740
javanna
added
>enhancement
:Search/Search
|
Documentation preview: |
|
Pinging @elastic/es-search (Team:Search) |
|
Hi @javanna, I've created a changelog YAML for you. |
2 tasks
pmpailis
approved these changes
Nov 21, 2023
rest-api-spec/src/main/resources/rest-api-spec/api/open_point_in_time.json
Outdated
Show resolved
Hide resolved
server/src/internalClusterTest/java/org/elasticsearch/action/search/PointInTimeIT.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/action/search/TransportOpenPointInTimeAction.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/action/search/TransportOpenPointInTimeAction.java
Show resolved
Hide resolved
|
Thanks @pmpailis ! |
8 tasks
javanna
added a commit
to javanna/elasticsearch
that referenced
this pull request
Jan 11, 2024
We recently introduced support for index_filter to the open point in time API (see elastic#102388). Open point in time supports executing against remote indices, in which case it will open a reader context against the target remote shards. With support for index_filter, shards that cannot match the filter are not even included in the PIT id that open PIT returns. When the following search is executed that includes such PIT id, there is one search shards call per cluster performed, which will return all shards from the targeted indices, including those that open PIT has filtered out. In that case, we should just ignore those shards instead of throwing exception when those are looked up in the search context id map built from the PIT id. Closes elastic#102596
javanna
added a commit
that referenced
this pull request
Jan 15, 2024
We recently introduced support for index_filter to the open point in time API (see #102388). Open point in time supports executing against remote indices, in which case it will open a reader context against the target remote shards. With support for index_filter, shards that cannot match the filter are not even included in the PIT id that open PIT returns. When the following search is executed that includes such PIT id, there is one search shards call per cluster performed, which will return all shards from the targeted indices, including those that open PIT has filtered out. In that case, we should just ignore those shards instead of throwing exception when those are looked up in the search context id map built from the PIT id. Closes #102596
javanna
added a commit
to javanna/elasticsearch
that referenced
this pull request
Jan 15, 2024
…ic#104288) We recently introduced support for index_filter to the open point in time API (see elastic#102388). Open point in time supports executing against remote indices, in which case it will open a reader context against the target remote shards. With support for index_filter, shards that cannot match the filter are not even included in the PIT id that open PIT returns. When the following search is executed that includes such PIT id, there is one search shards call per cluster performed, which will return all shards from the targeted indices, including those that open PIT has filtered out. In that case, we should just ignore those shards instead of throwing exception when those are looked up in the search context id map built from the PIT id. Closes elastic#102596
javanna
added a commit
that referenced
this pull request
Jan 15, 2024
We recently introduced support for index_filter to the open point in time API (see #102388). Open point in time supports executing against remote indices, in which case it will open a reader context against the target remote shards. With support for index_filter, shards that cannot match the filter are not even included in the PIT id that open PIT returns. When the following search is executed that includes such PIT id, there is one search shards call per cluster performed, which will return all shards from the targeted indices, including those that open PIT has filtered out. In that case, we should just ignore those shards instead of throwing exception when those are looked up in the search context id map built from the PIT id. Closes #102596
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The open point in time API accepts a list of indices and opens a point in time view against those indices. Like we do already for field caps, this commit allows users to provide an index_filter parameter as part of the request body, that will be used to execute the can match phase and exclude the indices that can't possibly match such filter.
Closes #99740