search with shard errors from local cluster no longer shows results in kibana (after elasticsearch 98725 is resolved) #164893
Assignees
Labels
bug
Fixes for quality problems that affect the customer experience
Feature:Search
Querying infrastructure in Kibana
impact:high
Addressing this issue will have a high level of impact on the quality/strength of our product.
loe:medium
Medium Level of Effort
Team:DataDiscovery
Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.
Comments
nreese
added
bug
|
Pinging @elastic/kibana-data-discovery (Team:DataDiscovery) |
nreese
added
the
impact:critical
kertal
added
impact:high
nreese
added a commit
that referenced
this issue
Oct 2, 2023
Closes #164893 ### Background "is partial" has 2 meanings 1) Results are incomplete because search is still running 2) Search is finished. Results are incomplete because there are shard failures (either in local or remote clusters) [async search](https://www.elastic.co/guide/en/elasticsearch/reference/current/async-search.html) defines 2 flags. 1) `is_running`: Whether the search is still being executed or it has completed 2) `is_partial`: When the query is no longer running, indicates whether the search failed or was successfully completed on all shards. While the query is being executed, is_partial is always set to true **note**: there is a bug in async search where requests to only local clusters return `is_partial:false` when there are shard errors on the local cluster. See elastic/elasticsearch#98725. This should be resolved in 8.11 Kibana's existing search implementation does not align with Elasticsearch's `is_running` and `is_partial` flags. Kibana defines "is partial" as definition "1)". Elasticsearch async search defines "is partial" as definition "2)". This PR aligns Kibana's usage of "is partial" with Elasticsearch's definition. This required the following changes 1) `isErrorResponse` renamed to `isAbortedResponse`. Method no longer returns true when `!response.isRunning && !!response.isPartial`. Kibana handles results with incomplete data. **Note** removed export of `isErrorResponse` from data plugin since its use outside of data plugin does not make sense. 2) Replace `isPartialResponse` with `isRunningResponse`. This aligns Kibana's definition with Elasticsearch async search flags. 3) Remove `isCompleteResponse`. The word "complete" is ambiguous. Does it mean the search is finished (no longer running)? Or does it mean the search has all results and there are no shard failures? --------- Co-authored-by: kibanamachine <[email protected]> Co-authored-by: Jatin Kathuria <[email protected]> Co-authored-by: Patryk Kopyciński <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
poll_search throws an error when
isErrorResponse(response)returns true. isErrorResponse has a bug where it returns true when!response.isRunning && !!response.isPartial. This is not causing any problems currently because Kibana is saved by an elasticsearch bug - async search returns is_partial:false when local cluster has shard errorsHowever, once elastic/elasticsearch#98725 is resolved, this will no longer be the case and the bug in
isErrorResponsewill prevent Kibana from displaying results when local clusters have shard errors. This will effect Kibana system wide, whendata.searchandsearchService.searchis used to fetch data, and be a large regression.A solution is being actively worked in #164506
The text was updated successfully, but these errors were encountered: