Fix Enterprise Search document search bug in indices without position data indexed #148397
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kderusso
added
bug
sphilipse
approved these changes
Jan 4, 2023
TattdCodeMonkey
approved these changes
Jan 4, 2023
| @@ -21,7 +21,7 @@ export const fetchSearchResults = async ( | |||
| from, | |||
| index: indexName, | |||
| size, | |||
| ...(!!query ? { q: JSON.stringify(query) } : {}), | |||
| ...(!!query ? { q: query.replace(/"/g, '\\"') } : {}), | |||
There was a problem hiding this comment.
I'm good with this, but should we do any follow-up to ensure the query string is sanitized further?
There was a problem hiding this comment.
@TattdCodeMonkey That's a great question. Did you have anything specific in mind?
@sphilipse seemed to think that was sufficient because you can't really inject anything into the query at that point plus it's already behind authorization. I really dislike the idea of ad-hoc manual safety String mutations for several reasons. We could consider adding a new issue to re-do the search form so that Stringify doesn't force everything to use multi-match, but I'm not really sure what the priority is. WDYT?
There was a problem hiding this comment.
@kderusso The call is a query_string query, right? Could you describe how the multi-word query fails due to the index not supporting multi-match, e.g. show the error message? Maybe we can fix it in a different way.
My concern is putting the expression in quotes forces a phrase match, which differs from the actual full text matching experience (OR semantics by default). But I agree this is low priority.
There was a problem hiding this comment.
@demjened Can you please help me better understand your concern here? JSON.stringify was quoting the query and forcing a multi-match query to be run. This removes this default behavior.
There was a problem hiding this comment.
Ah, I thought query was already coming in quoted from the UI. But then why the need for .replace()? Can't we just pass q: query as-is?
There was a problem hiding this comment.
Users can still input a quote in the front-end, which will force a multi-match if we don't escape them.
Quick question: do we also need to escape single quotes, or only double quotes?
There was a problem hiding this comment.
@sphilipse I have verified that single quotes don't need to be escaped, using them doesn't error.
@demjened If we use double quotes it will force the multi-match phrase query and error. Escaping them solves this.
💚 Build Succeeded
Metrics [docs]
To update your PR or re-run it, just comment with: |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jan 4, 2023
… data indexed (elastic#148397) ## Summary Fixes a bug where some searches including spaces would error in certain Enterprise Search indices that don't support multi-match queries in indexed metadata. ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit 07a19f1)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jan 4, 2023
… data indexed (elastic#148397) ## Summary Fixes a bug where some searches including spaces would error in certain Enterprise Search indices that don't support multi-match queries in indexed metadata. ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit 07a19f1)
💔 Some backports could not be created
Note: Successful backport PRs will be merged automatically after passing CI. Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Jan 4, 2023
…sition data indexed (#148397) (#148410) # Backport This will backport the following commits from `main` to `8.5`: - [Fix Enterprise Search document search bug in indices without position data indexed (#148397)](#148397) <!--- Backport version: 8.9.7 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Kathleen DeRusso","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-01-04T18:08:59Z","message":"Fix Enterprise Search document search bug in indices without position data indexed (#148397)\n\n## Summary\r\n\r\nFixes a bug where some searches including spaces would error in certain\r\nEnterprise Search indices that don't support multi-match queries in\r\nindexed metadata.\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"07a19f12d278ccf03a2a586f3bcef98f9d6f289c","branchLabelMapping":{"^v8.7.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:skip","Team:EnterpriseSearch","auto-backport-next","v8.6.0","v8.7.0","v7.17.9"],"number":148397,"url":"https://github.com/elastic/kibana/pull/148397","mergeCommit":{"message":"Fix Enterprise Search document search bug in indices without position data indexed (#148397)\n\n## Summary\r\n\r\nFixes a bug where some searches including spaces would error in certain\r\nEnterprise Search indices that don't support multi-match queries in\r\nindexed metadata.\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"07a19f12d278ccf03a2a586f3bcef98f9d6f289c"}},"sourceBranch":"main","suggestedTargetBranches":["8.6","7.17"],"targetPullRequestStates":[{"branch":"8.6","label":"v8.6.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.7.0","labelRegex":"^v8.7.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/148397","number":148397,"mergeCommit":{"message":"Fix Enterprise Search document search bug in indices without position data indexed (#148397)\n\n## Summary\r\n\r\nFixes a bug where some searches including spaces would error in certain\r\nEnterprise Search indices that don't support multi-match queries in\r\nindexed metadata.\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"07a19f12d278ccf03a2a586f3bcef98f9d6f289c"}},{"branch":"7.17","label":"v7.17.9","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Kathleen DeRusso <[email protected]>
kibanamachine
added a commit
that referenced
this pull request
Jan 5, 2023
…sition data indexed (#148397) (#148411) # Backport This will backport the following commits from `main` to `8.6`: - [Fix Enterprise Search document search bug in indices without position data indexed (#148397)](#148397) <!--- Backport version: 8.9.7 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Kathleen DeRusso","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-01-04T18:08:59Z","message":"Fix Enterprise Search document search bug in indices without position data indexed (#148397)\n\n## Summary\r\n\r\nFixes a bug where some searches including spaces would error in certain\r\nEnterprise Search indices that don't support multi-match queries in\r\nindexed metadata.\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"07a19f12d278ccf03a2a586f3bcef98f9d6f289c","branchLabelMapping":{"^v8.7.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:skip","Team:EnterpriseSearch","auto-backport-next","v8.6.0","v8.7.0","v7.17.9"],"number":148397,"url":"https://github.com/elastic/kibana/pull/148397","mergeCommit":{"message":"Fix Enterprise Search document search bug in indices without position data indexed (#148397)\n\n## Summary\r\n\r\nFixes a bug where some searches including spaces would error in certain\r\nEnterprise Search indices that don't support multi-match queries in\r\nindexed metadata.\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"07a19f12d278ccf03a2a586f3bcef98f9d6f289c"}},"sourceBranch":"main","suggestedTargetBranches":["8.6","7.17"],"targetPullRequestStates":[{"branch":"8.6","label":"v8.6.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.7.0","labelRegex":"^v8.7.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/148397","number":148397,"mergeCommit":{"message":"Fix Enterprise Search document search bug in indices without position data indexed (#148397)\n\n## Summary\r\n\r\nFixes a bug where some searches including spaces would error in certain\r\nEnterprise Search indices that don't support multi-match queries in\r\nindexed metadata.\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"07a19f12d278ccf03a2a586f3bcef98f9d6f289c"}},{"branch":"7.17","label":"v7.17.9","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Kathleen DeRusso <[email protected]>
|
The backport was not included on |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Fixes a bug where some searches including spaces would error in certain Enterprise Search indices that don't support multi-match queries in indexed metadata.
Checklist
Delete any items that are not applicable to this PR.
For maintainers