Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.x] Backport two PRs (#117246) (#117843) #117967

Merged

Conversation

* LOOKUP JOIN using field-caps for field mapping

Removes the hard-coded hack for languages_lookup, and instead does a field-caps check for the real join index.

* Update docs/changelog/117246.yaml

* Some code review comments
…l bugs found (elastic#117843)

Adds several more tests to lookup-join.csv-spec, and fixes the following bugs:

* FieldCaps on right hand side should ignore fieldNames method and just use "*" because currently the fieldNames search cannot handle lookup fields with aliases (should be fixed in a followup PR).
* Stop using the lookup index in the ComputeService (so we don’t get both indices data coming in from the left, and other weird behaviour).
* Ignore failing SearchStats checks on fields from the right hand side in the logical planner (so it does not plan EVAL field = null for all right hand fields). This should be fixed properly with the correct updates to TransportSearchShardsAction (or rather to making multiple use of that for each branch of the execution model).
@craigtaverner craigtaverner added backport Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) :Analytics/ES|QL AKA ESQL v8.18.0 labels Dec 4, 2024
@elasticsearchmachine elasticsearchmachine merged commit 5afbfda into elastic:8.x Dec 5, 2024
14 checks passed
@craigtaverner craigtaverner deleted the 8.x-backport-two-lookupjoin-prs branch December 5, 2024 11:29
craigtaverner added a commit that referenced this pull request Dec 10, 2024
When we back-ported the LOOKUP JOIN PRs to 8.x (see #117967), we found it necessary to disable all csv-spec tests since they create indices with mode:lookup, which is illegal in the cluster state of mixed clusters where other nodes do not understand the new index mode. We need to re-enable the tests, and make sure the tests are only disabled in mixed clusters with node versions too old to handle the new mode.
craigtaverner added a commit to craigtaverner/elasticsearch that referenced this pull request Dec 10, 2024
When we back-ported the LOOKUP JOIN PRs to 8.x (see elastic#117967), we found it necessary to disable all csv-spec tests since they create indices with mode:lookup, which is illegal in the cluster state of mixed clusters where other nodes do not understand the new index mode. We need to re-enable the tests, and make sure the tests are only disabled in mixed clusters with node versions too old to handle the new mode.
craigtaverner added a commit that referenced this pull request Dec 12, 2024
When we back-ported the LOOKUP JOIN PRs to 8.x (see #117967), we found it necessary to disable all csv-spec tests since they create indices with mode:lookup, which is illegal in the cluster state of mixed clusters where other nodes do not understand the new index mode. We need to re-enable the tests, and make sure the tests are only disabled in mixed clusters with node versions too old to handle the new mode.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) backport Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v8.18.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants