Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change the way that fields are selected for inclusion in all queries. #31655

Closed
wants to merge 1 commit into from

Conversation

jpountz
Copy link
Contributor

@jpountz jpountz commented Jun 28, 2018

Currently this is a hardcoded whitelist, which is problematic due to the fact
that plugins can't add new types to this list, we already have the problem with
the scaled_float type today. This change proposes to rely on exception types
of the termQuery factory method instead to know whether a field should be
searched.

Another option would be to add an API to MappedFieldType to check whether a
field must be considered for all queries, but I wanted to avoid adding a new
API.

Currently this is a hardcoded whitelist, which is problematic due to the fact
that plugins can't add new types to this list, we already have the problem with
the `scaled_float` type today. This change proposes to rely on exception types
of the `termQuery` factory method instead to know whether a field should be
searched.

Another option would be to add an API to MappedFieldType to check whether a
field must be considered for `all` queries, but I wanted to avoid adding a new
API.
@jpountz jpountz added >enhancement :Search/Search Search-related issues that do not fall into other categories labels Jun 28, 2018
@jpountz jpountz requested a review from markharwood June 28, 2018 12:57
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search-aggs

* @throws ElasticsearchParseException if {@code value} cannot be converted to the expected data type
* @throws UnsupportedOperationException if the field is not searchable regardless of options
* @throws QueryShardException if the field is not searchable regardless of options
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last 2 javadoc "throws" descriptions give identical reasons?

// Unmapped fields are not ignored
fields.put(fieldOrPattern, weight);
continue;
}
if (acceptMetadataField == false && mapper instanceof MetadataFieldMapper) {
if (acceptMetadataField == false && ft.name().startsWith("_")) {
// Ignore metadata fields
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this break BWC for some users? Underscored fieldnames were always by convention "ours" but are we now being stronger on this (either with docs or enforcing naming conventions?)

continue;
} catch (RuntimeException e) {
// other exceptions are parsing errors or not indexed fields: keep
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a similar capability-testing concern with field types that support phrase queries?

Copy link
Contributor

@jimczi jimczi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, another solution would be to remove the selection logic entirely and rely on the fact that we force lenient mode when all fields are requested (*).

jimczi added a commit to jimczi/elasticsearch that referenced this pull request Aug 21, 2018
This commit changes the query field expansion for query parsers
to not rely on an hardcoded list of field types. Instead we rely on
the type of exception that is thrown by MappedFieldType#termQuery to
include/exclude an expanded field.

Supersedes elastic#31655

Closes elastic#31798
@jimczi
Copy link
Contributor

jimczi commented Aug 21, 2018

Superseded by #33020

@jimczi jimczi closed this Aug 21, 2018
jimczi added a commit that referenced this pull request Aug 23, 2018
This commit changes the query field expansion for query parsers
to not rely on an hardcoded list of field types. Instead we rely on
the type of exception that is thrown by MappedFieldType#termQuery to
include/exclude an expanded field.

Supersedes #31655

Closes #31798
jimczi added a commit that referenced this pull request Aug 23, 2018
This commit changes the query field expansion for query parsers
to not rely on an hardcoded list of field types. Instead we rely on
the type of exception that is thrown by MappedFieldType#termQuery to
include/exclude an expanded field.

Supersedes #31655

Closes #31798
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>enhancement :Search/Search Search-related issues that do not fall into other categories
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants