Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for source_path to the field caps API. #52345

Closed
wants to merge 1 commit into from

Conversation

jtibshirani
Copy link
Contributor

This PR contains a draft of adding _source path information to the field caps API. It implements the API addition described in #49264 (comment). This addition focuses on the SQL and ML use case of looking up values through _source. (Kibana may also want to use source path information to better support field aliases -- we're still working on defining their use case + requirements.)

Some notes on the design:

  • To keep the field caps output fairly streamlined, we never include the field itself in the source path. When looking up values, clients should always assume that the source path could include the field itself.
  • There is no special handling for meta fields, clients need to understand that they are not returned in _source.
  • In addition to field alias targets and parent fields (for multi-fields), the source path includes fields whose contents were copied to the field through copy_to.
  • Looking up the source paths for a field is currently not transitive. For example, if a field alias had a multi-field as its target, then we would just return the name of that multi-field (and not trace the source path to its parent). My intuition is that lookup should be transitive -- that way a client wouldn’t have to make multiple calls to field caps to find the actual source paths.

It would be great to get feedback on these notes. I’m curious if/ how we currently handle copy_to when looking up fields in SQL and ML, and if the current design is a good fit. Supporting copy_to did make the API + implementation a bit more complex, since each field can have multiple source paths.

Other implementation notes:

  • The main functionality is exposed through MapperService#sourcePath. Tracking source path information within MapperService lets us avoid scanning over all the mappings on each call to field caps, and also makes the information available to other search components (maybe it could be useful to highlighting?)
  • Some TODOs remain, like cleaning up tests and adding Javadoc.

Addresses #49264.

@jtibshirani jtibshirani added >enhancement :Search Foundations/Mapping Index mappings, including merging and defining field types labels Feb 14, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (:Search/Mapping)

@jtibshirani
Copy link
Contributor Author

Closing, as we've decided to pursue #49028 instead of this addition to source_path. We'll likely end up reusing the 'source path lookup' logic from this PR.

@jtibshirani jtibshirani deleted the source-path branch April 24, 2020 18:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>enhancement :Search Foundations/Mapping Index mappings, including merging and defining field types
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants