Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] match_only_text type migration - Stage 0 #1396

Merged
merged 6 commits into from
May 11, 2021

Conversation

ebeahan
Copy link
Member

@ebeahan ebeahan commented May 5, 2021

Summary

Stage 0 proposal for migrating text type fields, including the message base field, to use the match_only_text data type being introduced in Elasticsearch 7.14.

Stage 0 (Strawperson) Criteria:

  • Discuss with domain or subject matter experts the utility of these changes
  • Discuss with the ECS team whether these changes seem appropriate for ECS

Preview of markdown proposal

@ebeahan ebeahan added the RFC label May 5, 2021
@ebeahan ebeahan self-assigned this May 5, 2021
Copy link
Contributor

@jpountz jpountz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great. I left a few minor clarifications.

rfcs/text/0000-match_only_text-data-type.md Outdated Show resolved Hide resolved
rfcs/text/0000-match_only_text-data-type.md Outdated Show resolved Hide resolved
@ebeahan ebeahan requested review from kgeller and djptek May 11, 2021 16:30
@ebeahan
Copy link
Member Author

ebeahan commented May 11, 2021

Thanks, @jpountz!

@kgeller @djptek, would you also review?

Copy link
Contributor

@kgeller kgeller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@ebeahan ebeahan merged commit 2f31b53 into elastic:master May 11, 2021
@ebeahan ebeahan deleted the rfc/match_only_text-migrate branch May 11, 2021 22:34
* Span queries are unsupported. If a span query is run, then shards where the field is mapped as match_only_text will be returned as failed in the search response and their hits will be ignored.
* Phrase and intervals queries run slower.

Like `text`, `match_only_text` fields do not support aggregations.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we change "do not support" to "support limited" because both text, match_only_text DO support a limited subset of aggs?

I am aware that's not what it says in the docs

I have verified this vs Elasticsearch master (8.0.0-SNAPSHOT)

See
Gist
with all you need to reproduce the results

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see related PR to Elasticsearch, I may cancel this review depending on the response from there also: elastic/elasticsearch#72985

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great catch! I'll note this and include the change in the stage 1 PR for this RFC.

rylnd added a commit to rylnd/ecs that referenced this pull request May 28, 2021
* master:
  Stage 2 changes for RFC 0018 - extending the `threat.*` field set (elastic#1438)
  Remove deprecated `host.user.*` fields (elastic#1439)
  Explicitly include user identifiers in `related.user` field description (elastic#1420)
  Set the merge date on RFC 0018 stage 2 (elastic#1429)
  [RFC] Extend Threat Fieldset - Stage 2 Proposal (elastic#1395)
  [Tooling] Add --exclude flag to Generator to support field removal testing (elastic#1411)
  Add `host.user.*` deprecation notice in field reuse description (elastic#1422)
  Stage 2 changes for RFC 0015 - `elf` header (elastic#1410)
  Stage 3 changes for RFC 0012 - `orchestrator` field set (elastic#1417)
  Support `match_only_text` in Go code generator (elastic#1418)
  Stage 3 Orchestrator RFC (elastic#1343)
  moving into folder (elastic#1416)
  removing use-cases (elastic#1405)
  removing --oss (elastic#1404)
  Set the merge date on RFC 0015 stage 2 (elastic#1409)
  Consolidate `Breaking changes` sections in `CHANGELOG.next` (elastic#1408)
  RFC-Stage-0: Proposal to add a "ticket" schema / field definition to ECS (elastic#1383)
  [RFC] `match_only_text` type migration - Stage 0 (elastic#1396)
  Client port is wrongly documented (elastic#1402) (elastic#1406)
ebadyano added a commit to ebadyano/rally-tracks that referenced this pull request Jun 8, 2021
ebadyano added a commit to elastic/rally-tracks that referenced this pull request Jun 14, 2021
ebadyano added a commit to ebadyano/rally-tracks that referenced this pull request Jun 15, 2021
ebadyano added a commit to elastic/rally-tracks that referenced this pull request Oct 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants