Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fields not searchable with subobjects: false and dynamic: false when sent as nested objects #117544

Closed
flash1293 opened this issue Nov 26, 2024 · 3 comments · Fixed by #117762
Closed
Assignees
Labels
>bug :StorageEngine/Mapping The storage related side of mappings Team:StorageEngine

Comments

@flash1293
Copy link
Contributor

flash1293 commented Nov 26, 2024

Elasticsearch Version

main

Installed Plugins

No response

Java Version

bundled

OS Version

Darwin Mac 24.1.0

Problem Description

When subobjects: false and dynamic: false are used together in an index, fields are not searchable if the documents are sent as nested objects.

Steps to Reproduce

Prepare:

PUT my-index
{
  "mappings": {
    "subobjects": false,
    "dynamic": false,
    "properties": {
      "my.keyword.field": {
        "type": "keyword"
      }
    }
  }
}

POST my-index/_doc
{
  "my": {
    "keyword.field": "abc"
  }
}
GET my-index*/_search
{
  "fields": [
    "*"
  ]
}

Returns

        "fields": {
          "my.keyword.field": [
            "abc"
          ]
        }

so it looks like my.keyword.field is indexed properly.

However,

GET my-index/_search
{
  "query": {
    "match": {
      "my.keyword.field": "abc"
    }
  }
}

does not return any hits.

  • When setting dynamic: true, the match-search works as expected (even though the mapping looks exactly the same)
  • When setting subobjects: true, the match-search works as expected
  • When sending the doc as { "my.keyword.field": "abc" }, the match-search works as expected

Logs (if relevant)

No response

@flash1293 flash1293 added >bug needs:triage Requires assignment of a team area label Team:StorageEngine :StorageEngine/Mapping The storage related side of mappings labels Nov 26, 2024
@flash1293
Copy link
Contributor Author

cc @martijnvg - maybe you have an idea

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

@elasticsearchmachine elasticsearchmachine removed the needs:triage Requires assignment of a team area label label Nov 26, 2024
@lkts
Copy link
Contributor

lkts commented Nov 27, 2024

The result of fields parameter is misleading here, not sure why. There is no field my.keyword.field in Lucene document in this case. I think that object auto-flattening indirectly depends on dynamic updates somehow causing this bug.

@kkrik-es kkrik-es self-assigned this Nov 28, 2024
flash1293 added a commit to elastic/kibana that referenced this issue Dec 10, 2024
With elastic/elasticsearch#117544 fixed,
`subobjects: false` can be set on wired streams.

This allows to map fields as scalars even if they share the same prefix,
e.g.
* `deeply.nested.path` as `keyword`
* `deeply.nested.path.and.more` as `keyword`
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Dec 10, 2024
With elastic/elasticsearch#117544 fixed,
`subobjects: false` can be set on wired streams.

This allows to map fields as scalars even if they share the same prefix,
e.g.
* `deeply.nested.path` as `keyword`
* `deeply.nested.path.and.more` as `keyword`

(cherry picked from commit 7f3c642)
kibanamachine added a commit to elastic/kibana that referenced this issue Dec 10, 2024
# Backport

This will backport the following commits from `main` to `8.x`:
- [🌊 Streams: Set subobjects: false
(#203002)](#203002)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Joe
Reuter","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-12-10T11:28:13Z","message":"🌊
Streams: Set subobjects: false (#203002)\n\nWith
elastic/elasticsearch#117544
fixed,\r\n`subobjects: false` can be set on wired streams.\r\n\r\nThis
allows to map fields as scalars even if they share the same
prefix,\r\ne.g.\r\n* `deeply.nested.path` as `keyword`\r\n*
`deeply.nested.path.and.more` as
`keyword`","sha":"7f3c642c0a3717a0f71275051fd35a48f88a1e0a","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:prev-minor","v8.18.0","Feature:Streams"],"title":"🌊
Streams: Set subobjects:
false","number":203002,"url":"https://github.com/elastic/kibana/pull/203002","mergeCommit":{"message":"🌊
Streams: Set subobjects: false (#203002)\n\nWith
elastic/elasticsearch#117544
fixed,\r\n`subobjects: false` can be set on wired streams.\r\n\r\nThis
allows to map fields as scalars even if they share the same
prefix,\r\ne.g.\r\n* `deeply.nested.path` as `keyword`\r\n*
`deeply.nested.path.and.more` as
`keyword`","sha":"7f3c642c0a3717a0f71275051fd35a48f88a1e0a"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/203002","number":203002,"mergeCommit":{"message":"🌊
Streams: Set subobjects: false (#203002)\n\nWith
elastic/elasticsearch#117544
fixed,\r\n`subobjects: false` can be set on wired streams.\r\n\r\nThis
allows to map fields as scalars even if they share the same
prefix,\r\ne.g.\r\n* `deeply.nested.path` as `keyword`\r\n*
`deeply.nested.path.and.more` as
`keyword`","sha":"7f3c642c0a3717a0f71275051fd35a48f88a1e0a"}},{"branch":"8.x","label":"v8.18.0","branchLabelMappingKey":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Joe Reuter <[email protected]>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Dec 12, 2024
With elastic/elasticsearch#117544 fixed,
`subobjects: false` can be set on wired streams.

This allows to map fields as scalars even if they share the same prefix,
e.g.
* `deeply.nested.path` as `keyword`
* `deeply.nested.path.and.more` as `keyword`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :StorageEngine/Mapping The storage related side of mappings Team:StorageEngine
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants