Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] 'ignore_malformed' applied to a malformed (boolean) numeric field does not work #11498

Closed
angeleg opened this issue Jun 4, 2015 · 5 comments · Fixed by #41261
Closed
Assignees
Labels
>bug good first issue low hanging fruit help wanted adoptme :Search Foundations/Mapping Index mappings, including merging and defining field types Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch

Comments

@angeleg
Copy link

angeleg commented Jun 4, 2015

When I try to index a doc with a boolean field that's mapped as numeric, even with ìgnored_malformedset totrue`, an exception is raised.

Repro steps:

Create a new index:
curl -XPUT localhost:9200/myindex

which yields { "acknowledged" : true }

Create a mapping for a type in that index, with one integer field that has ignore_malformed set to true and coerce set to true
curl -XPUT localhost:9200/myindex/_mapping/mytype -d
'{
    "mytype": {
      "properties": {
        "foo": {
        "type": "integer",
        "ignore_malformed": true,
        "coerce": false
        }
      }
  }
}'

which yields { "acknowledged" : true }

Index one document with a field whose value is false
curl -XPUT localhost:9200/myindex/mytype/one -d
'{
  a: 3,
  foo: false
}'

which yields an exception:

{
"error" : "MapperParsingException[failed to parse [foo]]; nested: JsonParseException[Current token (VALUE_FALSE) not numeric, can not use numeric value accessors at [Source: [B@3a98c595; line: 3, column: 13]]; ",
"status" : 400
}

According to ignored_malformed: true, even if false is not an integer, the doc should get indexed anyway.

@clintongormley clintongormley added >bug good first issue low hanging fruit help wanted adoptme :Search Foundations/Mapping Index mappings, including merging and defining field types labels Jun 4, 2015
@szroland
Copy link
Contributor

This will not only fail with booleans, but with other non-numeric values as well, e.g.

{
  a: 3,
  foo: {
    value: []
  }
}
type: "json_parse_exception"
reason: "Current token (START_ARRAY) not numeric, can not use numeric value accessors at [Source: org.elasticsearch.common.io.stream.InputStreamStreamInput@2725adf1; line: 4, column: 13]"

Ignoring malformed values depends on the code throwing an IllegalArgumentException or MapperParsingException but here a JsonParseException is thrown. The solution would be to check the token type in AbstractXContentParser.intValue(), shortValue(), etc. after the String check, and if it is not numeric, do not even try parsing the value, throw an IllegalArgumentException immediately.

This touches the same area of code as #11513. Happy to take a stab if you all agree.

@clintongormley
Copy link
Contributor

Closing in favour of #11513

@lwiskowski lwiskowski mentioned this issue May 5, 2016
@lwiskowski
Copy link

I changed AbstactXContentParser.java to address this issue. I will do a pull request with the updated code

@clintongormley
Copy link
Contributor

Related to #12366

@jpountz
Copy link
Contributor

jpountz commented Mar 13, 2018

cc @elastic/es-search-aggs

@cbuescher cbuescher self-assigned this Apr 16, 2019
cbuescher pushed a commit to cbuescher/elasticsearch that referenced this issue Apr 16, 2019
The `ignore_malformed` option currently works on numeric fields only when the
bad value isn't a string value but not if it is a boolean. In this case we get a
parsing error from the xContent parser which we need to catch in addition to the
field mapper.

Closes elastic#11498
cbuescher pushed a commit that referenced this issue Apr 17, 2019
The `ignore_malformed` option currently works on numeric fields only when the
bad value isn't a string value but not if it is a boolean. In this case we get a
parsing error from the xContent parser which we need to catch in addition to the
field mapper.

Closes #11498
cbuescher pushed a commit that referenced this issue Apr 17, 2019
The `ignore_malformed` option currently works on numeric fields only when the
bad value isn't a string value but not if it is a boolean. In this case we get a
parsing error from the xContent parser which we need to catch in addition to the
field mapper.

Closes #11498
cbuescher pushed a commit that referenced this issue Apr 17, 2019
The `ignore_malformed` option currently works on numeric fields only when the
bad value isn't a string value but not if it is a boolean. In this case we get a
parsing error from the xContent parser which we need to catch in addition to the
field mapper.

Closes #11498
gurkankaymak pushed a commit to gurkankaymak/elasticsearch that referenced this issue May 27, 2019
The `ignore_malformed` option currently works on numeric fields only when the
bad value isn't a string value but not if it is a boolean. In this case we get a
parsing error from the xContent parser which we need to catch in addition to the
field mapper.

Closes elastic#11498
@javanna javanna added the Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch label Jul 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug good first issue low hanging fruit help wanted adoptme :Search Foundations/Mapping Index mappings, including merging and defining field types Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants