Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mapping: Geopoint with array broken (dynamic mapping): geo_point expected #6939

Closed
pierrre opened this issue Jul 21, 2014 · 2 comments · Fixed by #7175
Closed

Mapping: Geopoint with array broken (dynamic mapping): geo_point expected #6939

pierrre opened this issue Jul 21, 2014 · 2 comments · Fixed by #7175

Comments

@pierrre
Copy link

pierrre commented Jul 21, 2014

I use Elasticsearch with Logstash and dynamic mapping.
I my logs, I have geopoints with the array syntax.
It was OK with ES 1.0.X, but it's broken with ES 1.2.2.

Template:

curl -XPUT localhost:9200/_template/logstash -d '{
    "template": "logstash_*",
    "settings" : {
        "refresh_interval": "30s"
    },
    "mappings": {
        "logs": {
            "_all" : {
                "enabled": false
            },
            "dynamic_templates": [
                {
                    "location": {
                        "match": "location*",
                        "mapping": {
                            "type": "geo_point"
                        }
                    }
                },
                {
                    "generic": {
                        "match": "*",
                        "match_mapping_type": "string",
                        "mapping": {
                            "type": "string",
                            "index": "not_analyzed"
                        }
                    }
                }
            ],
            "dynamic_date_formats": [
                "dateOptionalTime",
                "yyyy-MM-dd",
                "yyyy-MM-dd HH:mm:ss"
            ]
        }
    }
}'

Delete index:

curl -XDELETE localhost:9200/logstash_test

Try to add a doc/log:

curl -XPOST localhost:9200/logstash_test/logs -d '{
  "location_array": [
    2.3069244,
    48.8881598
  ]
}'

It fails with this message:

[2014-07-21 11:31:49,168][INFO ][cluster.metadata         ] [fr-dev-01] [logstash_test] creating index, cause [auto(index api)], shards [5]/[1], mappings [logs]
[2014-07-21 11:31:49,465][DEBUG][action.index             ] [fr-dev-01] [logstash_test][1], node[zzMEQe9JSS6qEgw0oRgdVA], [P], s[STARTED]: Failed to execute [index {[logstash_test][logs][PhqP9fpgQkS4tHfOs105GQ], source[{"location_array":[2.3069244,48.8881598]}]}]
org.elasticsearch.index.mapper.MapperParsingException: failed to parse
    at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:536)
    at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:462)
    at org.elasticsearch.index.shard.service.InternalIndexShard.prepareCreate(InternalIndexShard.java:373)
    at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:203)
    at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:534)
    at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:433)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:724)
Caused by: org.elasticsearch.ElasticsearchParseException: geo_point expected
    at org.elasticsearch.common.geo.GeoUtils.parseGeoPoint(GeoUtils.java:421)
    at org.elasticsearch.index.mapper.geo.GeoPointFieldMapper.parse(GeoPointFieldMapper.java:530)
    at org.elasticsearch.index.mapper.object.ObjectMapper.parseDynamicValue(ObjectMapper.java:819)
    at org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:639)
    at org.elasticsearch.index.mapper.object.ObjectMapper.serializeArray(ObjectMapper.java:625)
    at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:482)
    at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:515)
    ... 8 more

If I insert a doc with a geopoint as an object, it works, and the mapping is created dynamically.
After that, I can insert a doc with a geopoint as an array without error.

@bobpaulin
Copy link

This also appears to be happening on the master branch.

@bobpaulin
Copy link

Seems like it's blowing up since the geo_point is only parsing the first value instead of the 2 item array as a whole. It works the second time through because the mapper object within org.elasticsearch.index.mapper.object.ObjectMapper.serializeArray caches location_array as a geo_point so the array logic properly pulls the mapper. The first time through that entry is not there so the mapper comes up null and it tries to parse the array values individually. May I suggest adding some logic to the org.elasticsearch.index.mapper.object.ObjectMapper.serializeArray method to run the template builder logic to find the GeoMapper

private void serializeArray(ParseContext context, String lastFieldName) throws IOException {
    String arrayFieldName = lastFieldName;
    Mapper mapper = mappers.get(lastFieldName);
    //Start Check TemplateBuilder for geo
    if (mapper == null) {
        BuilderContext builderContext = new BuilderContext(context.indexSettings(), context.path());
        Mapper.Builder builder = context.root().findTemplateBuilder(context, arrayFieldName, "geo_point");
        if (builder != null) {
            mapper = builder.build(builderContext);
            mappers.put(lastFieldName, mapper);
        }
    }
    //End Check TemplateBuilder for geo
    if (mapper != null && mapper instanceof ArrayValueMapperParser) {
        mapper.parse(context);
    } else {

@jpountz jpountz removed the adoptme label Aug 1, 2014
colings86 added a commit that referenced this issue Aug 11, 2014
If a dynamic mapping for a geo_point field is defined and the first document specifies the value of the field as a geo_point array, the dynamic mapping throws an error as the array is broken into individual number before consulting the dynamic mapping configuration.  This change adds a check of the dynamic mapping before the array is split into individual numbers.

Closes #6939
@colings86 colings86 changed the title Geopoint with array broken (dynamic mapping): geo_point expected Mapping: Geopoint with array broken (dynamic mapping): geo_point expected Aug 12, 2014
colings86 added a commit that referenced this issue Sep 8, 2014
If a dynamic mapping for a geo_point field is defined and the first document specifies the value of the field as a geo_point array, the dynamic mapping throws an error as the array is broken into individual number before consulting the dynamic mapping configuration.  This change adds a check of the dynamic mapping before the array is split into individual numbers.

Closes #6939
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants