_field_stats call on geo_point field broken after upgrading from 2.4.4 -> 5.3.1

[n0othing]

Elasticsearch version: 2.4.4/5.3.1

A _field_stats call on a type geo_point field throws an exception for an index that was created in 2.4.4 and upgraded to 5.3.1. I also reproduced this on going from 2.3.3 -> 5.3.0.

This causes Kibana to not properly grab the index mappings when defining an index pattern rendering all fields as neither searchable nor aggregatable.

Steps to reproduce:

1. Create an index mapping with a geo_point field

PUT index
{
  "mappings": {
    "type": {
      "properties": {
        "geo_field": {
          "type": "geo_point"
        }
      }
    }
  }
}

2. Add a sample document

PUT index/type/1
{
  "geo_field": "33.8957, -112.0577"
}

3. Upgrade to 5.3.1 (I simply copied the data directory over)

4. Attempt a _field_stats call on the geo_field

GET index/_field_stats?fields=geo_field

The response

{
  "_shards": {
    "total": 5,
    "successful": 4,
    "failed": 1,
    "failures": [
      {
        "shard": 3,
        "index": "index",
        "status": "INTERNAL_SERVER_ERROR",
        "reason": {
          "type": "exception",
          "reason": "java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: 5",
          "caused_by": {
            "type": "execution_exception",
            "reason": "java.lang.ArrayIndexOutOfBoundsException: 5",
            "caused_by": {
              "type": "array_index_out_of_bounds_exception",
              "reason": "5"
            }
          }
        }
      }
    ]
  },
  "indices": {
    "_all": {
      "fields": {}
    }
  }
}

[jimczi]

This seems to be a deserialization error with geo_point encoding in 2.x. @nknize can you take a look ?

[clintongormley]

@n0othing also note that field stats is deprecated in favour of the new field caps API (5.4)

[Bargs]

We've had a couple reports of this affecting Kibana users already. Because we use field stats to figure out the searchable/aggregatable status of fields this effectively breaks Kibana for any index patterns containing a geo_point field after upgrading to 5.3.1 from 2.x

elastic/kibana#11379
elastic/kibana#11377
elastic/kibana#9571 (comment)

[nknize]

PR opened.... #24534

[ppf2]

Per the PR, this is fixed in 5.3.3 and 5.4.1, should we close?

[kobelb]

I'm still seeing this behavior when upgrading from 2.x to 5.4.1 and 5.3.3

[nknize]

There was a reversed ternary logic bug that wasn't caught by the munged test. Opened fix at #25211 for 5.4.2 release /cc @jimczi @clintongormley

[nknize]

fix is merged in #25211