Elasticsearch 7 reports a different key_as_string for dayOfWeek for date_histogram aggs than Elasticsearch 6 did

[centic9]

Elasticsearch version (bin/elasticsearch --version): 6.7.2 and 7.1.1

Plugins installed: []

JVM version (java -version): Java HotSpot(TM) 64-Bit Server VM/11.0.1/11.0.1+13-LTS

OS version (uname -a if on a Unix-like system): Windows 10

Description of the problem including expected versus actual behavior:

We are looking at upgrading from Elasticsearch 6.x to 7.x and one of our tests related to day-of-week in date-histogramm aggregations is failing. When investigating a bit more we found that there is actually a different response for the "key_as_string" for buckets of a date_histogramm aggregation which aggregates on "day of week" in Elasticsearch 7 compared to Elasticsearch 6.

I could not find any "breaking change" mentioned for this, so I believe this may be an unintended side-effect of switching from Joda Time to Java DateTime API (https://www.elastic.co/guide/en/elasticsearch/reference/7.0/breaking-changes-7.0.html#breaking_70_java_time_changes)?

Steps to reproduce:

curl -X PUT "localhost:9200/twitter/_doc/1" -H 'Content-Type: application/json' -d'
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
'

curl -XPOST 'http://localhost:9200/twitter/_search' -d '{
 "size": 0,
 "aggregations": {
  "q2": {
   "date_histogram": {
    "field": "post_date",
    "format": "e",
    "interval": "day",
    "offset": 0
   }
  }
 }
}'

On Elasticsearch 6.7.2 this returns:

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 0,
    "hits": [

    ]
  },
  "aggregations": {
    "q2": {
      "buckets": [
        {
          "key_as_string": "7",
          "key": 1258243200000,
          "doc_count": 1
        }
      ]
    }
  }
}

however on Elasticsearch 7.1.1 is returns (note the difference in "key_as_string":

{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": null,
    "hits": [

    ]
  },
  "aggregations": {
    "q2": {
      "buckets": [
        {
          "key_as_string": "1",
          "key": 1258243200000,
          "doc_count": 1
        }
      ]
    }
  }
}

Provide logs (if relevant):

[elasticmachine]

Pinging @elastic/es-analytics-geo

[DaveCTurner]

This reproduces for me. It looks like Joda (and hence Elasticsearch ≤6) maps Monday to 1 whereas Java (and hence Elasticsearch ≥7) maps Sunday to 1.

[elasticmachine]

Pinging @elastic/es-core-infra

[rjernst]

While this is definitely fixable, I think we might be past the point of reversing a breaking, and introducing a breaking change in and of itself. I'll mark this for discussion.

[centic9]

Seems not many people are using it in the first place if it went unnoticed for such a long time.

Also I would expect that most large deployments did not start upgrades to 7 yet, so the majority of affected users might not even have started to look at 7.x.

We are running a number of clusters and are upgrading gradually across them, so having to distinguish between v6 and v7 for this would be fairly painful on the client side.

[pgomulka]

relates #42588