Inconsistent behavior when loading docvalues for range fields

[jtibshirani]

Say we've indexed a document with an integer_range and date_range using version 7.4 (or above):

PUT range_index
{
  "mappings": {
    "properties": {
      "expected_attendees": {
        "type": "integer_range"
      },
      "time_frame": {
        "type": "date_range", 
        "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
      }
    }
  }
}

PUT range_index/_doc/1?refresh
{
  "expected_attendees" : { 
    "gte" : 10,
    "lte" : 20
  },
  "time_frame" : { 
    "gte" : "2015-10-31 12:00:00", 
    "lte" : "2015-11-01"
  }
}

If we specify "docvalues_fields": ["time_frame"] during a search, we get an unsupported operation exception because of a failure in DocValueFormat.format:

Caused by: java.lang.UnsupportedOperationException
	at org.elasticsearch.search.DocValueFormat.format(DocValueFormat.java:70) ~[elasticsearch-7.4.1.jar:7.4.1]
	at org.elasticsearch.search.fetch.subphase.DocValueFieldsFetchSubPhase.hitsExecute(DocValueFieldsFetchSubPhase.java:159) ~[elasticsearch-7.4.1.jar:7.4.1]
	at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:177) ~[elasticsearch-7.4.1.jar:7.4.1]

When specifying "docvalues_fields": ["expected_attendees"], we don't get an error but a binary representation is returned.

This behavior seems quite confusing. To me it would be best if we either (1) always returned a nicely formatted representation of the range, or (2) always threw a descriptive error.

[elasticmachine]

Pinging @elastic/es-search (:Search/Mapping)

[jtibshirani]

It looks like there was different behavior prior to 7.4: when requesting docvalues_fields for "time_frame" or "expected_attendees", we always get an error of the form "Fielddata is not supported on field [time_frame] of type [date_range]". I wonder if some of our recent work on running aggregations on range fields caused the change in behavior.

[elasticmachine]

Pinging @elastic/es-analytics-geo (:Analytics/Aggregations)

[romseygeek]

Histogram fields have a clear separation here in that they explicitly throw an error when you ask for their ScriptDocValues. I think we should extend this to range fields as well? Either that or expose an object of some kind that returns the range bounds so that they can be used in scripts.

[jtibshirani]

We decided not to invest in fixing this, so I'm closing it out. The recommended way to load these values is through the new fields option instead of docvalue_fields. As part of our testing efforts around field loading (#70929) we could look into throwing clearer errors for field types that aren't supported.