Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

epoch_millis filter doesn't allow negative values #39375

Closed
flash1293 opened this issue Feb 25, 2019 · 4 comments
Closed

epoch_millis filter doesn't allow negative values #39375

flash1293 opened this issue Feb 25, 2019 · 4 comments
Labels
:Core/Infra/Core Core issues without another label >docs General docs changes

Comments

@flash1293
Copy link
Contributor

Prior to 7.0 it was possible to use negative values in an epoc_millis range filter. This now fails with a parse exception

Steps to reproduce:

PUT my_index/_doc/1
{
  "datefield": "2015-03-03"
}

GET my_index/_search
{
  "query": {
    "range": {
      "datefield": {
        "gte": -500,
        "format": "epoch_millis"
      }
    }
  }
}

Version 6.6 correctly returns the search result containing one hit, 7.0.0 gives

{
  "error": {
    "root_cause": [
      {
        "type": "parse_exception",
        "reason": "failed to parse date field [-500] with format [epoch_millis]: [Text '-500' could not be parsed: Invalid value for NanoOfSecond (valid values 0 - 999999999): -500000000]"
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "my_index",
        "node": "Q0HRRl8ISLuL5b64A9n3IA",
        "reason": {
          "type": "parse_exception",
          "reason": "failed to parse date field [-500] with format [epoch_millis]: [Text '-500' could not be parsed: Invalid value for NanoOfSecond (valid values 0 - 999999999): -500000000]",
          "caused_by": {
            "type": "date_time_parse_exception",
            "reason": "Text '-500' could not be parsed: Invalid value for NanoOfSecond (valid values 0 - 999999999): -500000000",
            "caused_by": {
              "type": "date_time_exception",
              "reason": "Invalid value for NanoOfSecond (valid values 0 - 999999999): -500000000"
            }
          }
        }
      }
    ]
  },
  "status": 400
}

Server log stack trace:

 info archService.java:582) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:385) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.search.SearchService.access$100(SearchService.java:123) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:357) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:353) [elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.search.SearchService$4.doRun(SearchService.java:1068) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:751) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
   │      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
   │      	at java.lang.Thread.run(Thread.java:834) ~[?:?]
   │      Caused by: java.time.DateTimeException: Invalid value for NanoOfSecond (valid values 0 - 999999999): -500000000
   │      	at java.time.temporal.ValueRange.checkValidValue(ValueRange.java:311) ~[?:?]
   │      	at java.time.temporal.ChronoField.checkValidValue(ChronoField.java:717) ~[?:?]
   │      	at java.time.format.Parsed.resolveTimeFields(Parsed.java:454) ~[?:?]
   │      	at java.time.format.Parsed.resolveFields(Parsed.java:321) ~[?:?]
   │      	at java.time.format.Parsed.resolve(Parsed.java:253) ~[?:?]
   │      	at java.time.format.DateTimeParseContext.toResolved(DateTimeParseContext.java:331) ~[?:?]
   │      	at java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2052) ~[?:?]
   │      	at java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1874) ~[?:?]
   │      	at org.elasticsearch.common.time.JavaDateMathParser.parseDateTime(JavaDateMathParser.java:221) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.common.time.JavaDateMathParser.parse(JavaDateMathParser.java:75) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.index.mapper.DateFieldMapper$DateFieldType.parseToLong(DateFieldMapper.java:397) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.index.mapper.DateFieldMapper$DateFieldType.isFieldWithinQuery(DateFieldMapper.java:411) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.index.query.RangeQueryBuilder.getRelation(RangeQueryBuilder.java:447) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.index.query.RangeQueryBuilder.doRewrite(RangeQueryBuilder.java:464) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.index.query.AbstractQueryBuilder.rewrite(AbstractQueryBuilder.java:265) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.search.builder.SearchSourceBuilder.rewrite(SearchSourceBuilder.java:962) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.search.builder.SearchSourceBuilder.rewrite(SearchSourceBuilder.java:82) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:68) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:51) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.search.internal.ShardSearchLocalRequest$RequestRewritable.rewrite(ShardSearchLocalRequest.java:291) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.search.internal.ShardSearchLocalRequest$RequestRewritable.rewrite(ShardSearchLocalRequest.java:281) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:68) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.search.SearchService.createSearchContext(SearchService.java:659) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.search.SearchService.createSearchContext(SearchService.java:638) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.search.SearchService.createContext(SearchService.java:601) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:582) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:385) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.search.SearchService.access$100(SearchService.java:123) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:357) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:353) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.search.SearchService$4.doRun(SearchService.java:1068) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:751) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.0.0-SNAPSHOT.jar:7.0.0-SNAPSHOT]
   │      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
   │      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
   │      	at java.lang.Thread.run(Thread.java:834) ~[?:?]

Is this an intended breaking change? I don't find anything about this is the change docs but maybe I'm looking for the wrong keywords.
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra

@jasontedor jasontedor added :Core/Infra/Core Core issues without another label >docs General docs changes labels Feb 25, 2019
@jasontedor
Copy link
Member

These are deprecated in #36793 indeed.

@rjernst Can you ensure that a note is added to the 7.0.0 migration guide?

rjernst added a commit to rjernst/elasticsearch that referenced this issue Mar 6, 2019
@rjernst
Add note about negative timestamps to migration guide
945ee96 
This commit adds a note to the migration guide for 7.0 about negative
epoch timestamps no longer being supported.

closes elastic#39375
@rjernst rjernst mentioned this issue Mar 6, 2019
Merged
@rjernst
Copy link
Member

rjernst commented Mar 6, 2019

I opened #39734

rjernst added a commit that referenced this issue Mar 6, 2019
@rjernst
Add note about negative timestamps to migration guide (#39734)
7da62d3 
This commit adds a note to the migration guide for 7.0 about negative
epoch timestamps no longer being supported.

closes #39375
rjernst added a commit that referenced this issue Mar 6, 2019
@rjernst
Add note about negative timestamps to migration guide (#39734)
c77251b 
This commit adds a note to the migration guide for 7.0 about negative
epoch timestamps no longer being supported.

closes #39375
@jasontedor
Copy link
Member

Closed by #39734

@flash1293 flash1293 mentioned this issue Apr 1, 2019
Closed
@hanoch hanoch mentioned this issue Apr 8, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
:Core/Infra/Core Core issues without another label >docs General docs changes
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@rjernst @flash1293 @jasontedor @elasticmachine