Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NullPointerException if percolate query is used inside bool query #29429

Closed
Lumaraf opened this issue Apr 9, 2018 · 4 comments
Closed

NullPointerException if percolate query is used inside bool query #29429

Lumaraf opened this issue Apr 9, 2018 · 4 comments
Assignees
@martijnvg
Labels
>bug :Search Relevance/Percolator Reverse search: find queries that match a document Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch

Comments

@Lumaraf
Copy link

Lumaraf commented Apr 9, 2018

Elasticsearch version:
6.2.3

JVM version:
1.8.0_161

OS version (uname -a if on a Unix-like system):
Debian 4.9.30-2+deb9u2

Description of the problem including expected versus actual behavior:
A NullPointerException is thrown if a percolate query is used inside a bool query if other should clauses match, but not the percolate query.

Steps to reproduce:

PUT /percolator_bug
{
  "mappings": {
    "docs": {
      "properties": {
        "a": {
          "type": "keyword"
        },
        "b": {
          "type": "percolator"
        }
      }
    }
  }
}

POST /percolator_bug/docs/1?refresh=true
{
  "a": "test"
}

POST /percolator_bug/_search
{
  "query": {
    "bool": {
      "should": [
        {
          "term": {
            "a": "test"
          }
        },
        {
          "percolate": {
            "field": "b",
            "document": {}
          }
        }
      ]
    }
  }
}

Log:

[2018-04-09T12:00:41,319][WARN ][r.suppressed             ] path: /percolator_bug/_search, params: {index=percolator_bug}
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:274) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:132) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:243) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.action.search.InitialSearchPhase.onShardFailure(InitialSearchPhase.java:107) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.action.search.InitialSearchPhase.access$100(InitialSearchPhase.java:49) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.action.search.InitialSearchPhase$2.lambda$onFailure$1(InitialSearchPhase.java:217) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.action.search.InitialSearchPhase.maybeFork(InitialSearchPhase.java:171) [elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.action.search.InitialSearchPhase.access$000(InitialSearchPhase.java:49) [elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.action.search.InitialSearchPhase$2.onFailure(InitialSearchPhase.java:217) [elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.action.search.SearchExecutionStatsCollector.onFailure(SearchExecutionStatsCollector.java:73) [elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:51) [elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:527) [elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1098) [elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1191) [elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1175) [elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:66) [elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.action.search.SearchTransportService$6$1.onFailure(SearchTransportService.java:385) [elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.search.SearchService$2.onFailure(SearchService.java:324) [elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:318) [elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:312) [elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.search.SearchService$3.doRun(SearchService.java:1002) [elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:672) [elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) [elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.2.3.jar:6.2.3]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_161]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_161]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
Caused by: org.elasticsearch.ElasticsearchException$1
	at org.elasticsearch.ElasticsearchException.guessRootCauses(ElasticsearchException.java:619) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.action.search.SearchPhaseExecutionException.guessRootCauses(SearchPhaseExecutionException.java:170) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.action.search.SearchPhaseExecutionException.getCause(SearchPhaseExecutionException.java:111) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.apache.logging.log4j.core.impl.ThrowableProxy.<init>(ThrowableProxy.java:139) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.impl.ThrowableProxy.<init>(ThrowableProxy.java:122) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.impl.MutableLogEvent.getThrownProxy(MutableLogEvent.java:326) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.pattern.ExtendedThrowablePatternConverter.format(ExtendedThrowablePatternConverter.java:64) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.pattern.PatternFormatter.format(PatternFormatter.java:38) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.layout.PatternLayout$PatternSerializer.toSerializable(PatternLayout.java:333) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.layout.PatternLayout.toText(PatternLayout.java:232) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:217) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:57) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146) ~[log4j-core-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.spi.ExtendedLoggerWrapper.logMessage(ExtendedLoggerWrapper.java:217) ~[log4j-api-2.9.1.jar:2.9.1]
	at org.elasticsearch.common.logging.PrefixLogger.logMessage(PrefixLogger.java:102) ~[elasticsearch-core-6.2.3.jar:6.2.3]
	at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2116) ~[log4j-api-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2100) ~[log4j-api-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:1989) ~[log4j-api-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1851) ~[log4j-api-2.9.1.jar:2.9.1]
	at org.apache.logging.log4j.spi.AbstractLogger.warn(AbstractLogger.java:2589) ~[log4j-api-2.9.1.jar:2.9.1]
	at org.elasticsearch.rest.BytesRestResponse.build(BytesRestResponse.java:133) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.rest.BytesRestResponse.<init>(BytesRestResponse.java:96) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.rest.BytesRestResponse.<init>(BytesRestResponse.java:91) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.rest.action.RestActionListener.onFailure(RestActionListener.java:58) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.action.support.TransportAction$1.onFailure(TransportAction.java:91) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.raisePhaseFailure(AbstractSearchAsyncAction.java:222) ~[elasticsearch-6.2.3.jar:6.2.3]
�
more
Caused by: java.lang.NullPointerException
	at org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:675) ~[lucene-core-7.2.1.jar:7.2.1 b2b6438b37073bee1fca40374e85bf91aa457c0b - ubuntu - 2018-01-10 00:48:43]
	at org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:725) ~[lucene-core-7.2.1.jar:7.2.1 b2b6438b37073bee1fca40374e85bf91aa457c0b - ubuntu - 2018-01-10 00:48:43]
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:462) ~[lucene-core-7.2.1.jar:7.2.1 b2b6438b37073bee1fca40374e85bf91aa457c0b - ubuntu - 2018-01-10 00:48:43]
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:583) ~[lucene-core-7.2.1.jar:7.2.1 b2b6438b37073bee1fca40374e85bf91aa457c0b - ubuntu - 2018-01-10 00:48:43]
	at org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:568) ~[lucene-core-7.2.1.jar:7.2.1 b2b6438b37073bee1fca40374e85bf91aa457c0b - ubuntu - 2018-01-10 00:48:43]
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:493) ~[lucene-core-7.2.1.jar:7.2.1 b2b6438b37073bee1fca40374e85bf91aa457c0b - ubuntu - 2018-01-10 00:48:43]
	at org.elasticsearch.percolator.PercolatorMatchedSlotSubFetchPhase.hitsExecute(PercolatorMatchedSlotSubFetchPhase.java:90) ~[?:?]
	at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:170) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:376) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:351) ~[elasticsearch-6.2.3.jar:6.2.3]
	at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:316) ~[elasticsearch-6.2.3.jar:6.2.3]
	... 9 more
@martijnvg martijnvg added the :Search Relevance/Percolator Reverse search: find queries that match a document label Apr 9, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search-aggs

@martijnvg martijnvg self-assigned this Apr 9, 2018
@martijnvg
Copy link
Member

martijnvg commented Apr 9, 2018 •
edited
Loading

@Lumaraf Thanks for reporting this bug and sharing an easy way to reproduce it.

What I do wonder is why you are trying to use a percolate query inside a percolator query? Although the query dsl will allow this, we haven't really thought about this usage of the percolate query and as you have seen doesn't work as expected.

@Lumaraf
Copy link
Author

Lumaraf commented Apr 10, 2018

I don't have percolate queries stored in percolator fields. For the example mapping i only reduced the fields to the absolute minimum necessary to reproduce the bug. Fields used by the percolator and those stored in the documents are kept separate in my actual use case.

@martijnvg martijnvg mentioned this issue Apr 10, 2018
Merged
@martijnvg
Copy link
Member

@Lumaraf Ok, that makes sense.

martijnvg added a commit that referenced this issue Apr 10, 2018
@martijnvg
Fixed bug when non percolator docs end up in the search hits.
6964a95 
In the case that a document with a percolator field is matched when using the `percolate` query then
the fetch phase can fail due to the fact that the percolator can't resolve any query from that document.

Closes #29429
martijnvg added a commit that referenced this issue Apr 10, 2018
@martijnvg
Fixed bug when non percolator docs end up in the search hits.
684f3d8 
In the case that a document with a percolator field is matched when using the `percolate` query then
the fetch phase can fail due to the fact that the percolator can't resolve any query from that document.

Closes #29429
@javanna javanna added the Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch label Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@martijnvg martijnvg

Labels
>bug :Search Relevance/Percolator Reverse search: find queries that match a document Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@martijnvg @javanna @Lumaraf @elasticmachine