Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] NPE happens on short prefix query on a field with index_prefixes #2826

Closed
woby opened this issue Apr 11, 2022 · 3 comments
Closed

[BUG] NPE happens on short prefix query on a field with index_prefixes #2826

woby opened this issue Apr 11, 2022 · 3 comments
Assignees
Labels
bug Something isn't working Indexing & Search v2.0.0 Version 2.0.0

Comments

@woby
Copy link

woby commented Apr 11, 2022

Describe the bug
( index_prefixes is not documented in OpenSearch’s documentation but AFAIK OpenSearch is compatible with ElasticSearch 7.10, which has index_prefixes parameter.)

When a prefix query run on a field with index_prefixes, if the length of characters in the query is [min_chars of the index_prefixes option] - 1, an error is responded.
The root cause seems to be NullPointerException.

To Reproduce
Steps to reproduce the behavior:

  1. Create an index which has a field with index_prefixes option; its min_chars is implicitly 2.
PUT test
{
  "mappings": {
    "properties": {
      "t": {
        "type": "text",
        "index_prefixes": {}
      }
    }
  }
}
  1. Search by prefix query with one character on the field.
GET test/_search
{"query":{"prefix":{"t": "a"}}}
  1. An error is responded. It says the root cause is NullPointerException.
{
  "error" : {
    "root_cause" : [
      {
        "type" : "query_shard_exception",
        "reason" : "failed to create query: Cannot invoke \"Object.hashCode()\" because \"this.rewriteMethod\" is null",
        "index" : "test",
        "index_uuid" : "XPqfSP1xT7WJt9Sxzt65qg"
      }
    ],
    "type" : "search_phase_execution_exception",
    "reason" : "all shards failed",
    "phase" : "query",
    "grouped" : true,
    "failed_shards" : [
      {
        "shard" : 0,
        "index" : "test",
        "node" : "QA35rulpSza20Wsx6rNSEg",
        "reason" : {
          "type" : "query_shard_exception",
          "reason" : "failed to create query: Cannot invoke \"Object.hashCode()\" because \"this.rewriteMethod\" is null",
          "index" : "test",
          "index_uuid" : "XPqfSP1xT7WJt9Sxzt65qg",
          "caused_by" : {
            "type" : "null_pointer_exception",
            "reason" : "Cannot invoke \"Object.hashCode()\" because \"this.rewriteMethod\" is null"
          }
        }
      }
    ]
  },
  "status" : 400
}
  1. simple_query_string also fails.
# fails with the same error
GET test/_search
{"query":{"simple_query_string":{"fields":["t"], "query":"a*"}}}

Expected behavior
Run without errors. I know it is not efficient.

A similar query_string runs without any errors.

GET test/_search
{"query":{"query_string":{"fields":["t"], "query":"a*"}}}

I tried some patterns (min_chars = 2, 3, 5), and it seems that the prefix query with characters the length of which is min_chars - 1 fails. When the length is less than min_chars - 1, the query succeeds.

Plugins
N/A

Screenshots
N/A

Host/Environment (please complete the following information):

  • Version: 1.3.1

Additional context
Add any other context about the problem here.

@woby woby added bug Something isn't working untriaged labels Apr 11, 2022
@dblock dblock added the v2.0.0 Version 2.0.0 label Apr 11, 2022
@dblock
Copy link
Member

dblock commented Apr 11, 2022

@woby looks like a bug - if you/someone wants to contribute a unit test for this we can fix it faster

@saratvemulapalli
Copy link
Member

I took a stab at this to understand the problem.
I was able to reproduce the problem with the steps on the issue.

I did observe similar pattern as @woby.
When the search query is has one char less than min_chars, a Null pointer exception occurs.

I did the tests and only 3. (min_chars -1) is seeing problems.
From what I understand, Automaton query is created for the problematic search query with rewrite being null.

Also a side note, Lucene throws a null pointer exception but OpenSearch handles it and does not crash. Which says, it only impacts this feature as a customer, but does not take down the cluster fatally.

Setting up the index:

 % curl -XPUT localhost:9200/test --data '{                                                                             
"mappings": {
    "properties": {
      "t": {
        "type": "text",
        "index_prefixes": { "min_chars": "3" }
      }
    }
  }
}' -H "Content-Type:Application/json"
  1. Request: % curl 'localhost:9200/test/_search?pretty' --data '{"query":{"prefix":{"t": ""}}}' -H "Content-Type:Application/json"
    Logs:
[2022-04-12T00:16:26,203][INFO ][o.o.s.SearchService      ] [runTask-0] $SARAT$ 0. org.opensearch.search.internal.ReaderContext@6a02f968
[2022-04-12T00:16:26,212][INFO ][o.o.s.SearchService      ] [runTask-0] $SARAT$ 1. [ppa1kh7BSnmWJz-ZGfCwew][test][0] query=[t:*]
[2022-04-12T00:16:26,213][INFO ][o.o.s.DefaultSearchContext] [runTask-0] $SARAT$ 2. t:*
  1. Request: % curl 'localhost:9200/test/_search?pretty' --data '{"query":{"prefix":{"t": "s"}}}' -H "Content-Type:Application/json"
    Logs:
[2022-04-12T00:16:32,023][INFO ][o.o.s.SearchService      ] [runTask-0] $SARAT$ 0. org.opensearch.search.internal.ReaderContext@2e906b2f
[2022-04-12T00:16:32,024][INFO ][o.o.s.SearchService      ] [runTask-0] $SARAT$ 1. [ppa1kh7BSnmWJz-ZGfCwew][test][0] query=[t:s*]
[2022-04-12T00:16:32,024][INFO ][o.o.s.DefaultSearchContext] [runTask-0] $SARAT$ 2. t:s*
  1. Request: % curl 'localhost:9200/test/_search?pretty' --data '{"query":{"prefix":{"t": "st"}}}' -H "Content-Type:Application/json"
    Logs:
[2022-04-12T00:16:36,704][INFO ][o.o.s.SearchService      ] [runTask-0] $SARAT$ 0. org.opensearch.search.internal.ReaderContext@30e8328c
[2022-04-12T00:16:36,707][INFO ][o.o.s.SearchService      ] [runTask-0] $SARAT$ 1. [ppa1kh7BSnmWJz-ZGfCwew][test][0] query=[ConstantScore(t._index_prefix:AutomatonQuery {
org.apache.lucene.util.automaton.Automaton@53a98242} t:st)]
[2022-04-12T00:16:36,707][INFO ][o.o.s.DefaultSearchContext] [runTask-0] $SARAT$ 2. ConstantScore(t._index_prefix:AutomatonQuery {
org.apache.lucene.util.automaton.Automaton@53a98242} t:st)
org.opensearch.action.search.SearchPhaseExecutionException: all shards failed
        at org.opensearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:642) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:360) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:677) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:457) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.search.AbstractSearchAsyncAction$1.onFailure(AbstractSearchAsyncAction.java:291) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.search.SearchExecutionStatsCollector.onFailure(SearchExecutionStatsCollector.java:102) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:72) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:593) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.TransportService$6.handleException(TransportService.java:735) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1350) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1459) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1433) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:74) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.TransportChannel.sendErrorResponse(TransportChannel.java:69) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.support.ChannelActionListener.onFailure(ChannelActionListener.java:65) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.ActionRunnable.onFailure(ActionRunnable.java:101) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:57) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:792) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:50) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
        at java.lang.Thread.run(Thread.java:832) [?:?]
Caused by: org.opensearch.OpenSearchException$1: Cannot invoke "org.apache.lucene.search.MultiTermQuery$RewriteMethod.rewrite(org.apache.lucene.index.IndexReader, org.apache.lucene.search.MultiTermQuery)" because "this.rewriteMethod" is null
        at org.opensearch.OpenSearchException.guessRootCauses(OpenSearchException.java:679) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:358) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        ... 21 more
Caused by: java.lang.NullPointerException: Cannot invoke "org.apache.lucene.search.MultiTermQuery$RewriteMethod.rewrite(org.apache.lucene.index.IndexReader, org.apache.lucene.search.MultiTermQuery)" because "this.rewriteMethod" is null
        at org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:283) ~[lucene-core-9.1.0.jar:9.1.0 5b522487ba8e0f1002b50a136817ca037aec9686 - jtibs - 2022-03-16 10:32:40]
        at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:277) ~[lucene-core-9.1.0.jar:9.1.0 5b522487ba8e0f1002b50a136817ca037aec9686 - jtibs - 2022-03-16 10:32:40]
        at org.apache.lucene.search.ConstantScoreQuery.rewrite(ConstantScoreQuery.java:44) ~[lucene-core-9.1.0.jar:9.1.0 5b522487ba8e0f1002b50a136817ca037aec9686 - jtibs - 2022-03-16 10:32:40]
        at org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:789) ~[lucene-core-9.1.0.jar:9.1.0 5b522487ba8e0f1002b50a136817ca037aec9686 - jtibs - 2022-03-16 10:32:40]
        at org.opensearch.search.internal.ContextIndexSearcher.rewrite(ContextIndexSearcher.java:170) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.search.DefaultSearchContext.preProcess(DefaultSearchContext.java:322) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.search.query.QueryPhase.preProcess(QueryPhase.java:123) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.search.SearchService.createContext(SearchService.java:842) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.search.SearchService.executeQueryPhase(SearchService.java:505) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.search.SearchService$2.lambda$onResponse$0(SearchService.java:478) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:71) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.ActionRunnable$2.doRun(ActionRunnable.java:86) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:50) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
  1. Request: % curl 'localhost:9200/test/_search?pretty' --data '{"query":{"prefix":{"t": "str"}}}' -H "Content-Type:Application/json"
    Logs:
[2022-04-12T09:42:27,874][INFO ][o.o.s.SearchService      ] [runTask-0] $SARAT$ 0. org.opensearch.search.internal.ReaderContext@2f7504a8
[2022-04-12T09:42:27,884][INFO ][o.o.s.SearchService      ] [runTask-0] $SARAT$ 1. [T_NdxJzfSPaO1kdIoga3mQ][test][0] query=[ConstantScore(t._index_prefix:str)]
[2022-04-12T09:42:27,884][INFO ][o.o.s.DefaultSearchContext] [runTask-0] $SARAT$ 2. ConstantScore(t._index_prefix:str)

@saratvemulapalli
Copy link
Member

Thanks @VachaShah for taking care of this.

@woby the changes are merged to 2.0 and will be available in the next release.
Feel free to re-open if you have follow up questions, and thanks for reporting this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Indexing & Search v2.0.0 Version 2.0.0
Projects
None yet
Development

No branches or pull requests

5 participants