Multi-field and suggest api error

[mathieu007]

The following error occur while trying to do a simple phrase query on a multi-field using the suggest api:

"failures" : [ {
      "status" : 400,
      "reason" : "ElasticSearchIllegalArgumentException[generator field [title] doesn't exist]"
    }, {
      "status" : 400,
      "reason" : "RemoteTransportException[[Bella Donna][inet[/127.0.0.1:9301]][search/phase/query]]; nested: ElasticSearchIllegalArgumentException[generator field [title] doesn't exist]; "
    } ]

And here is the code:

curl -XDELETE "http://localhost:9200/test?pretty"
curl -XPOST "http://localhost:9200/test?pretty" -d '{
  "settings": {
    "index": {
      "number_of_replicas": 0,
      "analysis":{
        "analyzer":{
          "suggest":{
            "type": "custom",
            "tokenizer": "standard",
            "filter": [ "standard", "lowercase", "suggest_shingle" ]
          }
        },
        "filter":{
          "suggest_shingle":{
            "type": "shingle",
            "min_shingle_size": 2,
            "max_shingle_size": 5,
            "output_unigrams": true
          }
        }
      }
    }
  }
}'
curl -XPOST "http://localhost:9200/test/test/_mapping?pretty" -d '{
  "test": {
 "properties" : {
    "title": {
      "path": "just_name",
      "type": "multi_field",
      "fields": 
        {
          "title": {
            "type": "string",
            "index": "analyzed",
            "similarity": "BM25",
            "analyzer": "suggest"
          }
        }
      }
      
    }
  }
}'


curl -XPOST "http://localhost:9200/test/test?pretty" -d '{
  "title": "Just testing the suggestions api"
}'

curl -XPOST "http://localhost:9200/test/test?pretty" -d '{
  "title": "An other title"
}'

curl -XGET "http://localhost:9200/test/test/_search?pretty" -d '{
  "query": {
    "match_all": {}
  },
  "suggest": {
    "text": "tetsting sugestion",
    "phrase":{
      "phrase":{
        "field": "title",
        "max_errors": 5
      }
    }
  }
}'

[dadoonet]

Does it work if you search on title.title?

[dadoonet]

That's true, it fails.

But when using another sub field in multi field, it works fine:

curl -XDELETE "http://localhost:9200/test?pretty"
curl -XPOST "http://localhost:9200/test?pretty" -d '{
  "settings": {
    "index": {
      "number_of_replicas": 0,
      "analysis":{
        "analyzer":{
          "suggest":{
            "type": "custom",
            "tokenizer": "standard",
            "filter": [ "standard", "lowercase", "suggest_shingle" ]
          }
        },
        "filter":{
          "suggest_shingle":{
            "type": "shingle",
            "min_shingle_size": 2,
            "max_shingle_size": 5,
            "output_unigrams": true
          }
        }
      }
    }
  }
}'
curl -XPOST "http://localhost:9200/test/test/_mapping?pretty" -d '{
  "test": {
 "properties" : {
    "title": {
      "path": "just_name",
      "type": "multi_field",
      "fields": 
        {
           "title": {
            "type": "string",
            "index": "analyzed",
            "similarity": "BM25",
            "analyzer": "suggest"
          },
           "title_suggest": {
            "type": "string",
            "index": "analyzed",
            "similarity": "BM25",
            "analyzer": "suggest"
          }
        }
      }
    }
  }
}'


curl -XPOST "http://localhost:9200/test/test?pretty" -d '{
  "title": "Just testing the suggestions api"
}'

curl -XPOST "http://localhost:9200/test/test?pretty&refresh" -d '{
  "title": "An other title"
}'

curl -XGET "http://localhost:9200/test/test/_search?pretty" -d '{
  "suggest": {
    "text": "tetsting sugestion",
    "phrase":{
      "phrase":{
        "field": "title_suggest",
        "max_errors": 5
      }
    }
  }
}'

I'm going to look at it but I don't think you can use suggest with multifield and only one field declared under (the default one).

[dadoonet]

I found the issue which is not exactly an issue.

If you add a sleep in your script, it works fine:

curl -XDELETE "http://localhost:9200/test?pretty"
curl -XPOST "http://localhost:9200/test?pretty" -d '{
  "settings": {
    "index": {
      "number_of_replicas": 0,
      "analysis":{
        "analyzer":{
          "suggest":{
            "type": "custom",
            "tokenizer": "standard",
            "filter": [ "standard", "lowercase", "suggest_shingle" ]
          }
        },
        "filter":{
          "suggest_shingle":{
            "type": "shingle",
            "min_shingle_size": 2,
            "max_shingle_size": 5,
            "output_unigrams": true
          }
        }
      }
    }
  }
}'

curl -XPOST "http://localhost:9200/test/test/_mapping?pretty" -d '{
  "test": {
 "properties" : {
    "title": {
      "path": "just_name",
      "type": "multi_field",
      "fields": 
        {
           "title": {
            "type": "string",
            "index": "analyzed",
            "similarity": "BM25",
            "analyzer": "suggest"
          }
        }
      }
    }
  }
}'


curl -XPOST "http://localhost:9200/test/test?pretty" -d '{
  "title": "Just testing the suggestions api"
}'

curl -XPOST "http://localhost:9200/test/test?pretty&refresh" -d '{
  "title": "An other title"
}'

sleep 5

curl -XGET "http://localhost:9200/test/test/_search?pretty" -d '{
  "suggest": {
    "text": "tetsting sugestion",
    "phrase":{
      "phrase":{
        "field": "title",
        "max_errors": 5
      }
    }
  }
}'

[dadoonet]

Sorry. I was wrong.

You have two issues here:

• In your script, you need to add a refresh after the last index command
• You have 5 shards but only 2 documents. Suggestion on other empty shards does not work which cause the error you are seeing.

The following script is working fine:

curl -XDELETE "http://localhost:9200/test?pretty"
curl -XPOST "http://localhost:9200/test?pretty" -d '{
  "settings": {
    "index": {
      "number_of_shards": 1,
      "number_of_replicas": 0,
      "analysis":{
        "analyzer":{
          "suggest":{
            "type": "custom",
            "tokenizer": "standard",
            "filter": [ "standard", "lowercase", "suggest_shingle" ]
          }
        },
        "filter":{
          "suggest_shingle":{
            "type": "shingle",
            "min_shingle_size": 2,
            "max_shingle_size": 5,
            "output_unigrams": true
          }
        }
      }
    }
  }
}'

curl -XPOST "http://localhost:9200/test/test/_mapping?pretty" -d '{
  "test": {
 "properties" : {
    "title": {
      "path": "just_name",
      "type": "multi_field",
      "fields": 
        {
           "title": {
            "type": "string",
            "index": "analyzed",
            "similarity": "BM25",
            "analyzer": "suggest"
          }
        }
      }
    }
  }
}'


curl -XPOST "http://localhost:9200/test/test?pretty" -d '{
  "title": "Just testing the suggestions api"
}'

curl -XPOST "http://localhost:9200/test/test?pretty&refresh" -d '{
  "title": "An other title"
}'

curl -XGET "http://localhost:9200/test/test/_search?pretty" -d '{
  "suggest": {
    "text": "tetsting sugestion",
    "phrase":{
      "phrase":{
        "field": "title",
        "max_errors": 5
      }
    }
  }
}'

I keep this issue opened to see if we can not fail in case of empty shards.

[mathieu007]

Ah , yeah your right, i understand now why i received this error, it make sense...
Thanks for pointing me the errors in my script, will remember that for next time.

Math

[nik9000]

I've been noticing that suggestions fail on empty shards problem for the past few days. If no one gets to it the in next few days I'll have a crack at fixing the errors on empty shards.

[dadoonet]

Closing this one as there is now issue #3473 and PR #3475 relative to it.

[s1monw]

reopening since this is really a different issue than having an entirely empty shard. here we can have no value in a field rather than having no docs.