Clarify realtime get API documentation

[dadoonet]

Reported at: https://discuss.elastic.co/t/the-id-query-will-trigger-refresh-but-i-still-cant-search-with-query-qsl/195829

The documentation states:

If a document has been updated but is not yet refreshed, the get API will issue a refresh call in-place to make the document visible. This will also make other documents changed since the last refresh visible.

But the following script does not seem to prove it:

DELETE test
PUT test 
{
  "settings": {
    "refresh_interval": "-1"
  }
}
PUT test/_doc/1
{
  "foo": "bar"
}

This gives empty results:

GET test/_search

This gives empty results although we called the GET API:

GET test/_doc/1
GET test/_search

This works when we manually refresh the index:

POST test/_refresh
GET test/_search

So the documentation is not entirely exact about what is happening behind the scene. May be it's because I have only one document and some threshold are involved here?

[unlikezy]

I encounter the exactly same situation.
What I want to add is:
if you disable realtime when get

GET test/_doc/1?realtime=false

the result will be:

{
  "_index" : "test",
  "_type" : "_doc",
  "_id" : "1",
  "found" : false
}

and if you enable realtime, which is default, you can get the doc:

GET test/_doc/1?realtime=true

{
  "_index" : "test",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 1,
  "_seq_no" : 0,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "foo" : "bar"
  }
}

Changing HTTP method from GET to HEAD in above examples also illustrate this problem, but might see this more clearly.

I am curious about why GET work without refreshing data into Lucene index, and how is the performance.

[elasticmachine]

Pinging @elastic/es-distributed

[elasticmachine]

Pinging @elastic/es-docs

[jrodewig]

[docs issue triage]

Leaving open as this is still relevant.

[ywelsch]

The documentation was imprecise here. ES distinguishes between internal and external refreshes. Internal refreshes are used to build segments, whereas external refreshes are used to expose these newly built segments. In case where a document is accessed using a realtime get, and the document is still in the Lucene buffer (i.e. not internally refreshed yet), Elasticsearch triggers an internal refresh. This is why you can't observed the "internally refreshed" document.

Note that all of this is changing in ES 7.6+ (see #48843). From that version on, realtime get will not trigger a refresh anymore, making all these explanations moot. The docs in newer versions have been updated to that effect.