Scripting: ctx[_version] in update by query returns -1 since 6.7

[stu-elastic]

In TransportUpdateByQueryAction we always create AsyncIndexBySearchAction with needsSourceDocumentVersions=false, so the versions are never fetched. This was on the switch to sequence numbers: #37857.

The documentation still refers to ctx['_version'] and we still populate that key in the context.

[elasticmachine]

Pinging @elastic/es-core-infra (:Core/Infra/Scripting)

[stu-elastic]

TODO: test passing true to fetch version

[stu-elastic]

After checking fetch version, let's ask the distrib folks.

[Metalmacher]

Any update on this? Issue still exists in 7.16.2. Additionally, a similar problem can be observed when trying to set _version from an ingest pipeline:

PUT _ingest/pipeline/version_searchable
{
    "description" : "Moves _version to indexable field version",
    "processors" : [
      {
        "script" : {
          "source" : "ctx.version = ctx._version;"
        }
      }
    ]
}
POST testing_index/_update_by_query?pipeline=version_searchable

Resulting field has -3 for all documents, regardless of the actual _version.

[stu-elastic]

To get the version as part of the document you need to pass in version: true to the update by query request.

[stu-elastic]

version: true is no longer being honored.

diff --git a/modules/reindex/src/main/java/org/elasticsearch/reindex/TransportUpdateByQueryAction.java b/modules/reindex/src/main/java/org/elasticsearch/reindex/TransportUpdateByQueryAction.java
index 0f1a98ac085..6674a187cdd 100644
--- a/modules/reindex/src/main/java/org/elasticsearch/reindex/TransportUpdateByQueryAction.java
+++ b/modules/reindex/src/main/java/org/elasticsearch/reindex/TransportUpdateByQueryAction.java
@@ -18,6 +18,7 @@ import org.elasticsearch.client.internal.ParentTaskAssigningClient;
 import org.elasticsearch.cluster.ClusterState;
 import org.elasticsearch.cluster.service.ClusterService;
 import org.elasticsearch.common.inject.Inject;
+import org.elasticsearch.core.Booleans;
 import org.elasticsearch.index.reindex.BulkByScrollResponse;
 import org.elasticsearch.index.reindex.BulkByScrollTask;
 import org.elasticsearch.index.reindex.ScrollableHitSource;
@@ -101,7 +102,7 @@ public class TransportUpdateByQueryAction extends HandledTransportAction<UpdateB
             super(
                 task,
                 // use sequence number powered optimistic concurrency control
-                false,
+                Boolean.TRUE.equals(request.getSearchRequest().source().version()),
                 true,
                 logger,
                 client,