Runtime fields to optionally ignore script errors

docs/changelog/92380.yaml

@@ -0,0 +1,5 @@
+pr: 92380
+summary: Add option to allow script errors on runtime fields
+area: Search
+type: enhancement
+issues: []

docs/reference/mapping/runtime.asciidoc

@@ -53,7 +53,7 @@ combined use less resources and reduce your operating costs.
 Runtime fields can replace many of the ways you can use scripting with the
 `_search` API. How you use a runtime field is impacted by the number of
 documents that the included script runs against. For example, if you're using
-the `fields` parameter on the `_search` API to 
+the `fields` parameter on the `_search` API to
 <<runtime-retrieving-fields,retrieve the values of a runtime field>>, the script
 runs only against the top hits just like script fields do.
 
@@ -77,7 +77,7 @@ If you move a script from any of these sections in a search request to a
 runtime field that is computing values from the same number of documents, the
 performance should be about the same. The performance for these features is
 largely dependent upon the calculations that the included script is running and
-how many documents the script runs against. 
+how many documents the script runs against.
 
 [discrete]
 [[runtime-compromises]]
@@ -88,9 +88,9 @@ the runtime script.
 
 To balance search performance and flexibility, index fields that you'll
 frequently search for and filter on, such as a timestamp. {es} automatically
-uses these indexed fields first when running a query, resulting in a fast 
-response time. You can then use runtime fields to limit the number of fields 
-that {es} needs to calculate values for. Using indexed fields in tandem with 
+uses these indexed fields first when running a query, resulting in a fast
+response time. You can then use runtime fields to limit the number of fields
+that {es} needs to calculate values for. Using indexed fields in tandem with
 runtime fields provides flexibility in the data that you index and how you
 define queries for other fields.
 
@@ -111,7 +111,7 @@ You map runtime fields by adding a `runtime` section under the mapping
 definition and defining
 <<modules-scripting-using,a Painless script>>. This script has access to the
 entire context of a document, including the original `_source` via `params._source`
-and any mapped fields plus their values. At query time, the script runs and 
+and any mapped fields plus their values. At query time, the script runs and
 generates values for each scripted field that is required for the query.
 
 .Emitting runtime field values
@@ -227,6 +227,16 @@ with `params._source` (such as `params._source.day_of_week`). For simplicity,
 defining a runtime field in the mapping definition without a script is the
 recommended option, whenever possible.
 
+[[runtime-errorhandling]]
+==== Ignoring script errors on runtime fields
+
+Scripts can throw errors at runtime, e.g. on accessing missing or invalid values
+in documents or because of performing invalid operations. The `on_script_error`
+parameter can be used to control error behaviour when this happens. Setting this
+parameter to `continue` will have the effect of silently ignoring all errors on
+this runtime field. The default `fail` value will cause a shard failure which
+gets reported in the search response.
+
 [[runtime-updating-scripts]]
 ==== Updating and removing runtime fields
 
@@ -932,14 +942,14 @@ can define runtime fields in the
 decide to index a runtime field for greater performance, just move the full
 runtime field definition (including the script) to the context of an index
 mapping. {es} automatically uses these indexed fields to drive queries,
-resulting in a fast response time. This capability means you can write a 
+resulting in a fast response time. This capability means you can write a
 script only once, and apply it to any context that supports runtime fields.
 
 NOTE: Indexing a `composite` runtime field is currently not supported.
 
-You can then use runtime fields to limit the number of fields that {es} needs 
-to calculate values for. Using indexed fields in tandem with runtime fields 
-provides flexibility in the data that you index and how you define queries for 
+You can then use runtime fields to limit the number of fields that {es} needs
+to calculate values for. Using indexed fields in tandem with runtime fields
+provides flexibility in the data that you index and how you define queries for
 other fields.
 
 IMPORTANT: After indexing a runtime field, you cannot update the included
@@ -1417,9 +1427,9 @@ GET my-index-000001/_search
 
 [[runtime-examples-grok-composite]]
 ==== Define a composite runtime field
-You can also define a _composite_ runtime field to emit multiple fields from a 
-single script. You can define a set of typed subfields and emit a map of 
-values. At search time, each subfield retrieves the value associated with 
+You can also define a _composite_ runtime field to emit multiple fields from a
+single script. You can define a set of typed subfields and emit a map of
+values. At search time, each subfield retrieves the value associated with
 their name in the map. This means that you only need to specify your grok
 pattern one time and can return multiple values:
 
@@ -1467,11 +1477,11 @@ GET my-index-000001/_search
 ----
 // TEST[continued]
 
-The API returns the following result. Because `http` is a `composite` runtime 
+The API returns the following result. Because `http` is a `composite` runtime
 field, the response includes each of the sub-fields under `fields`, including
-any associated values that match the query. Without building your data structure 
+any associated values that match the query. Without building your data structure
 in advance, you can search and explore your data in meaningful ways to
-experiment and determine which fields to index. 
+experiment and determine which fields to index.
 
 [source,console-result]
 ----

modules/runtime-fields-common/src/yamlRestTest/resources/rest-api-spec/test/runtime_fields/14_keyword_errors.yml

@@ -0,0 +1,216 @@
+---
+setup:
+  - do:
+      indices.create:
+        index: testindex
+        body:
+          settings:
+            number_of_shards: 1
+          mappings:
+            runtime:
+              first_char:
+                type: keyword
+                script: |
+                  emit(doc['name'].value.substring(0,1));
+                on_script_error: continue
+              first_char_strict_error:
+                type: keyword
+                script: |
+                  emit(doc['name'].value.substring(0,1));
+                on_script_error: fail
+            properties:
+              name:
+                type: keyword
+
+  - do:
+      bulk:
+        index: testindex
+        refresh: true
+        body: |
+          {"index":{}}
+          {"name": "foo"}
+          {"index":{}}
+          {"name": ""}
+
+---
+"Query rtf with on_script_error continue":
+  - do:
+      search:
+        index: testindex
+        body:
+          query:
+            match:
+              first_char: "f"
+          fields: [ name, first_char ]
+  - match: { hits.total.value: 1 }
+  - match: { hits.hits.0.fields.name: [ foo ] }
+  - match: { hits.hits.0.fields.first_char: [ f ] }
+
+---
+"Query rtf with on_script_error fail":
+  - do:
+      catch: /type=script_exception, reason=runtime error/
+      search:
+        index: testindex
+        body:
+          query:
+            match:
+              first_char_strict_error: "f"
+          fields: [ name, first_char_strict_error ]
+
+---
+"Aggregate on rtf with on_script_error continue":
+  - do:
+      search:
+        index: testindex
+        body:
+          aggs:
+            firstchar:
+              terms:
+                field: first_char
+  - length: { aggregations.firstchar.buckets: 1 }
+  - match: { aggregations.firstchar.buckets.0.key: "f" }
+
+---
+"Aggregate on rtf with on_script_error fail":
+  - do:
+      catch: /type=script_exception, reason=runtime error/
+      search:
+        index: testindex
+        body:
+          aggs:
+            firstchar:
+              terms:
+                field: first_char_strict_error
+
+---
+"Fields retrieval with ignoring error":
+  - do:
+      search:
+        index: testindex
+        body:
+          query: { match_all: { } }
+          fields: [ name, first_char ]
+  - match: { hits.total.value: 2 }
+  - match: { hits.hits.0.fields.name: [ foo ] }
+  - match: { hits.hits.0.fields.first_char: [ f ] }
+  - match: { hits.hits.1.fields.name: [ "" ] }
+  - is_false: hits.hits.1.fields.first_char
+
+---
+"Fields retrieval with failing on error":
+  - do:
+      catch: /type=script_exception, reason=runtime error/
+      search:
+        index: testindex
+        body:
+          query: { match_all: { } }
+          fields: [ name, first_char_strict_error ]
+
+---
+"Sorting with ignoring error":
+  - do:
+      search:
+        index: testindex
+        body:
+          query: { match_all: { } }
+          fields: [ name ]
+          sort: first_char
+  - match: { hits.total.value: 2 }
+  - match: { hits.hits.0.fields.name: [ foo ] }
+  - match: { hits.hits.1.fields.name: [ "" ] }
+
+---
+"Sorting with with failing on error":
+  - do:
+      catch: /type=script_exception, reason=runtime error/
+      search:
+        index: testindex
+        body:
+          query: { match_all: { } }
+          fields: [ name ]
+          sort: first_char_strict_error
+
+---
+"Query search time rtf with on_script_error continue":
+  - do:
+      search:
+        index: testindex
+        body:
+          query:
+            match:
+              first_char_search: "f"
+          fields: [ name, first_char_search ]
+          runtime_mappings:
+            first_char_search:
+              type: keyword
+              script: |
+                emit(doc['name'].value.substring(0,1));
+              on_script_error: continue
+
+  - match: { hits.total.value: 1 }
+  - match: { hits.hits.0.fields.name: [ foo ] }
+  - match: { hits.hits.0.fields.first_char_search: [ f ] }
+
+---
+"Query search time rtf with on_script_error fail":
+  - do:
+      catch: /type=script_exception, reason=runtime error/
+      search:
+        index: testindex
+        body:
+          query:
+            match:
+              first_char_search: "f"
+          fields: [ name, first_char_search ]
+          runtime_mappings:
+            first_char_search:
+              type: keyword
+              script: |
+                emit(doc['name'].value.substring(0,1));
+              on_script_error: fail
+
+---
+"Change error behaviour for lenient runtime field":
+
+  - do:
+      indices.put_mapping:
+        index: testindex
+        body:
+          runtime:
+            first_char_variant:
+              type: keyword
+              script: |
+                emit(doc['name'].value.substring(0,1));
+
+  - do:
+      catch: /type=script_exception, reason=runtime error/
+      search:
+        index: testindex
+        body:
+          query:
+            match:
+              first_char_variant: "f"
+
+  - do:
+      indices.put_mapping:
+        index: testindex
+        body:
+          runtime:
+            first_char_variant:
+              type: keyword
+              script: |
+                emit(doc['name'].value.substring(0,1));
+              on_script_error: continue
+
+  - do:
+      search:
+        index: testindex
+        body:
+          query:
+            match:
+              first_char_variant: "f"
+          fields: [ name, first_char_variant ]
+  - match: { hits.total.value: 1 }
+  - match: { hits.hits.0.fields.name: [ foo ] }
+  - match: { hits.hits.0.fields.first_char_variant: [ f ] }