Ingest support for Scripting Fields API #79155
Labels
:Core/Infra/Scripting
Scripting abstractions, Painless, and Mustache
>enhancement
Team:Core/Infra
Meta label for core/infra team
Comments
stu-elastic
added
>enhancement
:Core/Infra/Scripting
|
Pinging @elastic/es-core-infra (Team:Core/Infra) |
stu-elastic
added a commit
that referenced
this issue
Sep 1, 2022
Adds `WriteField` to the ingest context via `field(<path>)`. `WriteField` implements APIs: * `String` `getName()`: The path * `boolean` `exists()`: Does the path exist in the document? * `WriteField` `set(def)`: Set the value at the path, creates nested path elements if necessary * `WriteField` `append(def)`: Appends value to the path, creates nested path elements if necessary, the value at path is always a List after this call? * `boolean` `isEmpty()`: Does the path contain a value? * `int` `size()`: How many elements does the path contain? * `Iterator` `iterator()`: Iterate over all elements at the path. * `def` `get(def`): Get the value at the path if it exists, otherwise return the given default * `def` `get(int, def)`: Get the value at the path and index if it exists, otherwise return the given default * `boolean` `hasValue(Predicate)`: Is there a value at the path that passes the filter? * `WriteField` `transform(Function)`: Change all values at the path * `WriteField` `deduplicate()`: Remove duplicates from the path * `WriteField` `removeValuesIf(Predicate)`: Remove a values from the path that pass the filter * `WriteField` `removeValue(int)`: Remove the index from the path, if it exists Some APIs remain unimplemented: * `void` `move(String)` * `void` `overwrite(String)` * `void` `remove()` This change does not handle equivalent paths, which are paths that differ in the source but flatten to the same field in the indexed document. Path resolution is basic, each path element is assumed to be a key in the current container, starting with the root. If there is not an entry in the map at a given level, the algorithm checks to see if the remaining path exists as a flat key. The nested then flat nature algorithm handles the common case of some or none nesting followed by flat keys. Refs: #79155
stu-elastic
added a commit
that referenced
this issue
Sep 8, 2022
Adds `WriteField` path manipulation APIs: * `WriteField` `move(String)`: Moves this path to the destination path. Throws an error if destination path exists. * `WriteField` `overwrite(String)`: Same as `move(String)` but overwrites the destination path if it exists; * `void` `remove()`: Remove the leaf value from this path. See also: #89738 Refs: #79155
stu-elastic
added a commit
that referenced
this issue
Sep 19, 2022
Adds support for nested documents to the write field API. A nested document is either a key with a map value or a key with a list of maps as the value. The `WriteField` API has new APIs * `NestedDocument doc()`: Append a new `NestedDocument` to this field. * `NestedDocument doc(int)`: Get the `NestedDocument` at the given index. Makes empty `NestedDocuments` between end of current index and the given index, if necessary. * `Iterable docs()`: Iterate over all `NestedDocument`s in this field. and updated APIs * `WriteField move(def)`: Takes a `String` or `Field` destination. Moves the field value to the destination. The value is moved relative to the current root (`NestedDocument` or source root) if a String, or the root of the `Field`. Throws if destination exists. * `WriteField overwrite(def)`: Same as `move(def)` but replaces destination if it exists. The `NestedDocument` has the API * `WriteField field(String)`: Get the field with this `NestedDocument` as the root. * `Stream fields(String)`: Unimplemented, iterate over all fields matching the given field glob. * `boolean isEmpty()`: Does this `NestedDocument` have any fields? * `int size()`: How many fields does this `NestedDocument` have? * `boolean exists()`: Has this `NestedDocument` been removed? * `void remove()`: Remove this `NestedDocument`. Refs: #79155
stu-elastic
added a commit
that referenced
this issue
Sep 19, 2022
Whitelists the write fields API to the reindex, update by query and update contexts. Refs: #79155
|
Start |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add ingest support for the fields api, #61388.
Important APIs for ingestion:
static Stream<Field> fields(fieldglob)Retrieve all fields within the document as a stream.
Field<T> interface
void remove()Remove this field from the document.
boolean removeIf(Predicate<Field> predicate)Removes this field if it matches the predicate.
void renameTo(String name)Rename this field to the specified name. Throws an exception if the specified name is already a field.
void overwriteTo(String name)Rename the field to the specified name. Overwrites an existing field of the same name if it exists.
void setValue(T value)Set the field to a single value.
void setValues(List<T> values)Set the field to a list of values.
void appendValue(T value)Adds a value to the field source. If the field is mapped, the type of the value is verified. If the field does not yet exist, it is created, including any intermediate fields.
void appendValues(List<T> value)List form of appendValue.
boolean removeValue(T value)Remove a value from this field if it exists, returns true if removed.
boolean removeValuesIf(Predicate<T> predicate)Removes the set of values from the field that match the predicate, returns true if any values have been removed
void removeDuplicateValues()Removes any duplicate values this field has.
void transformValues(Function<T, T> function)Replace each existing value with the return of the specified function.
The text was updated successfully, but these errors were encountered: