Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a specialized PainlessExecuteIndexAction #78825

Closed
wants to merge 5 commits into from
Closed

Add a specialized PainlessExecuteIndexAction #78825

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
06bcd0e
Add a specialized PainlessExecuteIndexAction
jimczi Oct 7, 2021
4155039
leftover
jimczi Oct 7, 2021
577f564
fix docs
jimczi Oct 7, 2021
78af49f
fix test and docs
jimczi Oct 7, 2021
87bd14c
Merge branch 'master' into painless_execute_index_action
jimczi Oct 11, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 25 additions & 39 deletions docs/painless/painless-guide/painless-execute-script.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ experimental::[]
The Painless execute API runs a script and returns a result.

[[painless-execute-api-request]]
==== {api-request-title}
`POST /_scripts/painless/_execute`
`POST /{index}/_scripts/painless/_execute`

[[painless-execute-api-desc]]
==== {api-description-title}
Expand Down Expand Up @@ -39,7 +39,7 @@ context is specified.
[%collapsible%open]
====
`painless_test`::
The default context if no other context is specified. See
The default context if no other context is specified. See
<<painless-execute-test,test context>>.

`filter`::
Expand All @@ -61,12 +61,12 @@ NOTE: Result ordering in the field contexts is not guaranteed.
--

****
`boolean_field`::
`boolean_field`::
The context for {ref}/boolean.html[`boolean` fields]. The script returns a `true`
or `false` response. See
<<painless-runtime-boolean,boolean_field context>>.

`date_field`::
`date_field`::
The context for {ref}/date.html[`date` fields]. `emit` takes a `long` value and
the script returns a sorted list of dates. See
<<painless-runtime-datetime,date_time context>>.
Expand Down Expand Up @@ -110,20 +110,16 @@ which is the default if no value is provided for `context`.
`document`:: (Required, string)
Document that's temporarily indexed in-memory and accessible from the script.

`index`:: (Required, string)
Index containing a mapping that's compatible with the indexed document.
====

`params`:: (`Map`, read-only)
Specifies any named parameters that are passed into the script as variables.

`query`:: (Optional, object)
NOTE: This parameter only applies when `score` is specified as the script
NOTE: This parameter only applies when `score` is specified as the script
`context`.
+
Use this parameter to specify a query for computing a score. Besides deciding
whether or not the document matches, the
{ref}/query-filter-context.html#query-context[query clause] also calculates a
whether or not the document matches, the
{ref}/query-filter-context.html#query-context[query clause] also calculates a
relevance score in the `_score` metadata field.

[[painless-execute-test]]
Expand Down Expand Up @@ -186,7 +182,7 @@ PUT /my-index-000001

[source,console]
----
POST /_scripts/painless/_execute
POST /my-index-000001/_scripts/painless/_execute
{
"script": {
"source": "doc['field'].value.length() <= params.max_length",
Expand All @@ -196,7 +192,6 @@ POST /_scripts/painless/_execute
},
"context": "filter",
"context_setup": {
"index": "my-index-000001",
"document": {
"field": "four"
}
Expand Down Expand Up @@ -240,7 +235,7 @@ PUT /my-index-000001

[source,console]
----
POST /_scripts/painless/_execute
POST /my-index-000001/_scripts/painless/_execute
{
"script": {
"source": "doc['rank'].value / params.max_rank",
Expand All @@ -250,7 +245,6 @@ POST /_scripts/painless/_execute
},
"context": "score",
"context_setup": {
"index": "my-index-000001",
"document": {
"rank": 4
}
Expand Down Expand Up @@ -282,7 +276,7 @@ Choose a field context based on the data type you want to return.
Use the `boolean_field` field context when you want to return a `true`
or `false` value from a script valuation. {ref}/boolean.html[Boolean fields]
accept `true` and `false` values, but can also accept strings that are
interpreted as either true or false.
interpreted as either true or false.

Let's say you have data for the top 100 science fiction books of all time. You
want to write scripts that return a boolean response such as whether books
Expand Down Expand Up @@ -318,7 +312,7 @@ whether a book was published before the year 1972:

[source,console]
----
POST /_scripts/painless/_execute
POST /my-index-000001/_scripts/painless/_execute
{
"script": {
"source": """
Expand All @@ -327,7 +321,6 @@ POST /_scripts/painless/_execute
},
"context": "boolean_field",
"context_setup": {
"index": "my-index-000001",
"document": {
"name": "Dune",
"author": "Frank Herbert",
Expand Down Expand Up @@ -357,7 +350,7 @@ least one character, but is less than five characters:

[source,console]
----
POST /_scripts/painless/_execute
POST /my-index-000001/_scripts/painless/_execute
{
"script": {
"source": """
Expand All @@ -367,7 +360,6 @@ POST /_scripts/painless/_execute
},
"context": "boolean_field",
"context_setup": {
"index": "my-index-000001",
"document": {
"name": "Dune",
"author": "Frank Herbert",
Expand Down Expand Up @@ -439,7 +431,7 @@ calculation of `pageTime` times the `page_count` to determine approximately

[source,console]
----
POST /_scripts/painless/_execute
POST /my-index-000001/_scripts/painless/_execute
{
"script": {
"source": """
Expand All @@ -455,7 +447,6 @@ POST /_scripts/painless/_execute
},
"context": "date_field",
"context_setup": {
"index": "my-index-000001",
"document": {
"name": "Revelation Space",
"author": "Alastair Reynolds",
Expand Down Expand Up @@ -496,7 +487,7 @@ runtime field.
You need to multiply this value, but only for
sensors that match a specific model number.

Add the following fields to your index mapping. The `voltage` field is a
Add the following fields to your index mapping. The `voltage` field is a
sub-field of the `measures` object.

[source,console]
Expand Down Expand Up @@ -530,7 +521,7 @@ that match the specified criteria.

[source,console]
----
POST /_scripts/painless/_execute
POST /my-index-000001/_scripts/painless/_execute
{
"script": {
"source": """
Expand All @@ -541,7 +532,6 @@ POST /_scripts/painless/_execute
},
"context": "double_field",
"context_setup": {
"index": "my-index-000001",
"document": {
"@timestamp": 1516470094000,
"model_number": "QVKC92Q",
Expand Down Expand Up @@ -573,7 +563,7 @@ define a geo-point field in several ways, and include values for latitude and
longitude in the document for your script.

If you already have a known geo-point, it's simpler to clearly state the
positions of `lat` and `lon` in your index mappings.
positions of `lat` and `lon` in your index mappings.

[source,console]
----
Expand All @@ -597,7 +587,7 @@ that retrieves the `lat` and `long` values.

[source,console]
----
POST /_scripts/painless/_execute
POST /my-index-000001/_scripts/painless/_execute
{
"script": {
"source": """
Expand All @@ -606,7 +596,6 @@ POST /_scripts/painless/_execute
},
"context": "geo_point_field",
"context_setup": {
"index": "my-index-000001",
"document": {
"lat": 41.12,
"lon": -71.34
Expand All @@ -617,7 +606,7 @@ POST /_scripts/painless/_execute
// TEST[continued]

Because this you're working with a geo-point field type, the response includes
results that are formatted as `coordinates`.
results that are formatted as `coordinates`.

[source,console-result]
----
Expand All @@ -638,7 +627,7 @@ results that are formatted as `coordinates`.
===== `ip_field`
The `ip_field` context is useful for data that includes IP addresses of type
{ref}/ip.html[`ip`]. For example, let's say you have a `message` field from an Apache
log. This field contains an IP address, but also other data that you don't need.
log. This field contains an IP address, but also other data that you don't need.

You can add the `message` field to your index mappings as a `wildcard` to accept
pretty much any data you want to put in that field.
Expand Down Expand Up @@ -667,17 +656,16 @@ value matching the IP address. If the pattern doesn’t match

[source,console]
----
POST /_scripts/painless/_execute
POST /my-index-000001/_scripts/painless/_execute
{
"script": {
"source": """
String clientip=grok('%{COMMONAPACHELOG}').extract(doc["message"].value)?.clientip;
if (clientip != null) emit(clientip);
if (clientip != null) emit(clientip);
"""
},
"context": "ip_field",
"context_setup": {
"index": "my-index-000001",
"document": {
"message": "40.135.0.0 - - [30/Apr/2020:14:30:17 -0500] \"GET /images/hm_bg.jpg HTTP/1.0\" 200 24736"
}
Expand Down Expand Up @@ -726,7 +714,7 @@ script in the `keyword_field` runtime field context:

[source,console]
----
POST /_scripts/painless/_execute
POST /my-index-000001/_scripts/painless/_execute
{
"script": {
"source": """
Expand All @@ -735,7 +723,6 @@ POST /_scripts/painless/_execute
},
"context": "keyword_field",
"context_setup": {
"index": "my-index-000001",
"document": {
"@timestamp": "2020-04-30T14:31:43-05:00"
}
Expand Down Expand Up @@ -777,7 +764,7 @@ PUT /my-index-000001/
"type": "long"
},
"end": {
"type": "long"
"type": "long"
}
}
}
Expand All @@ -792,7 +779,7 @@ field from the `measures` object and subtracts it from the `start` field:

[source,console]
----
POST /_scripts/painless/_execute
POST /my-index-000001/_scripts/painless/_execute
{
"script": {
"source": """
Expand All @@ -801,7 +788,6 @@ POST /_scripts/painless/_execute
},
"context": "long_field",
"context_setup": {
"index": "my-index-000001",
"document": {
"measures": {
"voltage": "4.0",
Expand All @@ -823,4 +809,4 @@ The response includes the calculated value from the script valuation:
8624909
]
}
----
----
13 changes: 6 additions & 7 deletions docs/reference/scripting/dissect-syntax.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ it will return those results to you.
[[dissect-syntax]]
==== Dissect patterns
Dissect patterns are comprised of _variables_ and _separators_. Anything
defined by a percent sign and curly braces `%{}` is considered a variable,
such as `%{clientip}`. You can assign variables to any part of data in a field,
defined by a percent sign and curly braces `%{}` is considered a variable,
such as `%{clientip}`. You can assign variables to any part of data in a field,
and then return only the parts that you want. Separators are any values between
variables, which could be spaces, dashes, or other delimiters.

Expand Down Expand Up @@ -52,7 +52,7 @@ this:
Using that same logic, you can create variables for the remaining chunks of
data. Double quotation marks are separators, so include those in your dissect
pattern. The pattern replaces `GET` with a `%{verb}` variable, but keeps `HTTP`
as part of the pattern.
as part of the pattern.

[source,js]
----
Expand All @@ -62,7 +62,7 @@ as part of the pattern.
----
// NOTCONSOLE

Combining the two patterns results in a dissect pattern that looks like this:
Combining the two patterns results in a dissect pattern that looks like this:

[source,js]
----
Expand Down Expand Up @@ -113,7 +113,7 @@ In this example, `message` is the `<field_name>` and `response` is the

[source,console]
----
POST /_scripts/painless/_execute
POST /my_index/_scripts/painless/_execute
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we not want to use my-index-000001 as in the other examples?

{
"script": {
"source": """
Expand All @@ -123,7 +123,6 @@ POST /_scripts/painless/_execute
},
"context": "long_field", <2>
"context_setup": {
"index": "my-index",
"document": { <3>
"message": """247.37.0.0 - - [30/Apr/2020:14:31:22 -0500] "GET /images/hm_nbg.jpg HTTP/1.0" 304 0"""
}
Expand Down Expand Up @@ -152,7 +151,7 @@ If you have a functional dissect pattern, you can add it to a runtime field to
manipulate data. Because runtime fields don't require you to index fields, you
have incredible flexibility to modify your script and how it functions. If you
already <<dissect-patterns-test,tested your dissect pattern>> using the Painless
execute API, you can use that _exact_ Painless script in your runtime field.
execute API, you can use that _exact_ Painless script in your runtime field.

To start, add the `message` field as a `wildcard` type like in the previous
section, but also add `@timestamp` as a `date` in case you want to operate on
Expand Down
8 changes: 6 additions & 2 deletions modules/lang-painless/src/main/java/org/elasticsearch/painless/PainlessPlugin.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,10 @@
import org.elasticsearch.common.xcontent.NamedXContentRegistry;
import org.elasticsearch.env.Environment;
import org.elasticsearch.env.NodeEnvironment;
import org.elasticsearch.painless.action.AbstractPainlessExecuteAction;
import org.elasticsearch.painless.action.PainlessContextAction;
import org.elasticsearch.painless.action.PainlessExecuteAction;
import org.elasticsearch.painless.action.PainlessExecuteIndexAction;
import org.elasticsearch.painless.spi.PainlessExtension;
import org.elasticsearch.painless.spi.Whitelist;
import org.elasticsearch.painless.spi.WhitelistLoader;
Expand Down Expand Up @@ -108,7 +110,7 @@ public final class PainlessPlugin extends Plugin implements ScriptPlugin, Extens
}
}
testWhitelists.add(WhitelistLoader.loadFromResourceFiles(PainlessPlugin.class, "org.elasticsearch.json.txt"));
whitelists.put(PainlessExecuteAction.PainlessTestScript.CONTEXT, testWhitelists);
whitelists.put(AbstractPainlessExecuteAction.PainlessTestScript.CONTEXT, testWhitelists);
}

private final SetOnce<PainlessScriptEngine> painlessScriptEngine = new SetOnce<>();
Expand Down Expand Up @@ -159,13 +161,14 @@ public void loadExtensions(ExtensionLoader loader) {

@Override
public List<ScriptContext<?>> getContexts() {
return Collections.singletonList(PainlessExecuteAction.PainlessTestScript.CONTEXT);
return Collections.singletonList(AbstractPainlessExecuteAction.PainlessTestScript.CONTEXT);
}

@Override
public List<ActionHandler<? extends ActionRequest, ? extends ActionResponse>> getActions() {
List<ActionHandler<? extends ActionRequest, ? extends ActionResponse>> actions = new ArrayList<>();
actions.add(new ActionHandler<>(PainlessExecuteAction.INSTANCE, PainlessExecuteAction.TransportAction.class));
actions.add(new ActionHandler<>(PainlessExecuteIndexAction.INSTANCE, PainlessExecuteIndexAction.TransportAction.class));
actions.add(new ActionHandler<>(PainlessContextAction.INSTANCE, PainlessContextAction.TransportAction.class));
return actions;
}
Expand All @@ -177,6 +180,7 @@ public List<RestHandler> getRestHandlers(Settings settings, RestController restC
Supplier<DiscoveryNodes> nodesInCluster) {
List<RestHandler> handlers = new ArrayList<>();
handlers.add(new PainlessExecuteAction.RestAction());
handlers.add(new PainlessExecuteIndexAction.RestAction());
handlers.add(new PainlessContextAction.RestAction());
return handlers;
}
Expand Down
Loading