Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filtering by non-trivial Painless scripted fields results in "compile error" #9024

Closed
tbragin opened this issue Nov 10, 2016 · 2 comments
Closed
Assignees
Labels
blocker bug Fixes for quality problems that affect the customer experience Feature:Scripted Fields Scripted fields features v5.1.1

Comments

@tbragin
Copy link
Contributor

tbragin commented Nov 10, 2016

ES/Kibana version:
5.0.0 running in Elastic Cloud

Browser version:
Chrome Version 54.0.2840.87 (64-bit) on MacOS El Capitan

Description of the problem including expected versus actual behavior:
Filtering by some scripted fields in Discover, Visualize, or Dashboard returns a compile error, while preview in Discover and aggregations in Visualize work.

Steps to reproduce:

  1. Create scripted field with one of the values below
  2. Filter on this scripted field
  3. Observe compile errors come back from Elasticsearch

Sample scripted fields that don't work

Note: This is with data from the getting started tutorial. https://www.elastic.co/guide/en/kibana/current/tutorial-load-dataset.html

LocalDateTime.ofInstant(Instant.ofEpochMilli(doc['@timestamp'].value), ZoneId.of('Z')).getDayOfWeek().getDisplayName(TextStyle.FULL, Locale.getDefault());

Note: Removing semicolon from the expression above gets rid of the filter problem.

if (doc['referer.keyword'].value =~ /error/) { 
return "error"
} else {
return "no error"
}
def m = /^.*\.([a-z]+)$/.matcher(doc['host.keyword'].value);
if ( m.matches() ) {
   return m.group(1)
} else {
   return "no match"
}

Errors in browser console (if relevant):
No browser console errors

screen shot 2016-11-09 at 8 37 54 pm

screen shot 2016-11-09 at 8 34 17 pm

screen shot 2016-11-09 at 8 33 26 pm

screen shot 2016-11-09 at 8 32 58 pm

@tbragin tbragin added :Management Feature:Scripted Fields Scripted fields features bug Fixes for quality problems that affect the customer experience labels Nov 10, 2016
Bargs added a commit to Bargs/kibana that referenced this issue Nov 15, 2016
Multi-statement scripts are currently incompatible with the way we
create filters on scripted fields. We're investigating the possibility
of enhancing elasticsearch to fix this issue, but we may have to
fallback on showing users a warning if we can't reach an agreement in
the near term.

See elastic#9024
Related elastic/elasticsearch#21479
epixa pushed a commit that referenced this issue Nov 17, 2016
Multi-statement scripts are currently incompatible with the way we
create filters on scripted fields. We're investigating the possibility
of enhancing elasticsearch to fix this issue, but we may have to
fallback on showing users a warning if we can't reach an agreement in
the near term.

See #9024
Related elastic/elasticsearch#21479
elastic-jasper added a commit that referenced this issue Nov 17, 2016
Backports PR #9090

**Commit 1:**
Add a note on multi-statement scripted field limitation

Multi-statement scripts are currently incompatible with the way we
create filters on scripted fields. We're investigating the possibility
of enhancing elasticsearch to fix this issue, but we may have to
fallback on showing users a warning if we can't reach an agreement in
the near term.

See #9024
Related elastic/elasticsearch#21479

* Original sha: b4aaed8
* Authored by Matthew Bargar <[email protected]> on 2016-11-15T22:47:26Z
elastic-jasper added a commit that referenced this issue Nov 17, 2016
Backports PR #9090

**Commit 1:**
Add a note on multi-statement scripted field limitation

Multi-statement scripts are currently incompatible with the way we
create filters on scripted fields. We're investigating the possibility
of enhancing elasticsearch to fix this issue, but we may have to
fallback on showing users a warning if we can't reach an agreement in
the near term.

See #9024
Related elastic/elasticsearch#21479

* Original sha: b4aaed8
* Authored by Matthew Bargar <[email protected]> on 2016-11-15T22:47:26Z
@epixa
Copy link
Contributor

epixa commented Nov 17, 2016

When this gets closed, we'll likely need also need to remove the note that we added in #9090

epixa pushed a commit that referenced this issue Nov 17, 2016
Backports PR #9090

**Commit 1:**
Add a note on multi-statement scripted field limitation

Multi-statement scripts are currently incompatible with the way we
create filters on scripted fields. We're investigating the possibility
of enhancing elasticsearch to fix this issue, but we may have to
fallback on showing users a warning if we can't reach an agreement in
the near term.

See #9024
Related elastic/elasticsearch#21479

* Original sha: b4aaed8
* Authored by Matthew Bargar <[email protected]> on 2016-11-15T22:47:26Z
epixa pushed a commit that referenced this issue Nov 17, 2016
Backports PR #9090

**Commit 1:**
Add a note on multi-statement scripted field limitation

Multi-statement scripts are currently incompatible with the way we
create filters on scripted fields. We're investigating the possibility
of enhancing elasticsearch to fix this issue, but we may have to
fallback on showing users a warning if we can't reach an agreement in
the near term.

See #9024
Related elastic/elasticsearch#21479

* Original sha: b4aaed8
* Authored by Matthew Bargar <[email protected]> on 2016-11-15T22:47:26Z
@tbragin
Copy link
Contributor Author

tbragin commented Nov 21, 2016

Now that this is merged in ES, we can address this specifically for Painless: elastic/elasticsearch#21635

@tbragin tbragin changed the title Filtering by some Painless scripted fields results in "compile error" Filtering by non-trivial Painless scripted fields results in "compile error" Nov 21, 2016
Bargs added a commit to Bargs/kibana that referenced this issue Nov 21, 2016
Painless scripted fields that consisted of more than a single expression
would break when the user attempted to filter on them in Discover or
Visualize because of the naive way we were building them. We now wrap
the user's script in a lambda so that it can be as complex at they need
it to be. The only special exception is that the user cannot define
named functions since those cannot go inside a Painless lambda.

Fixes elastic#9024
Related elastic/elasticsearch#21635
@epixa epixa added the blocker label Nov 22, 2016
epixa pushed a commit that referenced this issue Nov 29, 2016
* Fix filters for complex painless scripted fields

Painless scripted fields that consisted of more than a single expression
would break when the user attempted to filter on them in Discover or
Visualize because of the naive way we were building them. We now wrap
the user's script in a lambda so that it can be as complex at they need
it to be. The only special exception is that the user cannot define
named functions since those cannot go inside a Painless lambda.

Fixes #9024
Related elastic/elasticsearch#21635

* DRY it up

* Phrase tests

* Only include necessary comparators and add tests
elastic-jasper added a commit that referenced this issue Nov 29, 2016
Backports PR #9171

**Commit 1:**
Fix filters for complex painless scripted fields

Painless scripted fields that consisted of more than a single expression
would break when the user attempted to filter on them in Discover or
Visualize because of the naive way we were building them. We now wrap
the user's script in a lambda so that it can be as complex at they need
it to be. The only special exception is that the user cannot define
named functions since those cannot go inside a Painless lambda.

Fixes #9024
Related elastic/elasticsearch#21635

* Original sha: b2a86bb
* Authored by Matthew Bargar <[email protected]> on 2016-11-21T23:18:20Z

**Commit 2:**
DRY it up

* Original sha: 927de50
* Authored by Matthew Bargar <[email protected]> on 2016-11-22T16:21:21Z

**Commit 3:**
Phrase tests

* Original sha: 79e69bd
* Authored by Matthew Bargar <[email protected]> on 2016-11-22T16:48:12Z

**Commit 4:**
Only include necessary comparators and add tests

* Original sha: 5b9137b
* Authored by Matthew Bargar <[email protected]> on 2016-11-22T17:38:59Z
elastic-jasper added a commit that referenced this issue Nov 29, 2016
Backports PR #9171

**Commit 1:**
Fix filters for complex painless scripted fields

Painless scripted fields that consisted of more than a single expression
would break when the user attempted to filter on them in Discover or
Visualize because of the naive way we were building them. We now wrap
the user's script in a lambda so that it can be as complex at they need
it to be. The only special exception is that the user cannot define
named functions since those cannot go inside a Painless lambda.

Fixes #9024
Related elastic/elasticsearch#21635

* Original sha: b2a86bb
* Authored by Matthew Bargar <[email protected]> on 2016-11-21T23:18:20Z

**Commit 2:**
DRY it up

* Original sha: 927de50
* Authored by Matthew Bargar <[email protected]> on 2016-11-22T16:21:21Z

**Commit 3:**
Phrase tests

* Original sha: 79e69bd
* Authored by Matthew Bargar <[email protected]> on 2016-11-22T16:48:12Z

**Commit 4:**
Only include necessary comparators and add tests

* Original sha: 5b9137b
* Authored by Matthew Bargar <[email protected]> on 2016-11-22T17:38:59Z
epixa pushed a commit that referenced this issue Nov 29, 2016
Backports PR #9171

**Commit 1:**
Fix filters for complex painless scripted fields

Painless scripted fields that consisted of more than a single expression
would break when the user attempted to filter on them in Discover or
Visualize because of the naive way we were building them. We now wrap
the user's script in a lambda so that it can be as complex at they need
it to be. The only special exception is that the user cannot define
named functions since those cannot go inside a Painless lambda.

Fixes #9024
Related elastic/elasticsearch#21635

* Original sha: b2a86bb
* Authored by Matthew Bargar <[email protected]> on 2016-11-21T23:18:20Z

**Commit 2:**
DRY it up

* Original sha: 927de50
* Authored by Matthew Bargar <[email protected]> on 2016-11-22T16:21:21Z

**Commit 3:**
Phrase tests

* Original sha: 79e69bd
* Authored by Matthew Bargar <[email protected]> on 2016-11-22T16:48:12Z

**Commit 4:**
Only include necessary comparators and add tests

* Original sha: 5b9137b
* Authored by Matthew Bargar <[email protected]> on 2016-11-22T17:38:59Z
epixa pushed a commit that referenced this issue Nov 29, 2016
Backports PR #9171

**Commit 1:**
Fix filters for complex painless scripted fields

Painless scripted fields that consisted of more than a single expression
would break when the user attempted to filter on them in Discover or
Visualize because of the naive way we were building them. We now wrap
the user's script in a lambda so that it can be as complex at they need
it to be. The only special exception is that the user cannot define
named functions since those cannot go inside a Painless lambda.

Fixes #9024
Related elastic/elasticsearch#21635

* Original sha: b2a86bb
* Authored by Matthew Bargar <[email protected]> on 2016-11-21T23:18:20Z

**Commit 2:**
DRY it up

* Original sha: 927de50
* Authored by Matthew Bargar <[email protected]> on 2016-11-22T16:21:21Z

**Commit 3:**
Phrase tests

* Original sha: 79e69bd
* Authored by Matthew Bargar <[email protected]> on 2016-11-22T16:48:12Z

**Commit 4:**
Only include necessary comparators and add tests

* Original sha: 5b9137b
* Authored by Matthew Bargar <[email protected]> on 2016-11-22T17:38:59Z
@tbragin tbragin added v5.1.1 and removed v5.1.0 labels Dec 8, 2016
airow pushed a commit to airow/kibana that referenced this issue Feb 16, 2017
Backports PR elastic#9090

**Commit 1:**
Add a note on multi-statement scripted field limitation

Multi-statement scripts are currently incompatible with the way we
create filters on scripted fields. We're investigating the possibility
of enhancing elasticsearch to fix this issue, but we may have to
fallback on showing users a warning if we can't reach an agreement in
the near term.

See elastic#9024
Related elastic/elasticsearch#21479

* Original sha: b4aaed8
* Authored by Matthew Bargar <[email protected]> on 2016-11-15T22:47:26Z

Former-commit-id: 24840a9
airow pushed a commit to airow/kibana that referenced this issue Feb 16, 2017
Backports PR elastic#9171

**Commit 1:**
Fix filters for complex painless scripted fields

Painless scripted fields that consisted of more than a single expression
would break when the user attempted to filter on them in Discover or
Visualize because of the naive way we were building them. We now wrap
the user's script in a lambda so that it can be as complex at they need
it to be. The only special exception is that the user cannot define
named functions since those cannot go inside a Painless lambda.

Fixes elastic#9024
Related elastic/elasticsearch#21635

* Original sha: b2a86bb
* Authored by Matthew Bargar <[email protected]> on 2016-11-21T23:18:20Z

**Commit 2:**
DRY it up

* Original sha: 927de50
* Authored by Matthew Bargar <[email protected]> on 2016-11-22T16:21:21Z

**Commit 3:**
Phrase tests

* Original sha: 79e69bd
* Authored by Matthew Bargar <[email protected]> on 2016-11-22T16:48:12Z

**Commit 4:**
Only include necessary comparators and add tests

* Original sha: 5b9137b
* Authored by Matthew Bargar <[email protected]> on 2016-11-22T17:38:59Z

Former-commit-id: 313794b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocker bug Fixes for quality problems that affect the customer experience Feature:Scripted Fields Scripted fields features v5.1.1
Projects
None yet
Development

No branches or pull requests

3 participants