[Scripted Fields] No inline scripting languages are shown

[ycombinator]

When users attempt to create a scripted field in Kibana, they are supposed to be shown a list of inline scripting languages enabled in Elasticsearch. This list used to show languages such as "painless" and "expression". Now (with the latest Elasticsearch master build) this list is empty:

The list is populated by Kibana by calling the http://localhost:9200/_cluster/settings?include_defaults=true&filter_path=**.script.engine.*.inline Elasticsearch API at this point in the code.

This API call used to return a response like this:

 {
  "defaults": {
    "script": {
      "engine": {
        "mustache": {
          "inline": "true"
        },
        "painless": {
          "inline": "true"
        },
        "expression": {
          "inline": "true"
        }
      }
    }
  }
}

The same API call now returns:

{
}

[ycombinator]

I've traced the cause of this Kibana bug to this change in Elasticsearch: elastic/elasticsearch@1196dfb. Given this change, Kibana will now need to figure out the list of enabled inline scripting languages some other way.

@jdconrad do you know if there's another Elasticsearch API Kibana could call now to generate this list for the user?

[jdconrad]

By default all scripting languages will be inline enabled now since we no longer have languages without a sandbox. However, users can both disable script types and script contexts. The settings for those are script.allowed_types and script.allowed_contexts. Sorry for the inconvenience here. These settings changes are important for work @rjernst and I are doing to improve scripting performance and user experience.

[ycombinator]

@jdconrad helped me understand script types and contexts (thanks Jack!). Based on my understanding, Kibana code that shows the list of scripting languages needs to do the following now:

1. Check that the inline scripting type is enabled in Elasticsearch (as Kibana specifies scripts inline in its requests to Elasticsearch). This is the case by default but users may disable it via the script.allowed_types cluster-level setting.

2. Check that the search and aggs scripting contexts are enabled in Elasticsearch (as Kibana allows scripted fields in these two contexts). This is the case by default but users may disable them via the script.allowed_contexts cluster-level setting.

3. If the previous two checks pass, show the list of inline scripting languages. For robustness, I think it would be nice if there was some way to retrieve this list from Elasticsearch (as opposed to hardcoding it in Kibana). That way, if a custom scripting engine was registered by a plugin, users could benefit from it. What do you think @jdconrad?

[jdconrad]

On issue 3, i I was thinking the same thing as soon as I realized Kibana was depending on the settings for what languages existed. I will have to think about the best way to do this.

[spalger]

@ycombinator @jdconrad any news here?

[jdconrad]

Apologies, I've been heavily bogged down by script contexts lately. We should probably add something like /_scripts that will return all the script engines and what contexts they support and possibly what settings are enabled.

[rjernst]

Note that in 6.0, there is no more "disabling" script languages. So whatever script engines we have are available. I think this could be hardcoded in kibana at first (painless and expression). I don't see exposing which script engines are installed as a priority.

[marius-dr]

This is also a blocker in 5.5.0 now.

[epixa]

@marius-dr The Elasticsearch API changed in 5.5?

[epixa]

I've confirmed with @marius-dr that this is not a problem in 5.5 after all.

[chrisronline]

I talked about this with @ycombinator and we agreed that this is not a blocker for 6.0.0. The solution @spalger added is sufficient for 6.0.0 and we'll readdress this once/if there is a better way.

[flash1293]

@spalger @rjernst Sorry to dig this one up again but the workaround with the hardcoded language array in the API is still in place. Is there a way to fetch supported languages by now (didn't see anything in the docs)? If not we should at least implement @ycombinator suggestion 1 and 2 from #11959 (comment) , right? This comes up because we are in the process of moving those parts to the Kibana platform. I'm more than happy to ditch the check completely, but it seems like it makes a lot of sense and the absence is a bug in the current product.

[stu-elastic]

@flash1293 We've added the API with 426c7a5 in 7.6 and 4a07802 in 8.0. Let's chat about how to use this when you get around to it.

[elasticmachine]

Pinging @elastic/kibana-app-arch (Team:AppArch)

[mattkime]

I'm closing this as Scripted Fields are deprecated in favor of runtime fields.