Runtime fields on nested fields

[ES-Learner]

Description

I have an index with a nested field 'roles':

"roles": {
    "type": "nested",
    "properties": {
        "name": {
            "type": "text",
            "fields": {
                "raw": {
                    "type": "keyword",
                    "analyzer": "keylower"
                }
            }
        },
        "responsibilities": {
            "properties": {
                "name": {
                    "type": "text",
                    "fields": {
                        "raw": {
                            "type": "keyword",
                            "analyzer": "keylower"
                        }
                    }
                }
            }
        }
    }
}

The values in these fields are arrays, for eg.:

"roles": [
        {
            "name": "Programmer",
            "responsibilities": [
                {
                    "name": "Software Development"
                },
                {
                    "name": "Software Maintenance"
                }
            ]
        },
        {
            "name": "Data Analyst",
            "responsibilities": [
                {
                    "name": "Data analysis"
                },
                {
                    "name": "Reporting"
                }
            ]
        }
    ]

I have to build Kibana visualizations on these fields separately. Since it is a nested field and kibana doesn't support it yet (?), I thought of creating runtime fields for each of these fields.

This is what I have tried so far:

PUT employee/_mappings
{
  "runtime": {
    "empRoles": {
      "type": "keyword",
      "script": """if (doc["roles.name.raw"].size()!=0 ) {
        String[] empRoles;
        for(int i=0; i < doc["roles.name.raw"].size(); i++) {
          empRoles[i] = doc["roles.name.raw"].value ;
          
        }
         emit(empRoles);}"""
    }
  }
}

Result:

"caused_by" : {
        "type" : "class_cast_exception",
        "reason" : "Cannot cast from [java.lang.String[]] to [java.lang.String]."
      }

2. I modified the script to emit value multiple times:

PUT employee/_mappings
{
  "runtime": {
    "empRoles": {
      "type": "keyword",
      "script": """if (doc["roles.name.raw"].size()!=0 ) {
       for(int i=0; i < doc["roles.name.raw"].size(); i++) {
          emit(doc["roles.name.raw"].value);
        }
         }"""
    }
  }
}

Now I am not getting any error, but neither am I getting the value in the field.

I performed the following search query:

GET /employee/_search?pretty
{
  "_source": false, 
  "query": {
          "match": {
            "emailId": "[email protected]"
          }    
  },
  "fields": [
    "empRoles","roles.name.raw"
  ]
}

I got the following response:

{
  "took" : 8,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 12.575342,
    "hits" : [
      {
        "_index" : "employee",
        "_type" : "_doc",
        "_id" : "0bb26551-dfeb-4fbd-9c37-96016894b843",
        "_score" : 12.575342,
        "fields" : {
          "roles" : [
            {
              "name.raw" : [
                "Programmer", "Data Analyst"
              ]
            }
          ]
        }
      }
    ]
  }
}

The runtime field was not populated.

3. I have tried using params._source to access the nested field:

PUT employee/_mappings
{
  "runtime": {
    "empRoles": {
      "type": "keyword",
      "script": """String[] empRoles;
        if (doc["roles.name.raw"].size()!=0 ) {
        int i=0;
        for(item in params._source.roles.name.raw) {
          emit(item) ;
        }
        
         }
         """
    }
  }
}

Here also, the runtime field is not getting populated.

PUT employee/_mappings
{
  "runtime": {
    "empRoles": {
      "type": "keyword",
      "script": """String[] empRoles;
        if (doc["roles.name.raw"].size()!=0 ) {
        int i=0;
        for(item in params._source.roles.name.raw) {
          empRoles[i++]=item
        }
        
         }
return empRoles;
         """
    }
  }
}

This results in error cannot cast from [java.lang.String[]] to [void], as expected.

Please add the feature to enable creating runtime fields based on nested fields.

[elasticsearchmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)

[kingherc]

Since it is a nested field and kibana doesn't support it yet (?), I thought of creating runtime fields for each of these fields.

For the Kibana feature for nested fields, you can also read this issue.

For the scripting question, I tagged the relevant team.