[BUG] Nested aggregation result is discarded from serialized SearchResponse

[chanon-onman]

What is the bug?

When using SearchResponse.toJsonString function to serialized nested aggregation result. The nested aggregation result will be discarded.

How can one reproduce the bug?

1. Create sample document

PUT my-index/_doc/sample_1
{
  "l1_term": "l1_value",
  "l2_term": "l2_value",
  "count": 1
}

2. Test aggregation query on OpenSearch dashboard (Kibana)

GET my-index/_search
{
  "size": 0, 
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "l1_term": "l1_value"
          }
        }
      ]
    }
  },
  "aggs": {
    "grp_by_l1": {
      "terms": {
        "field": "l1_term.keyword"
      },
      "aggs": {
        "l1_result": {
          "filter": {
            "term": {
              "l2_term.keyword": "l2_value"
            }
          },
          "aggs": {
            "l2_result": {
              "sum": {
                "field": "count"
              }
            }
          }
        }
      }
    }
  }
}

And got this result from OpenSearch dashboard, it has l2_result value in aggregation result.

{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 2,
    "successful": 2,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "grp_by_l1": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [
        {
          "key": "l1_value",
          "doc_count": 1,
          "l1_result": {
            "doc_count": 1,
            "l2_result": {
              "value": 1
            }
          }
        }
      ]
    }
  }
}

3. But when execute the same aggregation query and serialize result via SearchResponse class the l2_result is missing.

String nestedAggQueryJsonStr = "{\n" +
        "  \"size\": 0,\n" +
        "  \"query\": {\n" +
        "    \"bool\": {\n" +
        "      \"must\": [\n" +
        "        {\n" +
        "          \"term\": {\n" +
        "            \"l1_term\": \"l1_value\"\n" +
        "          }\n" +
        "        }\n" +
        "      ]\n" +
        "    }\n" +
        "  },\n" +
        "  \"aggs\": {\n" +
        "    \"grp_by_l1\": {\n" +
        "      \"terms\": {\n" +
        "        \"field\": \"l1_term.keyword\"\n" +
        "      },\n" +
        "      \"aggs\": {\n" +
        "        \"l1_result\": {\n" +
        "          \"filter\": {\n" +
        "            \"term\": {\n" +
        "              \"l2_term.keyword\": \"l2_value\"\n" +
        "            }\n" +
        "          },\n" +
        "          \"aggs\": {\n" +
        "            \"l2_result\": {\n" +
        "              \"sum\": {\n" +
        "                \"field\": \"count\"\n" +
        "              }\n" +
        "            }\n" +
        "          }\n" +
        "        }\n" +
        "      }\n" +
        "    }\n" +
        "  }\n" +
        "}\n";
SearchRequest nestedAggReq = SearchRequest._DESERIALIZER.deserialize(new JacksonJsonpParser(new JsonFactory().createParser(nestedAggQueryJsonStr)), new JacksonJsonpMapper());
SearchResponse nestedAggResp = client.search(nestedAggReq, JsonData.class);
System.out.println(nestedAggResp.toJsonString());

Result from code above, note that l2_value is missing.

{
  "took": 5,
  "timed_out": false,
  "_shards": {
    "failed": 0,
    "successful": 6,
    "total": 6,
    "skipped": 0
  },
  "hits": {
    "total": {
      "relation": "eq",
      "value": 1
    },
    "hits": []
  },
  "aggregations": {
    "sterms#grp_by_l1": {
      "buckets": [
        {
          "filter#l1_result": {
            "doc_count": 1
          },
          "doc_count": 1,
          "key": "l1_value"
        }
      ],
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0
    }
  }
}

This value can be seen in IDE debugger

What is the expected behavior?

Nested aggregation should be included in serialized output.

What is your host/environment?

• opensearch-java 2.18.0

[dblock]

Looks like a bug, want to try to fix it @chanon-onman?

[chanon-onman]

@dblock It's quite hard for me. Can you please help fix this?

[Xtansia]

I've raised a PR to fix this #1350

[Xtansia]

@chanon-onman The fix for this was released as part of v2.19.0