[KQL] Do not generate nested AST/Elasticsearch queries for same-level AND/OR clauses

[lukasolson]

Summary

Fixes #69649.
Fixes #89473.

This PR changes the KQL parser to no longer generate nested AST nodes for same-level and/or clauses. This also removes the recursive nature of the parsing, which eliminates the possibility of hitting the maximum call stack size limit.

For example, prior to this PR, a clause such as foo and bar and baz would generate the following AST (simplified for illustration purposes):

{
  "and": [
    {"is": [null, "foo"]},
    {
      "and": [
        {"is": [null, "bar"]},
        {"is": [null, "baz"]}
      ]
    }
  ]
}

Following this PR, it generates the following AST:

{
  "and": [
    {"is": [null, "foo"]},
    {"is": [null, "bar"]},
    {"is": [null, "baz"]}
  ]
}

The corresponding Elasticsearch queries generated from these ASTs have also been simplified, so that they don't use nested bool queries.

Checklist

Delete any items that are not applicable to this PR.

• Unit or functional tests were updated or added to match the most common scenarios

For maintainers

• This was checked for breaking API changes and was labeled appropriately

Release note

KQL will no longer generate nested Elasticsearch bool queries for same-level AND/OR clauses.

[elasticmachine]

Pinging @elastic/kibana-app-services (Team:AppServices)

[lukeelmers]

Changes to core tests LGTM

[elasticmachine]

Pinging @elastic/uptime (Team:uptime)

[tsullivan]

LGTM. Thanks for walking me through the code and how to read the Peg file!

[andrewvc]

LGTM from uptime

[kibanamachine]

💛 Build succeeded, but was flaky

• continuous-integration/kibana-ci/pull-request

• Commit: e4010b2

• Storybooks Preview

• Flaky suites:

  • xpack-securitySolutionCypressChrome

---

Test Failures

Kibana Pipeline / general / "before all" hook for "should contain the right query".Timeline query tab Query tab "before all" hook for "should contain the right query"

Link to Jenkins

Stack Trace

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

AssertionError: Timed out retrying after 60000ms: Expected to find element: `[data-test-subj="title-c424c520-7e11-11eb-bca4-8dccd5f64726"]`, but never found it.

Because this error occurred during a `before all` hook we are skipping the remaining tests in the current suite: `Timeline query tab`

Although you have test retries enabled, we do not retry tests when `before all` or `after all` hooks fail
    at Object.openTimelineById (http://localhost:6121/__cypress/tests?p=cypress/integration/timelines/query_tab.spec.ts:16059:15)
    at Context.eval (http://localhost:6121/__cypress/tests?p=cypress/integration/timelines/query_tab.spec.ts:15045:28)

---

Metrics [docs]

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
data	814.0KB	815.6KB	+1.7KB

History

• 💛 Build #111220 was flaky e4010b2
• 💔 Build #110933 failed 077107f
• 💔 Build #110892 failed c3d7b0a

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @lukasolson