[Observability] Load hasData call asynchronously

[cauemarcondes]

This PR intends to improve the performance of the Observability page.

The current architecture waits for all hasData requests to be completed to then verify to which page it should redirect the user.

The new architecture waits until at least one app returns true to the hasData request and immediately redirects the user to the correct page, while the remaining calls are being finished in background.
Before:

After:

Before:

After:

[elasticmachine]

Pinging @elastic/apm-ui (Team:apm)

[elasticmachine]

Pinging @elastic/uptime (Team:uptime)

[dgieselaar]

Why does the UX app have a different interface for its hasData handler? And looks like it will not be updated because it's not a dependency?

[cauemarcondes]

I think @shahzad31 is the right person to answer this question.

[dgieselaar]

Maybe @justinkambic knows?

[shahzad31]

@dgieselaar i believe it was a compromise between performance and consistency. For UX case we are displaying data for the service with most traffic, so in the same query where we are getting if it has rum data , it is also getting service name with most traffic via terms aggs.

I believe we need better types here to clear this to avoid this if conditions. Or maybe we can get service name via a different query or maybe make two es queries in data request. IMO consistency here isn't worth an additional query.

There was also discussion about this in original PR where this was added.

[dgieselaar]

Hmm, I would expect that to be unlikely. The fact that you're running a terms aggregation to get the service name means that you cannot terminate early, because all documents have to be collected and aggregated (within the given time range). That will make your hasData call significantly slower. I don't think it's worth it, in fact, it's probably worse in terms of performance. Instead of running things in parallel, and getting certain parts of the data when they're ready, it's now one long blocking request.

[shahzad31]

it might be a case that terminate early flag isn't being passed correctly to es request.

[dgieselaar]

dev-next doesn't have a lot of data. IIRC, half of our users have more data, and significantly less nodes. So, if it's slow on dev-next it's most likely an issue, but if it's fast, it doesn't really mean that it will be fast for our users. In this example, you'll likely have relatively high overhead of security checks etc, which happen for each request, so it's hard to tell how efficient the search is based on network timings.

I think it's good here to take a step back and analyze what's happening with this search:

{
  size: 0,
  query: {
    bool: {
      filter: [{ term: { [TRANSACTION_TYPE]: TRANSACTION_PAGE_LOAD } }],
    },
  },
  aggs: {
    services: {
      filter: {
        range: rangeFilter(start, end),
      },
      aggs: {
        mostTraffic: {
          terms: {
            field: SERVICE_NAME,
            size: 1,
          },
        },
      },
    },
  },
}

• First all page load transactions in all indices are collected, because there is no range query on @timestamp.
• Then, a filter aggregation runs to select only documents that match the given time range.
• Then, another aggregation runs on all the documents for the given time range. size only removes the bucket from the response - it will still be calculated.

All these steps run sequentially (per document).

Especially the first step could be pretty slow for users that have a lot of RUM data.

If you separate this however, you'd have two searches that you can run in parallel, plus, even the total duration will be faster.

• For the hasData search, you can set terminate_after to 1, meaning it will exit as soon as it has found a document. The performance cost of this search is negligible in most cases.
• For the getServiceName search, you can simply add a range query in your bool clause. This will remove the need for a filter aggregation.

[dgieselaar]

One example from DPP:

• one dual-purpose search: 2000ms
• hasData: 40ms
• getServiceName: 90ms

[dgieselaar]

Queries:

GET apm-*/_search?request_cache=false
{
  "size": 0,
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "transaction.type": "page-load"
          }
        }
      ]
    }
  },
  "aggs": {
    "services": {
      "filter": {
        "range": {
          "@timestamp": {
            "gte": "2020-05-12T10:00:00.000Z",
            "lt": "2020-05-12T11:00:00.000Z"
          }
        }
      },
      "aggs": {
        "mostTraffic": {
          "terms": {
            "field": "service.name",
            "size": 1
          }
        }
      }
    }
  }
}

GET apm-*/_search?request_cache=false
{
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "transaction.type": "page-load"
          }
        }
      ]
    }
  },
  "size": 0,
  "track_total_hits": true,
  "terminate_after": 1
}

GET apm-*/_search?request_cache=false
{
  "size": 0,
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "transaction.type": "page-load"
          }
        },
        {
          "range": {
            "@timestamp": {
              "gte": "2020-05-12T10:00:00.000Z",
              "lt": "2020-05-12T11:00:00.000Z"
            }
          }
        }
      ]
    }
  },
  "aggs": {
    "mostTraffic": {
      "terms": {
        "field": "service.name",
        "size": 1
      }
    }
  }
}

[shahzad31]

++

[dgieselaar]

We can remove these statements, and add a hidden prop on the EUIFlexItem component. That will ensure the component itself will start loading the data.

[cauemarcondes]

Are you sure the hidden property works? I haven't found any description of it here https://elastic.github.io/eui/#/layout/flex, and I tried it and it always shows the child element.

[dgieselaar]

It's a native HTML attribute. It should work, but if it doesn't, there might be some styles from EUI overriding it, or React special-casing it. We don't have to use hidden, but the point is to render the component so it starts the request, but hide it from the UI.

[justinkambic]

Functionally, looks good. I agree with some of the previous comments that we can probably improve code clarity a bit.

[cauemarcondes]

@elasticmachine merge upstream

[cauemarcondes]

Looks like there are two issues still outstanding, namely that the hasData interface for the UX app has a different interface,

@dgieselaar @shahzad31 since I'm going to be on PTO for the next weeks, I'd rather doing this change in another PR. I also want to refactor the fetchData, today it's called inside each component, I want to change it to also use the new Context created.

and the components still wait until hasData has returned for at least one app

I'm going to fix it.

[cauemarcondes]

retest

[cauemarcondes]

@elasticmachine merge upstream

[sorenlouv]

Why the special handling for alert and ux? I thought we had the same interface for all apps

[cauemarcondes]

For ux there's a tread where @dgieselaar and @shahzad31 discussed it, #80644 (comment). To remove it is necessary some refactor on the UX side, which would be better if we could fo it in another PR.

[cauemarcondes]

For alert, I can solve it if I manually register the alert service with registerDataHandler. WDYT?

[sorenlouv]

We should get rid of the special logic for alert and ux here if possible

[cauemarcondes]

We can remove the alert logic based on my comment here. Ux will demand more changes with I'm not familiar to, maybe we could leave it like this for now and improve it in another PR.

[sorenlouv]

Is it the right place to fetch alerts in the HasDataContextProvider?

[cauemarcondes]

I thought would be nice to centralize all requests to fill the page in a single place. Alert would have behaved in the same way as the other apps if I registered it. Maybe that is the change I should do.

[sorenlouv]

overall lgtm. Just a few comments

[cauemarcondes]

@dgieselaar @sqren @shahzad31 I don't want to spend more time on this PR, it has already consumed me a lot. I have improved the way hasData is fetched, now it doesn't wait until all APPs return to navigate, it waits until the first app that returns true. That has already improved the loading time.

I agree we must fix the UX hasData, to use the same signature like the others, but it can be done on another PR since this is already big.

[sorenlouv]

lgtm

[weltenwort]

Logs UI changes LGTM

and nice job at improving the responsiveness of the overview 👏

[cauemarcondes]

@elasticmachine merge upstream

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 080edfe

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
observability	108	111	+3

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
observability	160.7KB	169.1KB	+8.4KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
observability	72.1KB	68.6KB	-3.5KB

History

• 💚 Build #89242 succeeded 6d8a567
• 💔 Build #89233 failed bf819f8
• 💔 Build #89225 failed 4f0332d
• 💔 Build #89203 failed c7a12d6
• 💚 Build #86225 succeeded 88fd2e4

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream