"Create Index Pattern" page hang when there are too many indices/shards

[mattkime]

Kibana version:
6.4.3 - 7.4

Describe the bug:

User has 3 node cluster with 3500 indicies and 11k shards. Clicking 'Create Pattern' in the index pattern management page hangs for over a minute while all the index data is queried.

The query -

GET _search
{"size":0,"aggs":{"":{"terms":{"field":"_index","size":200}}}}

Expected behavior:
Responsive user interface

---

The current user interfaces ensures the user provides an index pattern that matches an index. Its acting as a gatekeeper and if the gatekeeper isn't working nothing passes. Perhaps we could modify the user interface so that its more passive and allows the user to enter and create index patterns independent of the index pattern list in the browser. I think we'd still need to verify that an index exists but we could do that after the user clicks in 'Next Step'. This would function but be slow, likely an acceptable compromise.

[elasticmachine]

Pinging @elastic/kibana-app-arch

[mattkime]

Potential to optimize the query, quoting @jasontedor

It looks like the purpose of this search is to find the names of the indices that contain at least one document. Why not hit the indices stats endpoint /_stats/docs and filter on those indices where primaries.docs.count is greater than zero?

[justdaven]

This issue appears for us as well. We have a hosted cluster 12 data nodes with almost 2800 indices and over 11100 shards.

It looks like the interface is gathering more information than just a list of the indices, when that is really all that is needed at that stage of the process. running GET _cat/indices/ only takes a few seconds. modifying the call so that it only collects an index list should fix the issue.

[mattkime]

This should be greatly improved by #70271 - available in v7.9