[UnifiedSearch] Remove refreshing fields request when switching data data views

[kertal]

In #136256, 8.4, in a cleanup we introduced a change which refreshes the data view fields every time a user switches data view on good intend. Before it was just done when users entered Discover. The motivation was that users get always the most recent field list.

This change works well in local environments and quick requests. But the longer a request for fields takes, the longer the switching of data view in the UI takes, and there's no loading indicator showing that a request is not finished. In large CCS scenarios, where it might take several seconds to return a large field list, the ui is then blocked for this amount of time. Here's an example with a slowed down Network

Kapture.2023-10-19.at.12.07.56.mp4

This behavior was reported at #167706, on a customer call, and is related to #167221 (where a duplicate request when switching data views was reported, this issue is about the first request)

We should consider to revert this change to the old behavior, so the fields should be refreshed when entering Discover or a saved search is being loaded. While not ideal, there would be a loading indicator for this case. This change would also apply to Lens, which also consumes the data view switcher.

The code

This is where the refresh is happening today

kibana/src/plugins/unified_search/public/dataview_picker/change_dataview.tsx

Lines 308 to 313 in e430a7f

	try {
	// refreshing the field list
	await dataViews.get(newId, false, true);
	} catch (e) {
	//
	}

The good:

Faster data view switching, everywhere UnifiedSearch is used with the DataView switcher, which is Discover, Visualize, Lens

The bad:

In case there are new fields ingested they would show up in the field list if they are in the results, but they would be unmapped since this information is not available yet. However, if a field shows up in this way, it's also not transparent to the user how to get the field information (By switching data views? )

The future:

How could we enable adding new fields with mappings to the field list, if they are not present at the initial loading of the DataView? The UnifiedFieldList is triggering a request with an index_filter when in Discover a search request is started. This request is returning the fields of the indices that match the actual search query. So if there are new fields, we could use these to update the displayed fields of the UnifiedFieldList without the need to request all the fields.
Alternatively: Just provide UI in the Fieldlist to fetch all fields, which is updating the cache, just keep it simple

[elasticmachine]

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

[SHolzhauer]

It feels like this issue doesn't only impact the discover app; We face this issue and the observability and security apps are very slow too.

[kertal]

@SHolzhauer yes, this is not limited to Discover. So the parts of Kibana using the DataView Switcher of UnifiedSearch, they will all be faster when switching data views.