Search service should not require pre-fetching

[ek68794998]

Problem statement

Current implementation of the search service expects that all operations are prefetched, and then on the client side, these can be either listed out by connector, or searched using Fuse. However, for Power Automate's consumption, we want to be able to control the results entirely from the server side, and do not want to do pre-fetching, as previously discussed with LAUX team.

We had an update made by @almaasrawi recently (PR linked below) that allowed actual search queries to go through the search service to the host, where we could control the results based on a server-side API call. However, this still required all operations to be pre-fetched; otherwise, when clicking a search result, the operation could not be found, and could not be added to the designer.

Work so far

Recently, we've had work completed in the following PRs:

• feat(designer): Adding search service 2 to allow for custom search results #3201
• fix(designer): bind the search operations methods #3285

Proposed solution

We are still working on a design for the proposed solution. Currently the thinking is:

1. In search service:
   • Add a new method along the lines of getOperationById that asynchronously can fetch a single operation.
   • Add a new method along the lines of getOperationsByConnector that asynchronously can fetch all operations based on which connector.
2. When clicking an operation (recommendationPanelContext.tsx:onOperationClick):
   • If the above getOperationById is not defined in the search service, operate as normal. (Same case as today.)
   • If getOperationById is defined, call the above getOperationById method to see if we can get the operation that way.
3. When clicking a connector (in list/browse view or search view):
   • Basically the same as above for operations, but based on the above getOperationsByConnector function.
4. The host service would then be able to look up individual operations that are cached following the search lookup, or to pull that item from the server if it is not.

@almaasrawi is working on testing this change locally and will bring the change as a pull request, we expect either today or tomorrow. That said, whether this change is something that LAUX will support is something to address as part of this discussion item. Edit: The change can be found in a fork here.

Future work

In addition to the above, we want to open this for discussion as to whether this is a good long-term fix, or whether there is a more elegant solution. In an ideal world, we could completely re-design the search service interface to more concretely decouple search/recommendation from pre-fetching, but this is obviously a significant undertaking as well as a breaking change.

[ek68794998]

We've opened a PR for the proposed (initial/short-term) solution here for review: #3525

[rllyy97]

I've actually considered implementing something similar to what you're doing for our own Logic Apps use-cases.
As far as your proposed solution goes I don't really have any issues / changes to suggest, it doesn't look like it will have any impact on the LA side unless we specifically want it to.
I think some integration can actually be done on the logic apps side to use some of the functionality you've built with the explicit search PR.
Thanks for the detailed breakdown! I'll take a look at the PR

Once this work is in we could also probably take a look at marking a lot of the current search methods as optional and just run them if they're present, then we'd be able to use one or the other or both in a hybrid sense

[rllyy97]

Closing discussion