Pre-configured connectors to become available on the Cloud

[arisonl]

We have a couple of requests now from users for pre-configured connectors to become available on the Cloud (ESS and ECE). The most recent one involves allowing users to be able to create connectors that apply across Spaces in environments with many Kibana Spaces and users across organisations. The internal platform admins in such use cases would like to be able to provide connectors out of the box to their internal customers applicable across Spaces (@pmuellr and myself were on the call, but it is not the first time we hear this request). The particular customer operates both ECE and self-managed deployments and noted that these product discrepancies should not exist.

Moreover, we have started building additional functionality on top of pre-configured connectors, e.g.: #94909 As a result the gap cascades.

---

As a side-issue: the pre-configured history connector setting is wrongly marked as "available on the Cloud" in the documentation: https://www.elastic.co/guide/en/kibana/7.13/alert-action-settings-kb.html

[elasticmachine]

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

[mikecote]

@arisonl is there a reason this customer would prefer pre-configured connectors instead of sharing connectors across spaces?

[ymao1]

As a side-issue: the pre-configured history connector setting is wrongly marked as "available on the Cloud" in the documentation: https://www.elastic.co/guide/en/kibana/7.13/alert-action-settings-kb.html

@arisonl The config has been added via this PR but I'm not sure how cloud releases correlate to Kibana releases. It is tagged as cloud release ms-56

[arisonl]

Thanks @ymao1. Are pre-configured connectors available on the Cloud? They were not last time I checked at least. How will this work on the Cloud if that's still the case?

[arisonl]

@mikecote We talked to them about shared connectors and they have not specified a preference. I can see however how a pre-conf connector is of value in a setup similar to theirs, where they provide a cloud platform to internal "customers" in the context of a large enterprise. For the record, shared saved objects are not a 7.X thing.

[ymao1]

Thanks @ymao1. Are pre-configured connectors available on the Cloud? They were not last time I checked. How will this work on the Cloud if pre-config'ed connectors are not available?

@arisonl You are correct that preconfigured connectors are not configurable on cloud. This is a true/false flag which, if true, sets up this connector as part of plugin start contract...it does not try to read any connector information from the kibana config

[mikecote]

@mikecote We talked to them about shared connectors and they have not specified a preference. I can see however how a pre-conf connector is of value in a setup similar to theirs, where they provide a cloud platform to internal "customers" in the context of a large enterprise. For the record, shared saved objects are not a 7.X thing.

I see, ok. Should we merge this discussion with #73073? I'm not opposed to allowing such capability but wanted to make sure if there's a better experience (ex: sharing connectors) that we explore that path as well.

[pmuellr]

Re: shared connectors vs pre-configured connectors: pre-configured connectors are available today :-)

The customer mentioned a case of having many clusters with many spaces, and some "admin" ish email connector for some of their own monitoring. Pre-configured connectors seem like they'd work out quite well there. They are actually using ECE and not ESS, so ESS allow-listing is actually not an issue for this customer, but I think we should enable it anyway.

One thing I'm not sure of is if our Docker allow-listing then gets inherited by ECE.

I seem to remember that Peter may have had a concern about pre-configured connectors for cloud, but ... not sure. I know he was sensitive to viewing any info about them like we allow with other connectors, and I get that, and think that still holds. But I think allowing them on ESS is just adding them to the allowlist, and I don't see any issues (security-wise) of doing this.

[mikecote]

We should also think about the import/export experience. Since the pre-configured connectors won't move from system to system, what is the expected behaviour when rules reference them? Do we assume the same experience as if a user deleted a connector and no longer exists? (cc @ymao1)

[ymao1]

Currently, if the kibana config is changed to remove a preconfigured connector, the UX for rules that use that connector is the same as for non preconfigured connectors that are deleted. So for the import/export case, I think it makes sense that importing a rule that references a preconfigured connector that doesn't exist in the new system should have that same UX.

In all these cases, I think it'd be useful to have some sort of indicator on the rule details page that there's something wrong with the connector. Currently, there's no indication, a user would have to open the edit flyout to see the error, which they likely wouldn't do until the action starts throwing errors, which for imported rules, wouldn't happen until the rule is enabled. This indicator would be nice so that the user could see they need to fix something before re-enabling the rule.

[mikecote]

Closing as duplicate of https://github.com/elastic/cloud/issues/80803.