[Proposal][MD] Enable importing saved objects from external data sources

[BionIT]

Context

When multiple data source feature is enabled and with #5712 introduced in 2.12, users can import saved objects from external data sources by uploading ndjson files. However, as a user, I would need to download the file first by login into the external dashboard, download the file, then go to the new dashboard then upload the file which is a bit of work.

Since we already established the connection to external data sources, we can add feature to support import saved objects directly via the connection. Ideally, this experience can be its own page, but since we already have saved object management page as well as the flyout to import objects, we can simply add the fetch from remote option in the flyout

Alternatives

• go with the download then upload option
• have the fetch from remote in its own page instead of having it in the flyout

Proposed solution

In order to implement the feature, we need:

1. feature flag for the fetch from remote under multiple data source plugin
2. in the flyout, we will have fetch from remote option along with upload file option
3. if fetch from remote is clicked, then data source picker should show up
4. add new API to fetch saved objects from remote data source, note if the remote datasource has multiple tenants, it will fetch from global tenant using the credentials provided when data source is connected
5. if fetch from remote, clicking on the import button should call the new API

To improve option 4, we can add option to select the tenants that the current user has access to.

POC

PoC has been conducted to get saved objects from remote data source by selecting the cluster then tenant

tenant_demo.mp4

Next Steps

• Add API to get tenants
• Add API to get saved object from remote cluster
• UI change of the flyout

[BionIT]

Happy to work on it

[seraphjiang]

@BionIT, this might be an Epic item. I'm not sure if we are aiming to finish it in 2.14; if not, let's target it for 2.15.

This is going to increase the adoption rate for MDS. I'm also wondering if this is somehow overlap with workspace object copy?

Let's brainstorm with the UX team @kgcreative and workspace team @ruanyl @SuZhou-Joe

[BionIT]

Yeah, we can target this for 2.15.