[Plugin][Flytekit] flytekit storage layer as plugins

[tvetterli-fn]

Why would this plugin be helpful to the Flyte community
When users want to connect to use gcp to store flyte workflow inputs and outputs, gsutil
has to be installed in the docker image.

It would be nice if a flytekit plugin could help us intall gsutil.

Type of Plugin

• Python/Java interface only plugin
• Web Service (e.g. AWS Sagemaker, GCP DataFlow, Qubole etc...)
• Kubernetes Operator (e.g. TfOperator, SparkOperator, FlinkK8sOperator, etc...)
• Customized Plugin using native kubernetes constructs
• Other

Can you help us with the implementation?

• Yes
• No

[kanterov]

One clarification, relevant only for Python SDK. Java SDK doesn't use gsutil.

[kumare3]

A quick idea for the solution here,
The DataProxy object in flytekit is currently hard-coded. This can be easily converted to an extensible system using a pattern like the - https://github.com/flyteorg/flytekit/blob/master/flytekit/core/type_engine.py#L226.

This has been done for taskplugins and types in the latest flytekit and doing this will allow bringing in new implementations for similar data objects.

One of the eventual goals of the system is to use flytecopilot as the data-sidecar to handle data input and output