Orchestration in VDMS with Kubernetes for easier setup of user-defined function environments both locally and on remote servers

[rv355]

Problem Description
VDMS supports an ad-hoc method using which users of VDMS can serve user-defined functions (UDF) locally or on a remote server. Setting up the UDF environment in the manner above suffers from the following problems

• The user will have to manually set up the local/remote UDF environment.
• User has to specify the URL to the remote server explicitly in the query itself
• There is no notion of automatically tackling points of failure in the system
• Resource management (required vs available remote servers) by VDMS is missing

Solution

• Support for an orchestration framework (like Kubernetes) in VDMS should be included, which automatically spins up the required number of remote servers.
• The addition of a scheduler inside VDMS could help make decisions about where a particular query could be served.
• Kubernetes gives some off-the-shelf solutions to deploy applications in such a way that points of failure are minimized.
• A controller inside VDMS could help in resource management in order to create/delete remote servers and services to support remote operations.
• An easy setup script and config file are required to spin up the cluster with the least effort from the VDMS user/administrator.