[ENH] - Allow for deploying managed kubernetes within VPC for each cloud provider

[costrouc]

Feature description

Related to #1210. In #1210 we are adding support for deploying the load balancer within an existing VPC. However this does not completely solve the problem in that the kubernetes infrastructure itself must also be deployed within the vpc. This is a common issue that we have seen on multiple client deployments. Currently our workaround is to require the client deploys a managed kubernetes cluster and we use the local deployment option to deploy onto the existing kubernetes cluster.

We however do not want to require this. In order to support deploying within an existing vpc we need to add some conditional bits to the 02-infrastructure stage. This will require custom terraform for each provider so I recommend starting with a cloud provider that you are most comfortable with. I'd suggest adding a terraform_overrides key to each providers key. E.g.:

• google_cloud_platform.terraform_overrides, amazon_web_services.terraform_overrides, etc. following the same pattern as [ENH] - Support for pinning the IP address of the load balancer and ability to use internal VPC #1210. The goal here is to limit the number of options exposed in the configuration while allowing us to have all these knobs in the terraform modules.

My guess for gcp would be that the qhub-config.yaml would look like:

google_cloud_platform:
    terraform_overrides:
       vpc_id: ....

These overrides would be passed to https://github.com/Quansight/qhub/blob/main/qhub/template/stages/02-infrastructure/gcp/variables.tf.

Value and/or benefit

Ability to deploy qhub within existing VPCs on all cloud providers.

Anything else?

No response

[costrouc]

Azure should be a priority for support.

[costrouc]

Azure is not supported with #1253 being closed

[rsignell-usgs]

Is there an example of how this would look for using existing VPCs on AWS?
We need to do this to deploy Qhub at USGS!