Error when switching cluster from zonal to regional

[kupson]

Warning: changing cluster type removes and re-creates whole cluster. I changed type strictly for testing.

Errors like:

Error: Get "http://localhost/api/v1/namespaces/nginx-ingress": dial tcp [::1]:80: connect: connection refused

Upstream issue - hashicorp/terraform-provider-kubernetes#1102

Workaround:

./scripts/terraform_local_dev.sh apply -target=module.gke.google_container_cluster.primary
./scripts/terraform_local_dev.sh apply -target=helm_release.cert_manager
./scripts/terraform_local_dev.sh apply

Second line is to fix #5.

[kupson]

There is another error when switching between zonal<=>regional. The module.gcloud_no_default_standard_storageclass.module.gcloud_kubectl.null_resource.run_command resource tries to remove annotation from non-existing cluster and fails.

To fix the problem you can remove problematic entries from TF state (as they don't exists any more anyway):

./scripts/terraform_local_dev.sh state list | grep no_default
./scripts/terraform_local_dev.sh state rm module.gcloud_no_default_standard_storageclass.module.gcloud_kubectl.null_resource.run_destroy_command
...

[kupson]

I tried to make it work but there are problems:

• terraform is confused when used-to-exists cluster is completely replaced by a new one (failing on missing CRDs while checking state of resources, missing data sources etc)
• even after careful dance with multiple terraform invocations with different -target module.cluster-* options there are plenty of resources left in limbo (especially around Ingress settings)

Like I suspected initially - This settings cannot be changed on existing cluster. Full cluster re-creation required. already in README.md.