terraform import fails with the zone does not have enough resources available to fulfill the request

[rohitvyavahare]

I tried to create node-pool using terraform and it failed with following error during apply :

Error reading NodePool "test" from cluster "kube": Nodepool "test" has status "RUNNING_WITH_ERROR" with message "us-west1-c: Deploy error: 
Not all instances running in IGM after 45.677691201s. Expect 1. Current errors: [ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS]: 
Instance 'gke-kube-test-92aefcaa-7t8l' creation failed: 
The zone 'projects/kube-env-181/zones/us-west1-c' does not have enough resources available to fulfill the request.  '(resource type:compute)'. - ; ."

But I can see node-pool created and all 3 nodes (us-west1-a, us-west1-b, us-west1-c) running in node-pool. Now every time I making change in terrform and try terraform apply, it tries to create same node-pool and fails with already exist error. If I try to import with following command:

terraform import  -provider google-beta.gbeta-us-west1 -module.node_pool_test.google_container_node_pool.node_pool kube-env-181/us-west1/kube-env/test

It gives following error :

Acquiring state lock. This may take a few moments...
module.node_pool_test.google_container_node_pool.node_pool: Importing from ID "kube-env-181/us-west1/kube/test"...
module.node_pool_test.google_container_node_pool.node_pool: Import complete!
Imported google_container_node_pool (ID: us-west1/kube/test)
module.node_pool_test.google_container_node_pool.node_pool: Refreshing state... (ID: us-west1/kube/test)

Error: module.node_pool_test.google_container_node_pool.node_pool (import id: kube-env-181/us-west1/kube/test): 1 error(s) occurred:

* import module.node_pool_test.google_container_node_pool.node_pool result: us-west1/kube/test: google_container_node_pool.node_pool: 
Error reading NodePool "test" from cluster "kube": Nodepool "test" has status "RUNNING_WITH_ERROR" with message "us-west1-c: Deploy error: 
Not all instances running in IGM after 45.677691201s. Expect 1. Current errors: [ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS]: 
Instance 'gke-kube-test-92aefcaa-7t8l' creation failed: 
The zone 'projects/kube-env-181/zones/us-west1-c' does not have enough resources available to fulfill the request.  '(resource type:compute)'. - ; ."

Terraform Version

Terraform v0.11.11

• provider.google v2.1.0
• provider.google-beta v2.1.0

Affected Resource(s)

• google_node_pool

[mattnworb]

I've run into a similar issue. If you gcloud container node-pool describe the pool, it seems that the status reported by the GKE API stays as ERROR or RUNNING_WITH_ERROR long after the underlying error has been resolved.

I managed to work around this by triggering a version upgrade on the node pool, which seemed to have the side effect of clearing the node pool's status field.

[rileykarson]

Hmm- I don't think Terraform can do much to mitigate this without losing the ability to handle node pools in a real error state. I filed an issue, so we'll see if that goes anywhere.

[mattnworb]

I think bailing on ERROR makes sense, but what about RUNNING_WITH_ERROR? It is not fully clear to me how the API differentiates between the two.

[rileykarson]

Good point! I'll give that some thought as well.

[Dhanabalan-pop]

I've run into a similar issue. If you gcloud container node-pool describe the pool, it seems that the status reported by the GKE API stays as ERROR or RUNNING_WITH_ERROR long after the underlying error has been resolved.

I managed to work around this by triggering a version upgrade on the node pool, which seemed to have the side effect of clearing the node pool's status field.

We are facing same issue, Is there any other solution other than node-pool upgrade to fix the issue

[rileykarson]

Following the 3.10.0 release ~ Feb 24, this behaviour should be improved if not fixed altogether, as Terraform won't bail out on clusters / pools in error states.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!