terraform refresh attempts to dial localhost

[swtch1]

Terraform Version

$ terraform -v
Terraform v0.12.3
+ provider.google v2.10.0
+ provider.google-beta v2.10.0
+ provider.kubernetes v1.7.0

Terraform Configuration Files

../modules/gke_cluster

resource "google_container_cluster" "gke_cluster" {
  provider                 = "google-beta"
  project                  = var.project_id
  name                     = var.name
  description              = var.description
  location                 = var.location
  network                  = var.network
  subnetwork               = var.subnetwork
  cluster_ipv4_cidr        = var.cluster_ipv4_cidr
  logging_service          = "logging.googleapis.com/kubernetes"
  monitoring_service       = "monitoring.googleapis.com/kubernetes"
  remove_default_node_pool = true
  initial_node_count       = var.initial_node_count
  master_authorized_networks_config {
    cidr_blocks {
      cidr_block   = "207.11.1.0/24"
      display_name = "SSC Web-Proxies"
    }
    cidr_blocks {
      cidr_block   = "207.11.39.0/24"
      display_name = "ATC Web-Proxies"
    }
    cidr_blocks {
      cidr_block   = "207.11.113.0/24"
      display_name = "SSC NAT Range"
    }
    cidr_blocks {
      cidr_block   = "165.130.255.119/32"
      display_name = "QA Web-Proxy"
    }
  }
  maintenance_policy {
    daily_maintenance_window {
      # Time Specified in UTC. EDT=UTC-4, EST=UTC-5 
      start_time = "07:00"
    }
  }
  ip_allocation_policy {
    use_ip_aliases = true
  }
  private_cluster_config {
    enable_private_nodes   = var.enable_private_nodes
  }
}

resource "google_container_node_pool" "default-pool" {
  name     = "default-pool"
  cluster  = google_container_cluster.gke_cluster.name
  location = var.location
  node_config {
    machine_type = "n1-standard-4"
    oauth_scopes = [
      "https://www.googleapis.com/auth/cloud-platform"
    ]
  }
  initial_node_count = var.initial_node_count
  autoscaling {
    min_node_count = var.default_pool_min_node_count
    max_node_count = var.default_pool_max_node_count
  }
  management {
    auto_repair  = true
    auto_upgrade = true
  }
}

resource "kubernetes_namespace" "namespace" {
  count = length(var.namespaces)
  metadata {
    name   = var.namespaces[count.index].name
    labels = var.namespaces[count.index].labels
  }
}

#Send GKE Logs to BigQuery
resource "google_bigquery_dataset" "dataset" {
  dataset_id  = "GKE_LOGS"
  description = "Dataset used to store GKE Logs"
  location    = "US"
  labels = {
    team    = "sre",
    purpose = "logs"
  }
  access {
    role          = "WRITER"
    user_by_email = "[email protected]"
  }
  lifecycle {
    ignore_changes = [access]
  }
}

resource "google_logging_project_sink" "log_sink" {
  name                   = "gke_logs"
  destination            = "bigquery.googleapis.com/projects/${var.project_id}/datasets/${google_bigquery_dataset.dataset.dataset_id}"
  filter                 = "resource.labels.cluster_name=\"${google_container_cluster.gke_cluster.name}\""
  unique_writer_identity = false
}

resource "google_logging_project_exclusion" "log_exclusion" {
  name        = "gke_logs"
  description = "Exclude all GKE logs"
  filter      = "resource.labels.cluster_name=\"${google_container_cluster.gke_cluster.name}\""
}

variable "project_id" {
  description = "GCP project ID. See all accessible project IDs with `gcloud projects list` (required)"
}

variable "name" {
  description = "(Required) Cluster name. ref: https://www.terraform.io/docs/providers/google/r/container_cluster.html#name"
}

variable "description" {
  description = "Description of the cluster."
}

variable "location" {
  description = "Cluster location. ref: https://www.terraform.io/docs/providers/google/r/container_cluster.html#location.  Use https://cloud.google.com/compute/docs/regions-zones/ to find valid zones."
  default     = "us-east1-b"
}

variable "network" {
  description = "VPC network for the cluster nodes. https://www.terraform.io/docs/providers/google/r/container_cluster.html#network"
  default     = null
}

variable "subnetwork" {
  description = "https://www.terraform.io/docs/providers/google/r/container_cluster.html#subnetwork"
  default     = null
}

variable "cluster_ipv4_cidr" {
  description = "Referenced in the Kubernetes console as 'pod address range.'. https://www.terraform.io/docs/providers/google/r/container_cluster.html#cluster_ipv4_cidr"
  default     = null
}

variable "enable_private_nodes" {
  description = "https://www.terraform.io/docs/providers/google/r/container_cluster.html#enable_private_nodes"
  default     = false
}

variable "initial_node_count" {
  description = "https://www.terraform.io/docs/providers/google/r/container_cluster.html#initial_node_count"
  default     = 1
}

variable "default_pool_min_node_count" { # TODO: this will likely need to be refactored into an object so we can create several node pools
  description = "https://www.terraform.io/docs/providers/google/r/container_node_pool.html#min_node_count"
  default     = 1
}

variable "default_pool_max_node_count" { # TODO: this will likely need to be refactored into an object so we can create several node pools
  description = "https://www.terraform.io/docs/providers/google/r/container_node_pool.html#max_node_count"
  default     = 3
}

variable "namespaces" {
  type = list(object({
    name = string,
    labels = object({
      team    = string,
      purpose = string
    })
  }))
  description = "List of cluster namespaces and associated properties like labels."
  default     = []
}

main.tf

#Variable Declarations
variable "project_id" {
  description = "GCP project ID. See all accessible project IDs with `gcloud projects list` (required)"
  type        = "string"
}

#Resource Definitions
provider "google" {
  version = "~> 2.10.0"
  project = var.project_id
}

provider "google-beta" {
  version = "~> 2.10.0"
  project = var.project_id
}

data "google_client_config" "default" {}

terraform {
  backend "gcs" {
    bucket = "com-tf-state"
    prefix = "np-com-internal" # TODO: this really should be np-com-internal-thd, but this is a breaking change that needs to be specially handled
  }
}

provider "kubernetes" {
  version                = "1.7" # provider version, not Kubernetes version
  host                   = "https://${module.common_gke_cluster.endpoint}"
  token                  = data.google_client_config.default.access_token
  cluster_ca_certificate = base64decode(module.common_gke_cluster.cluster_ca_certificate)
  load_config_file       = false
}

module "common_gke_cluster" {
  source                      = "../modules/gke_cluster"
  project_id                  = var.project_id
  name                        = "common-east"
  description                 = "Shared cluster in East region for generalized workloads in lower lifecycle."
  location                    = "us-east1"
  enable_private_nodes        = true
  network                     = "vpc-cassandra"
  subnetwork                  = "cassandra-east-np"
  default_pool_min_node_count = 2
  default_pool_max_node_count = 4
  namespaces = [
    {
      name   = "prometheus",
      labels = { team = "sre", purpose = "application_monitoring" }
    },
    {
      name   = "debug",
      labels = { team = "sre", purpose = "cluster_debugging" }
    },
  ]
}

Debug Output

terraform refresh trace

Expected Behavior

I expected Terraform to refresh the state.

Actual Behavior

Error: Get http://localhost/api/v1/namespaces/prometheus: dial tcp 127.0.0.1:80: connect: connection refused
Error: Get http://localhost/api/v1/namespaces/debug: dial tcp 127.0.0.1:80: connect: connection refused

Steps to Reproduce

terraform refresh

Additional Context

The two resources in the error (/namespaces/prometheus and /namespaces/debug/) are namespaces for my Kubernetes cluster.

[ghost]

This issue has been automatically migrated to hashicorp/terraform-provider-kubernetes#546 because it looks like an issue with that provider. If you believe this is not an issue with the provider, please reply to hashicorp/terraform-provider-kubernetes#546.

[swtch1]

So the terraform-provider-kubernetes issue created #546 was closed. This has now been closed on both sides without resolution.

Is this scoped as a missing feature? How should this be handled so it's not just abandoned by both parties?

[teamterraform]

@swtch1 As Paul mentioned in the comment when closing hashicorp/terraform-provider-kubernetes#546

This seems like the upstream progressive apply issue: #4149
You cannot currently (reliably) chain together a provider's config with the output of a resource.

The resolution is therefore #4149

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.