Enabling private nodes for GKE clusters #538

sgibson91 · 2021-07-21T09:45:33Z

Summary

This PR enables private nodes for our GKE clusters in the terraform config and also provisions a VPC and CloudNAT as network support. This is mostly to comply with organisational constraints discovered in #489, and prepare for any future collaborator that may have similar requirements. I learned a lot from this tutorial when designing this config.

Acceptance Criteria

Must be controlled with a flag to prevent force-recreating our other clusters which would cause a large outage
kubectl and gcloud compute ssh access must be preserved for engineer's local machines and our CI/CD workflows

Tests

Project deployed to	Deployed config (public or private?)	terraform completed?	`kubeconfig` entry fetched successfully?	`kubectl get nodes` successful?	`gcloud compute ssh NODE_NAME` successful?
`two-eye-two-see-sandbox`	public	✅	✅	✅	✅
`two-eye-two-see-sandbox`	private	✅	✅	✅	✅
`pangeo-integration-te-3eea`	private	✅	✅	✅	✅

*Note: public config cannot be tested on pangeo-integration-te-3eea as this violates the controls in place on the tenancy

Topics to Discuss

We must provide IP CIDR ranges for the private_cluster_config block. I used values from the tutorial (linked above). Should we change them? If so, what to?

private_cluster_config:

https://github.com/2i2c-org/pilot-hubs/blob/c5bd53d5604eeb0ea131000b42065ffe44009395/terraform/cluster.tf#L23-L24

Seems like a lot of stuff is getting changed after deployment by something (see this comment). We need to understand where these changes are coming from and what to do about them, as I believe it will cause a force-replacement of the cluster if applied without using the ignore_changes meta-argument. Update: I think I have fixed this by no longer relying on the network and cloudnat terraform modules.

What should a reviewer concentrate their feedback on?

Everything looks ok?

I'm hoping this preserves kubectl access for our local machines and GitHub Actions

sgibson91 · 2021-07-21T10:03:16Z

I have been trying this out on #489. First speed bump is that master_ipv4_cidr_block must be set.

│ Error: 1 error occurred:
│       * master_ipv4_cidr_block must be set if enable_private_nodes is true
│ 
│   with google_container_cluster.cluster,
│   on cluster.tf line 1, in resource "google_container_cluster" "cluster":
│    1: resource "google_container_cluster" "cluster" {

What happens internally and can we copy that?

May be able to steal some defaults from this PR

sgibson91 · 2021-07-21T10:13:53Z

Other q's to consider:

Will we need to deploy a Cloud NAT separately or will it be done for us?
Are our current clusters "VPS native"? If so, what are the disadvantages of this? Are there pod density restrictions?

sgibson91 · 2021-07-21T10:20:01Z

Looks like we are Routes based by default from the terraform config: https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/container_cluster#networking_mode

If that is a switch we want, it will need to be behind a flag as it will force-recreate our other GCP clusters

sgibson91 · 2021-07-22T10:21:35Z

@yuvipanda recommended watching https://www.youtube.com/watch?v=fI-5LkBDap8

We will definitely need a Cloud NAT

sgibson91 · 2021-07-23T13:13:11Z

Ok, here's how I think I'm going to approach this:

Learn how to deploy the private, VPC Native cluster with CloudNAT and public master endpoint
Work out a terraform config that will also achieve 1
Work out the difference between the config in 2 and what we have in pilot-hubs, and also exactly what needs to be behind a flag for this PR

sgibson91 · 2021-07-23T16:43:24Z

I have a simple terraform config that successfully deploys a private cluster with VPC and CloudNAT to the pangeo-integration-te project: https://gist.github.com/sgibson91/ba1885e640e2bb736367dc42c203d12f

Based off much of the terraform in this tutorial:https://cloud.google.com/nat/docs/gke-example#terraform

Using terraform v1.0 allows us to use the count parameter in module definitions

For private clusters, we dynamically provision the following attribute blocks: - private_cluster_config - ip_allocation_policy - Parse network and subnetwork from the VPC

Deploy a VPC and Cloud NAT with router and a firewall rule to allow SSH

…to tf-private-nodes

sgibson91 · 2021-07-27T16:58:24Z

The terraform config in this PR coupled with that in #489 successfully deployed to pangeo-integration-te-3eea. However, received an error when trying to generate the kubeconfig entry:

$ gcloud container clusters get-credentials pangeo-hubs-cluster
Fetching cluster endpoint and auth data.
ERROR: (gcloud.container.clusters.get-credentials) ResponseError: code=403, message=Required "container.clusters.get" permission(s) for "projects/pangeo-integration-te-3eea/zones/us-central1-b/clusters/pangeo-hubs-cluster".

sgibson91 · 2021-07-28T10:48:08Z

The terraform config in this PR coupled with that in #489 successfully deployed to pangeo-integration-te-3eea. However, received an error when trying to generate the kubeconfig entry:
$ gcloud container clusters get-credentials pangeo-hubs-cluster
Fetching cluster endpoint and auth data.
ERROR: (gcloud.container.clusters.get-credentials) ResponseError: code=403, message=Required "container.clusters.get" permission(s) for "projects/pangeo-integration-te-3eea/zones/us-central1-b/clusters/pangeo-hubs-cluster".

This was a combination of unhelpful error message and me being unobservant. trying to pull credentials for a cluster called pangeo-hubs-cluster, but the cluster I'd actually deployed was called test-cluster.

I redeployed this morning, paying attention to my prefixes, and now I've got credentials for kubectl.

sgibson91 · 2021-07-28T16:22:16Z

So it seems that things are being changes to the deployment after running terraform apply (which is not me clicking things in the console!). ~~Wonder if this would happen anyway or if it's stuff that is imposed by CUIT?~~ [Update: I tested again on our sandbox project and the same output happens there.] Either way, I guess we should explicitly fix some of this config.

Outside Terraform changes

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply":

  # google_storage_bucket.user_buckets["pangeo-scratch"] has been changed
  ~ resource "google_storage_bucket" "user_buckets" {
        id                          = "pangeo-hubs-pangeo-scratch"
      + labels                      = {}
        name                        = "pangeo-hubs-pangeo-scratch"
        # (10 unchanged attributes hidden)
    }
  # google_compute_firewall.firewall_rules[0] has been changed
  ~ resource "google_compute_firewall" "firewall_rules" {
        id                      = "projects/pangeo-integration-te-3eea/global/firewalls/allow-ssh"
        name                    = "allow-ssh"
      + source_service_accounts = []
      + source_tags             = []
      + target_service_accounts = []
      + target_tags             = []
        # (9 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }
  # google_container_node_pool.core has been changed
  ~ resource "google_container_node_pool" "core" {
        id                  = "projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/core-pool"
        name                = "core-pool"
        # (9 unchanged attributes hidden)



      ~ node_config {
          + tags              = []
            # (12 unchanged attributes hidden)


            # (2 unchanged blocks hidden)
        }

        # (3 unchanged blocks hidden)
    }
  # google_container_node_pool.notebook["user"] has been changed
  ~ resource "google_container_node_pool" "notebook" {
        id                  = "projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/nb-user"
        name                = "nb-user"
        # (9 unchanged attributes hidden)



      ~ node_config {
          + tags              = []
            # (12 unchanged attributes hidden)


            # (2 unchanged blocks hidden)
        }

        # (3 unchanged blocks hidden)
    }
  # google_artifact_registry_repository.registry has been changed
  ~ resource "google_artifact_registry_repository" "registry" {
        id            = "projects/pangeo-integration-te-3eea/locations/us-central1/repositories/pangeo-hubs-registry"
      + labels        = {}
        name          = "pangeo-hubs-registry"
        # (6 unchanged attributes hidden)
    }
  # google_container_node_pool.dask_worker["worker"] has been changed
  ~ resource "google_container_node_pool" "dask_worker" {
        id                  = "projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster/nodePools/dask-worker"
        name                = "dask-worker"
        # (9 unchanged attributes hidden)



      ~ node_config {
          + tags              = []
            # (12 unchanged attributes hidden)


            # (2 unchanged blocks hidden)
        }

        # (3 unchanged blocks hidden)
    }
  # google_container_cluster.cluster has been changed
  ~ resource "google_container_cluster" "cluster" {
        id                          = "projects/pangeo-integration-te-3eea/locations/us-central1-b/clusters/pangeo-hubs-cluster"
      ~ instance_group_urls         = [
          + "https://www.googleapis.com/compute/beta/projects/pangeo-integration-te-3eea/zones/us-central1-b/instanceGroups/gke-pangeo-hubs-cluster-core-pool-b0e96b88-grp",
          + "https://www.googleapis.com/compute/beta/projects/pangeo-integration-te-3eea/zones/us-central1-b/instanceGroups/gke-pangeo-hubs-cluster-nb-user-b210cb87-grp",
          + "https://www.googleapis.com/compute/beta/projects/pangeo-integration-te-3eea/zones/us-central1-b/instanceGroups/gke-pangeo-hubs-cluster-dask-worker-5d233f1a-grp",
        ]
        name                        = "pangeo-hubs-cluster"
      + resource_labels             = {}
        # (26 unchanged attributes hidden)









      + node_config {
          + disk_size_gb      = 30
          + disk_type         = "pd-standard"
          + guest_accelerator = []
          + image_type        = "COS_CONTAINERD"
          + labels            = {
              + "hub.jupyter.org/node-purpose" = "core"
              + "k8s.dask.org/node-purpose"    = "core"
            }
          + local_ssd_count   = 0
          + machine_type      = "n1-highmem-4"
          + metadata          = {
              + "disable-legacy-endpoints" = "true"
            }
          + oauth_scopes      = [
              + "https://www.googleapis.com/auth/cloud-platform",
            ]
          + preemptible       = false
          + service_account   = "pangeo-hubs-cluster-sa@pangeo-integration-te-3eea.iam.gserviceaccount.com"
          + tags              = []
          + taint             = []

          + shielded_instance_config {
              + enable_integrity_monitoring = true
              + enable_secure_boot          = false
            }

          + workload_metadata_config {
              + node_metadata = "GKE_METADATA_SERVER"
            }
        }

      + node_pool {
          + initial_node_count  = 1
          + instance_group_urls = [
              + "https://www.googleapis.com/compute/v1/projects/pangeo-integration-te-3eea/zones/us-central1-b/instanceGroupManagers/gke-pangeo-hubs-cluster-core-pool-b0e96b88-grp",
            ]
          + max_pods_per_node   = 110
          + name                = "core-pool"
          + node_count          = 1
          + node_locations      = [
              + "us-central1-b",
            ]
          + version             = "1.19.9-gke.1900"

          + autoscaling {
              + max_node_count = 5
              + min_node_count = 1
            }

          + management {
              + auto_repair  = true
              + auto_upgrade = false
            }

          + node_config {
              + disk_size_gb      = 30
              + disk_type         = "pd-standard"
              + guest_accelerator = []
              + image_type        = "COS_CONTAINERD"
              + labels            = {
                  + "hub.jupyter.org/node-purpose" = "core"
                  + "k8s.dask.org/node-purpose"    = "core"
                }
              + local_ssd_count   = 0
              + machine_type      = "n1-highmem-4"
              + metadata          = {
                  + "disable-legacy-endpoints" = "true"
                }
              + oauth_scopes      = [
                  + "https://www.googleapis.com/auth/cloud-platform",
                ]
              + preemptible       = false
              + service_account   = "pangeo-hubs-cluster-sa@pangeo-integration-te-3eea.iam.gserviceaccount.com"
              + tags              = []
              + taint             = []

              + shielded_instance_config {
                  + enable_integrity_monitoring = true
                  + enable_secure_boot          = false
                }

              + workload_metadata_config {
                  + node_metadata = "GKE_METADATA_SERVER"
                }
            }

          + upgrade_settings {
              + max_surge       = 1
              + max_unavailable = 0
            }
        }
      + node_pool {
          + initial_node_count  = 0
          + instance_group_urls = [
              + "https://www.googleapis.com/compute/v1/projects/pangeo-integration-te-3eea/zones/us-central1-b/instanceGroupManagers/gke-pangeo-hubs-cluster-nb-user-b210cb87-grp",
            ]
          + max_pods_per_node   = 110
          + name                = "nb-user"
          + node_count          = 0
          + node_locations      = [
              + "us-central1-b",
            ]
          + version             = "1.19.9-gke.1900"

          + autoscaling {
              + max_node_count = 20
              + min_node_count = 0
            }

          + management {
              + auto_repair  = true
              + auto_upgrade = false
            }

          + node_config {
              + disk_size_gb      = 100
              + disk_type         = "pd-standard"
              + guest_accelerator = []
              + image_type        = "COS_CONTAINERD"
              + labels            = {
                  + "hub.jupyter.org/node-purpose" = "user"
                  + "k8s.dask.org/node-purpose"    = "scheduler"
                }
              + local_ssd_count   = 0
              + machine_type      = "n1-highmem-4"
              + metadata          = {
                  + "disable-legacy-endpoints" = "true"
                }
              + oauth_scopes      = [
                  + "https://www.googleapis.com/auth/cloud-platform",
                ]
              + preemptible       = false
              + service_account   = "pangeo-hubs-cluster-sa@pangeo-integration-te-3eea.iam.gserviceaccount.com"
              + tags              = []
              + taint             = [
                  + {
                      + effect = "NO_SCHEDULE"
                      + key    = "hub.jupyter.org_dedicated"
                      + value  = "user"
                    },
                ]

              + shielded_instance_config {
                  + enable_integrity_monitoring = true
                  + enable_secure_boot          = false
                }

              + workload_metadata_config {
                  + node_metadata = "GKE_METADATA_SERVER"
                }
            }

          + upgrade_settings {
              + max_surge       = 1
              + max_unavailable = 0
            }
        }
      + node_pool {
          + initial_node_count  = 0
          + instance_group_urls = [
              + "https://www.googleapis.com/compute/v1/projects/pangeo-integration-te-3eea/zones/us-central1-b/instanceGroupManagers/gke-pangeo-hubs-cluster-dask-worker-5d233f1a-grp",
            ]
          + max_pods_per_node   = 110
          + name                = "dask-worker"
          + node_count          = 0
          + node_locations      = [
              + "us-central1-b",
            ]
          + version             = "1.19.9-gke.1900"

          + autoscaling {
              + max_node_count = 100
              + min_node_count = 0
            }

          + management {
              + auto_repair  = true
              + auto_upgrade = false
            }

          + node_config {
              + disk_size_gb      = 100
              + disk_type         = "pd-ssd"
              + guest_accelerator = []
              + image_type        = "COS_CONTAINERD"
              + labels            = {
                  + "k8s.dask.org/node-purpose" = "worker"
                }
              + local_ssd_count   = 0
              + machine_type      = "n1-highmem-4"
              + metadata          = {
                  + "disable-legacy-endpoints" = "true"
                }
              + oauth_scopes      = [
                  + "https://www.googleapis.com/auth/cloud-platform",
                ]
              + preemptible       = true
              + service_account   = "pangeo-hubs-cluster-sa@pangeo-integration-te-3eea.iam.gserviceaccount.com"
              + tags              = []
              + taint             = [
                  + {
                      + effect = "NO_SCHEDULE"
                      + key    = "k8s.dask.org_dedicated"
                      + value  = "worker"
                    },
                ]

              + shielded_instance_config {
                  + enable_integrity_monitoring = true
                  + enable_secure_boot          = false
                }

              + workload_metadata_config {
                  + node_metadata = "GKE_METADATA_SERVER"
                }
            }

          + upgrade_settings {
              + max_surge       = 1
              + max_unavailable = 0
            }
        }





        # (13 unchanged blocks hidden)
    }
  # module.cloud-nat[0].google_compute_router_nat.main has been changed
  ~ resource "google_compute_router_nat" "main" {
      + drain_nat_ips                       = []
        id                                  = "pangeo-integration-te-3eea/us-central1/pangeo-hubs-router/pangeo-hubs-cloud-nat"
        name                                = "pangeo-hubs-cloud-nat"
      + nat_ips                             = []
        # (11 unchanged attributes hidden)
    }
  # module.vpc_module[0].module.subnets.google_compute_subnetwork.subnetwork["us-central1/pangeo-hubs-us-central1-subnet"] has been changed
  ~ resource "google_compute_subnetwork" "subnetwork" {
        id                         = "projects/pangeo-integration-te-3eea/regions/us-central1/subnetworks/pangeo-hubs-us-central1-subnet"
        name                       = "pangeo-hubs-us-central1-subnet"
      ~ private_ip_google_access   = false -> true
      ~ secondary_ip_range         = [
          + {
              + ip_cidr_range = "10.8.0.0/14"
              + range_name    = "gke-pangeo-hubs-cluster-pods-c0c2b09a"
            },
          + {
              + ip_cidr_range = "10.12.0.0/20"
              + range_name    = "gke-pangeo-hubs-cluster-services-c0c2b09a"
            },
        ]
        # (8 unchanged attributes hidden)
    }

Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using ignore_changes, the following plan may include actions to undo or respond to these changes.

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.vpc_module[0].module.subnets.google_compute_subnetwork.subnetwork["us-central1/pangeo-hubs-us-central1-subnet"] will be updated in-place
  ~ resource "google_compute_subnetwork" "subnetwork" {
        id                         = "projects/pangeo-integration-te-3eea/regions/us-central1/subnetworks/pangeo-hubs-us-central1-subnet"
        name                       = "pangeo-hubs-us-central1-subnet"
      ~ private_ip_google_access   = true -> false
      ~ secondary_ip_range         = [
          - {
              - ip_cidr_range = "10.8.0.0/14"
              - range_name    = "gke-pangeo-hubs-cluster-pods-c0c2b09a"
            },
          - {
              - ip_cidr_range = "10.12.0.0/20"
              - range_name    = "gke-pangeo-hubs-cluster-services-c0c2b09a"
            },
        ]
        # (8 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

yuvipanda · 2021-07-29T12:01:43Z

So it seems that things are being changes to the deployment after running terraform apply

I've encountered this before too. It's always been either things getting set to empty values (like {} or []), or a default version changing (particularly on k8s). I've just explicitly specified them to change it. Often these will also 'flap' - switch back and forth between unset and empty values on each run...

…to tf-private-nodes

It belongs in 2i2c-org#489

sgibson91 · 2021-08-03T15:54:16Z

I think we may as well get this reviewed / merged while I think about the sops issue for deploying a hub. This coupled with #489 has already been deployed to the Pangeo project.

yuvipanda

This looks great! I left some minor comments about style.

Each GCP Project comes with a 'default network', and that's where our GKE clusters are launched into right now. Can we continue to use this? This also removes a lot of decisions from our hand, like what IP ranges to use - those can just be the defaults. We'll just need to add the cloud NAT. Terraform data blocks might be needed here?

If we can't use the default network (and its subnetworks), we need to figure out:

What are the sizes of the various IP ranges we pick for pod, node and service IPs? Picking the defaults seem ok, but let's make sure we know how many pods, nodes, and services we can support with those choices.
If we change those values in the future, does it recreate the entire cluster?

Thanks a lot for working on this, @sgibson91!!!

terraform/gcp/network.tf

terraform/gcp/cluster.tf

yuvipanda · 2021-08-03T16:29:43Z

terraform/gcp/cluster.tf

+
+    content {
+      // Decide if this CIDR block is sensible or not
+      master_ipv4_cidr_block  = "172.16.0.0/28"


Can we leave this unset?

Alas, no :( #538 (comment)

Co-authored-by: Yuvi Panda <[email protected]>

Replaces creating our own network and subnetwork. This means we don't need to worry about address spaces for pods/services/etc

sgibson91 · 2021-08-03T17:32:51Z

Each GCP Project comes with a 'default network', and that's where our GKE clusters are launched into right now. Can we continue to use this?

I think I have achieved this in 632048d

yuvipanda · 2021-08-04T05:50:54Z

terraform/gcp/network.tf

+  source_ranges = ["35.235.240.0/20"]
+}
+
+resource "google_compute_router" "router" {


I tried to see if there exists a default router we can use too, and based on reading the docs and running gcloud compute routers list I've come to the conclusion we have to create our own. So this LGTM

terraform/gcp/variables.tf

yuvipanda · 2021-08-04T06:01:53Z

I've updated the docstring for enable_private_cluster, and satisfied myself that we can't drop any more resources here. Thank you for working on this, @sgibson91!

choldgraf · 2021-08-04T06:04:54Z

Amazing! 🎉

sgibson91 added 2 commits July 21, 2021 10:43

Enable private nodes in GKE cluster

67a168b

Disable private endpoint in GKE cluster

e170da4

I'm hoping this preserves kubectl access for our local machines and GitHub Actions

sgibson91 self-assigned this Jul 23, 2021

sgibson91 added 🏷️ pangeo Enhancement An improvement to something or creating something new. labels Jul 23, 2021

sgibson91 mentioned this pull request Jul 26, 2021

Team Sync - Jul 26, 2021 2i2c-org/team-compass#172

Closed

sgibson91 added 7 commits July 27, 2021 15:03

Set a minimum version of terraform

f0b2ecb

Using terraform v1.0 allows us to use the count parameter in module definitions

Add a boolean flag to enable private clusters

07efc60

Dynamically provision cluster attribute blocks

894a7e4

For private clusters, we dynamically provision the following attribute blocks: - private_cluster_config - ip_allocation_policy - Parse network and subnetwork from the VPC

Add networking support for private clusters

bf19129

Deploy a VPC and Cloud NAT with router and a firewall rule to allow SSH

Restructure code block so terraform lint passes

fe42c83

Restructure code block so terraform lint passes

51664ee

Merge branch 'tf-private-nodes' of github.com:sgibson91/pilot-hubs in…

c5bd53d

…to tf-private-nodes

This was referenced Jul 27, 2021

Deploying Cluster for Pangeo #489

Merged

Deploy a cluster into GCP pangeo-integration-te-3eea project for Pangeo pilot hubs #488

Closed

New Hub: Pangeo JupyterHub (GCP) #482

Closed

sgibson91 added 2 commits July 28, 2021 17:39

Add links to module docs in network.tf

8a4d9c2

Tweak condition for ip_allocation_policy in cluster.tf

dbf3bcd

sgibson91 and others added 4 commits August 2, 2021 19:44

Merge branch 'master' into tf-private-nodes

405ad13

Merge branch 'tf-private-nodes' of github.com:sgibson91/pilot-hubs in…

d41b1c5

…to tf-private-nodes

Move network file under GCP dir

da2b3c5

Shouldn't add the deployer key in this PR

8cd8202

It belongs in 2i2c-org#489

This was referenced Aug 2, 2021

Provision a separate nodepool for ohw #573

Merged

New Hub: OceanHackWeek 2021 #549

Closed

sgibson91 added 2 commits August 3, 2021 16:50

Add source IP ranges to firewall rule

333e141

Deploy and Cloud Router and Cloud NAT to handle outbound traffic

87b6fb6

sgibson91 marked this pull request as ready for review August 3, 2021 15:53

sgibson91 requested a review from yuvipanda August 3, 2021 15:54

yuvipanda requested changes Aug 3, 2021

View reviewed changes

sgibson91 and others added 5 commits August 3, 2021 17:59

Change name of firewall rule to be more descriptive

96ec6b9

Co-authored-by: Yuvi Panda <[email protected]>

Use name attribute instead of self_link for consistency

d3bc70c

Document source ranges for IAP SSH ingress firewall rule

a3a17c6

Use the project's default network and subnetwork

632048d

Replaces creating our own network and subnetwork. This means we don't need to worry about address spaces for pods/services/etc

Explicitly set more "flappy" variables

4eccd46

yuvipanda reviewed Aug 4, 2021

View reviewed changes

terraform/gcp/variables.tf Outdated Show resolved Hide resolved

Update doc for enable_private_cluster

48d86ee

yuvipanda merged commit dc65e0b into 2i2c-org:master Aug 4, 2021

sgibson91 deleted the tf-private-nodes branch August 4, 2021 09:25

sgibson91 mentioned this pull request Aug 9, 2021

Team Sync - Aug 09, 2021 2i2c-org/team-compass#196

Closed

choldgraf mentioned this pull request Sep 4, 2021

Blog post: 2i2c quarterly update Q3 2i2c-org/team-compass#235

Closed

4 tasks

choldgraf mentioned this pull request Sep 13, 2021

Blog post: Pangeo hub deployment 2i2c-org/team-compass#244

Open

2 tasks

sgibson91 mentioned this pull request Jan 29, 2022

Columbia GKE VMs can't have external IPs. Use NAT instead? pangeo-forge/pangeo-forge-gcs-bakery#29

Closed

cisaacstern mentioned this pull request Feb 1, 2022

Private GKE nodes pangeo-forge/pangeo-forge-gcs-bakery#30

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enabling private nodes for GKE clusters #538

Enabling private nodes for GKE clusters #538

sgibson91 commented Jul 21, 2021 •

edited

Loading

sgibson91 commented Jul 21, 2021 •

edited

Loading

sgibson91 commented Jul 21, 2021

sgibson91 commented Jul 21, 2021

sgibson91 commented Jul 22, 2021

sgibson91 commented Jul 23, 2021

sgibson91 commented Jul 23, 2021 •

edited

Loading

sgibson91 commented Jul 27, 2021

sgibson91 commented Jul 28, 2021

sgibson91 commented Jul 28, 2021 •

edited

Loading

yuvipanda commented Jul 29, 2021

sgibson91 commented Aug 3, 2021

yuvipanda left a comment

yuvipanda Aug 3, 2021

sgibson91 Aug 3, 2021 •

edited

Loading

sgibson91 commented Aug 3, 2021

yuvipanda Aug 4, 2021

yuvipanda commented Aug 4, 2021

choldgraf commented Aug 4, 2021

Enabling private nodes for GKE clusters #538

Enabling private nodes for GKE clusters #538

Conversation

sgibson91 commented Jul 21, 2021 • edited Loading

Summary

Acceptance Criteria

Tests

Topics to Discuss

What should a reviewer concentrate their feedback on?

sgibson91 commented Jul 21, 2021 • edited Loading

sgibson91 commented Jul 21, 2021

sgibson91 commented Jul 21, 2021

sgibson91 commented Jul 22, 2021

sgibson91 commented Jul 23, 2021

sgibson91 commented Jul 23, 2021 • edited Loading

sgibson91 commented Jul 27, 2021

sgibson91 commented Jul 28, 2021

sgibson91 commented Jul 28, 2021 • edited Loading

yuvipanda commented Jul 29, 2021

sgibson91 commented Aug 3, 2021

yuvipanda left a comment

Choose a reason for hiding this comment

yuvipanda Aug 3, 2021

Choose a reason for hiding this comment

sgibson91 Aug 3, 2021 • edited Loading

Choose a reason for hiding this comment

sgibson91 commented Aug 3, 2021

yuvipanda Aug 4, 2021

Choose a reason for hiding this comment

yuvipanda commented Aug 4, 2021

choldgraf commented Aug 4, 2021

sgibson91 commented Jul 21, 2021 •

edited

Loading

sgibson91 commented Jul 21, 2021 •

edited

Loading

sgibson91 commented Jul 23, 2021 •

edited

Loading

sgibson91 commented Jul 28, 2021 •

edited

Loading

sgibson91 Aug 3, 2021 •

edited

Loading