GKE Module Design #1

robmorgan · 2019-01-11T07:49:54Z

I intend to use this PR to discuss and design the initial module structure for the GKE package. The goal for release v0.0.1 is to write a Terraform module(s) with tests and examples that can create a GKE cluster.

Proposed Structure

The proposed directory layout is as follows:

├── .gitignore
├── LICENSE
├── README.md
├── examples
│   ├── README.md
│   ├── gke-regional-private-cluster
│   ├── gke-regional-public-cluster
│   ├── gke-zonal-private-cluster
│   ├── gke-zonal-public-cluster
├── modules
│   └── gke-cluster
└── test
    └── Gopkg.toml

Design Decisions

We are postponing support of Zonal clusters until Terraform 0.12 or until the Google provider outputs a single location field.

Open Questions

~~Should we have a parent module gke-cluster that conditionally invokes submodules depending on the desired parameters?~~ Terraform modules don't support count.

Notes

Private clusters require VPC Network Peering.

Tasks

Terratest

Write cluster tests.

brikis98 · 2019-01-11T12:30:49Z

A reminder of a few items we discussed during the team sync today:

I know @yorinasub17 had to spend some time on the following for K8S on AWS, so look into if you need to do the same for GCP:

Rolling update for worker nodes
How does kubectl auth to your cluster in the cloud
How do you do role mapping

Also, add to the docs a comment on why we have several cluster modules rather than 1 due to Terraform 0.11 limitations.

Should we have a parent module gke-cluster that conditionally invokes submodules depending on the desired parameters?

Unfortunately, the Terraform module does not support count.

robmorgan · 2019-01-11T12:34:38Z

@brikis98 thanks Jim 👍

rileykarson · 2019-01-14T19:13:38Z

I'm curious if we need to support both zonal and regional clusters out of the gate; looking at the terraform-google-modules equivalent module, they need to do a lot of bending over backwards to support both right now.

I expect that we could defer zonal support until 0.12 is out and/or the Google provider outputs a single location field (this is something we're thinking about doing right now); if it's still necessary to add all that boilerplate after both of those, we can do so then.

examples/gke-regional-public-cluster/main.tf

examples/gke-regional-public-cluster/README.md

rileykarson · 2019-01-15T20:28:07Z

modules/gke-cluster/main.tf

+  Get available container engine versions
+ *****************************************/
+data "google_container_engine_versions" "region" {
+  zone    = "${data.google_compute_zones.available.names[0]}"


A regional cluster doesn't necessarily support the same version(s) as any/all of its corresponding zones. We'll need to supply the region parameter when working with regional clusters.

rileykarson · 2019-01-15T20:31:00Z

modules/gke-cluster/main.tf

+  project = "${local.network_project_id}"
+}
+
+resource "random_shuffle" "available_zones" {


Just for context, the reason this / the corresponding datasource are useful (https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/blob/3f7527e583ffa07e6a06250844e07c38556a4488/cluster_regional.tf#L29) is because it lets you explicitly define which zones in the region your regional / multi-zonal (afaik regional supersedes multi-zonal) cluster resides in.

examples/gke-regional-public-cluster/variables.tf

Co-Authored-By: robmorgan <[email protected]>

Remove extraneous line.

Add an initial node count to gke-cluster, explain why it's necessary.

…example

[WIP] Add Terratest of the first example

…vate clusters. Minor wording changes.

Configuration fixes for tests

rileykarson

Just realised I had some changes pending in a review pass I thought I'd submitted.

rileykarson · 2019-01-18T16:47:33Z

modules/gke-cluster/outputs.tf

+  value       = "${var.master_authorized_networks_config}"
+}
+
+output "kubernetes_dashboard_enabled" {


This seems like an output we don't need, since the user will have supplied this variable anyways. Or is that standard convention?

yeah can probably go, I must have copied it from the other repos.

rileykarson · 2019-01-18T16:47:52Z

modules/gke-cluster/outputs.tf

+  value       = "${google_container_cluster.cluster.min_master_version}"
+}
+
+output "logging_service" {


Same as dashboard - these are always user supplied (logging, monitoring).

rileykarson · 2019-01-18T16:48:41Z

modules/gke-cluster/outputs.tf

+  value       = "${google_container_cluster.cluster.region}"
+}
+
+output "endpoint" {


Can you add the outputs from https://github.com/gruntwork-io/terraform-google-gke/pull/10/files#diff-c49d0ae81c71a97455954c7c48bc0eca without the base64decode step? I think they'll be necessary when integrating Tiller.

modules/gke-cluster/variables.tf

rileykarson · 2019-01-18T16:51:14Z

modules/gke-cluster/main.tf

+  }
+}
+
+// TODO


This datasource is present at the bottom of the file.

rileykarson · 2019-01-18T16:51:59Z

modules/gke-cluster/main.tf

+  # part of Create. This leaves us in our desired state- with a cluster master
+  # with no node pools.
+  remove_default_node_pool = true
+


Suggested change

To make it clear both lines are connected

I think terraform fmt is causing me problems here at the moment

Co-Authored-By: rileykarson <[email protected]>

Co-Authored-By: robmorgan <[email protected]>

Integrate GKE cluster admin recommendations, clean up module

Add READMEs for gke-cluster, regional examples

…vate clusters. Minor wording changes.

Co-Authored-By: rileykarson <[email protected]>

add working circleci config

[WIP] Better cluster tests

robmorgan · 2019-01-31T03:27:30Z

Hi @rileykarson I'm going to merge this as is. I thought about removing the cluster cert & key outputs, but figured we can do this in the upcoming Helm work. I'll also change all of the other PR bases to master

merge ci changes from upstream

remove legacy files

d2fa373

robmorgan added 5 commits January 15, 2019 08:34

ignore terratest output

f8caca8

Merge branch 'master' into module-design

1deea1a

expand readme

3e8a0d9

add initial module design and planned examples

f081fe1

Merge branch 'master' into module-design

bcd5288

rileykarson reviewed Jan 15, 2019

View reviewed changes

rileykarson and others added 20 commits January 16, 2019 10:29

Update examples/gke-regional-public-cluster/variables.tf

80e7533

Co-Authored-By: robmorgan <[email protected]>

clarify minimum terraform version

7c48e39

project_id to project

14435b9

change to google-beta provider. comment out network stuff

a5405ec

start removing zonal support from module

62ca09b

Add an initial node count, explain why it's necessary.

a83fa51

Remove extraneous line.

a410079

Add an initial node count, explain why it's necessary.

06e7633

Remove extraneous line.

25452bf

wip

8d817cc

Merge pull request #7 from gruntwork-io/module-design-patch-2

6718373

Remove extraneous line.

Merge pull request #6 from gruntwork-io/module-design-patch

5a58c56

Add an initial node count to gke-cluster, explain why it's necessary.

break the node pool out of the gke-cluster module and move it to the …

61eb7c5

…example

only use one of auto_upgrade or version

eed633b

Merge branch 'module-design' into module-design-test

627d58a

Merge pull request #8 from gruntwork-io/module-design-test

f4ebb09

[WIP] Add Terratest of the first example

get project from env vars and test random region instead

f4c7520

Fill out READMEs for GKE Cluster, regional public + private clusters

981452c

Cross-link between private, public examples. Explicitly recommend pri…

8d0ac51

…vate clusters. Minor wording changes.

Update for split out node pools

f8f3e9b

rileykarson and others added 5 commits January 17, 2019 08:31

Configuration fixes for tests

aa783e3

Merge pull request #9 from gruntwork-io/module-design-test-fix

a895a81

Configuration fixes for tests

add circleci config

ac3ed78

clean up module definition

4041c96

output the cluster endpoint

3f8c90c

rileykarson reviewed Jan 22, 2019

View reviewed changes

rileykarson and others added 3 commits January 22, 2019 11:55

Add comment w/ TODO to update regional private cluster docs.

b9be14f

Fix HTML comment

263bdda

Co-Authored-By: rileykarson <[email protected]>

Update modules/gke-cluster/variables.tf

8c5cbb1

Co-Authored-By: robmorgan <[email protected]>

robmorgan changed the title ~~GKE Module Design~~ [WIP] GKE Module Design Jan 24, 2019

rileykarson and others added 12 commits January 24, 2019 10:00

Clean up the GKE cluster master module.

7746681

Clean up gke-regional-public-cluster example

9fe4b2f

Merge pull request #11 from gruntwork-io/module-design-cleanup

d1fef5c

Integrate GKE cluster admin recommendations, clean up module

Merge pull request #5 from gruntwork-io/module-design-rdd

8824e42

Add READMEs for gke-cluster, regional examples

Fill out READMEs for GKE Cluster, regional public + private clusters

12e4210

Cross-link between private, public examples. Explicitly recommend pri…

34e500c

…vate clusters. Minor wording changes.

Update for split out node pools

f983b35

Add comment w/ TODO to update regional private cluster docs.

9b833c0

Fix HTML comment

cac6051

Co-Authored-By: rileykarson <[email protected]>

shift cluster tests into stages

3299135

add working circleci config

swap order

462bb06

Merge pull request #12 from gruntwork-io/better-cluster-tests

8103773

[WIP] Better cluster tests

robmorgan merged commit 83d6816 into master Jan 31, 2019

robmorgan changed the title ~~[WIP] GKE Module Design~~ GKE Module Design Jan 31, 2019

yorinasub17 pushed a commit that referenced this pull request Mar 24, 2020

Merge pull request #1 from gruntwork-io/master

edf8ea2

merge ci changes from upstream

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GKE Module Design #1

GKE Module Design #1

robmorgan commented Jan 11, 2019 •

edited

Loading

brikis98 commented Jan 11, 2019

robmorgan commented Jan 11, 2019

rileykarson commented Jan 14, 2019

rileykarson Jan 15, 2019

rileykarson Jan 15, 2019

rileykarson left a comment

rileykarson Jan 18, 2019

robmorgan Jan 24, 2019

rileykarson Jan 18, 2019

rileykarson Jan 18, 2019

rileykarson Jan 18, 2019

rileykarson Jan 18, 2019

robmorgan Jan 24, 2019

robmorgan commented Jan 31, 2019

+                }
+              }
+              // TODO

GKE Module Design #1

GKE Module Design #1

Conversation

robmorgan commented Jan 11, 2019 • edited Loading

Proposed Structure

Design Decisions

Open Questions

Notes

Tasks

Terratest

brikis98 commented Jan 11, 2019

robmorgan commented Jan 11, 2019

rileykarson commented Jan 14, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rileykarson left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

robmorgan commented Jan 31, 2019

robmorgan commented Jan 11, 2019 •

edited

Loading