Skip to content
This repository has been archived by the owner on Mar 29, 2023. It is now read-only.

GKE Module Design #1

Merged
merged 46 commits into from
Jan 31, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
d2fa373
remove legacy files
robmorgan Jan 11, 2019
f8caca8
ignore terratest output
robmorgan Jan 15, 2019
1deea1a
Merge branch 'master' into module-design
robmorgan Jan 15, 2019
3e8a0d9
expand readme
robmorgan Jan 15, 2019
f081fe1
add initial module design and planned examples
robmorgan Jan 15, 2019
bcd5288
Merge branch 'master' into module-design
robmorgan Jan 15, 2019
80e7533
Update examples/gke-regional-public-cluster/variables.tf
rileykarson Jan 16, 2019
7c48e39
clarify minimum terraform version
robmorgan Jan 16, 2019
14435b9
project_id to project
robmorgan Jan 16, 2019
a5405ec
change to google-beta provider. comment out network stuff
robmorgan Jan 16, 2019
62ca09b
start removing zonal support from module
robmorgan Jan 16, 2019
a83fa51
Add an initial node count, explain why it's necessary.
rileykarson Jan 16, 2019
a410079
Remove extraneous line.
rileykarson Jan 16, 2019
06e7633
Add an initial node count, explain why it's necessary.
rileykarson Jan 16, 2019
25452bf
Remove extraneous line.
rileykarson Jan 16, 2019
8d817cc
wip
rileykarson Jan 16, 2019
6718373
Merge pull request #7 from gruntwork-io/module-design-patch-2
robmorgan Jan 17, 2019
5a58c56
Merge pull request #6 from gruntwork-io/module-design-patch
robmorgan Jan 17, 2019
61eb7c5
break the node pool out of the gke-cluster module and move it to the …
robmorgan Jan 17, 2019
eed633b
only use one of auto_upgrade or version
robmorgan Jan 17, 2019
627d58a
Merge branch 'module-design' into module-design-test
robmorgan Jan 17, 2019
f4ebb09
Merge pull request #8 from gruntwork-io/module-design-test
robmorgan Jan 17, 2019
f4c7520
get project from env vars and test random region instead
robmorgan Jan 17, 2019
981452c
Fill out READMEs for GKE Cluster, regional public + private clusters
rileykarson Jan 15, 2019
8d0ac51
Cross-link between private, public examples. Explicitly recommend pri…
rileykarson Jan 16, 2019
f8f3e9b
Update for split out node pools
rileykarson Jan 17, 2019
aa783e3
Configuration fixes for tests
rileykarson Jan 17, 2019
a895a81
Merge pull request #9 from gruntwork-io/module-design-test-fix
robmorgan Jan 17, 2019
ac3ed78
add circleci config
robmorgan Jan 18, 2019
4041c96
clean up module definition
robmorgan Jan 18, 2019
3f8c90c
output the cluster endpoint
robmorgan Jan 18, 2019
b9be14f
Add comment w/ TODO to update regional private cluster docs.
rileykarson Jan 22, 2019
263bdda
Fix HTML comment
yorinasub17 Jan 22, 2019
8c5cbb1
Update modules/gke-cluster/variables.tf
rileykarson Jan 24, 2019
7746681
Clean up the GKE cluster master module.
rileykarson Jan 24, 2019
9fe4b2f
Clean up gke-regional-public-cluster example
rileykarson Jan 24, 2019
d1fef5c
Merge pull request #11 from gruntwork-io/module-design-cleanup
robmorgan Jan 25, 2019
8824e42
Merge pull request #5 from gruntwork-io/module-design-rdd
robmorgan Jan 25, 2019
12e4210
Fill out READMEs for GKE Cluster, regional public + private clusters
rileykarson Jan 15, 2019
34e500c
Cross-link between private, public examples. Explicitly recommend pri…
rileykarson Jan 16, 2019
f983b35
Update for split out node pools
rileykarson Jan 17, 2019
9b833c0
Add comment w/ TODO to update regional private cluster docs.
rileykarson Jan 22, 2019
cac6051
Fix HTML comment
yorinasub17 Jan 22, 2019
3299135
shift cluster tests into stages
robmorgan Jan 25, 2019
462bb06
swap order
robmorgan Jan 31, 2019
8103773
Merge pull request #12 from gruntwork-io/better-cluster-tests
robmorgan Jan 31, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions examples/gke-regional-private-cluster/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
TODO
13 changes: 13 additions & 0 deletions examples/gke-regional-public-cluster/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# GKE Regional Public Cluster

This example creates a Regional Public GKE Cluster with some sane defaults.

## How do you run these examples?

1. Install [Terraform](https://www.terraform.io/).
rileykarson marked this conversation as resolved.
Show resolved Hide resolved
rileykarson marked this conversation as resolved.
Show resolved Hide resolved
1. Make sure you have Python installed (version 2.x) and in your `PATH`.
1. Open `variables.tf`, set the environment variables specified at the top of the file, and fill in any other variables that
don't have a default.
1. Run `terraform get`.
1. Run `terraform plan`.
1. If the plan looks good, run `terraform apply`.
63 changes: 63 additions & 0 deletions examples/gke-regional-public-cluster/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# ---------------------------------------------------------------------------------------------------------------------
# DEPLOY A GKE REGIONAL PUBLIC CLUSTER IN GOOGLE CLOUD
# This is an example of how to use the gke-cluster module to deploy a regional public Kubernetes cluster in GCP with a
# Load Balancer in front of it.
# ---------------------------------------------------------------------------------------------------------------------

provider "google" {
project = "${var.project_id}"
region = "${var.region}"
}

# Use Terraform 0.10.x so that we can take advantage of Terraform GCP functionality as a separate provider via
rileykarson marked this conversation as resolved.
Show resolved Hide resolved
# https://github.com/terraform-providers/terraform-provider-google
terraform {
required_version = ">= 0.10.3"
}

module "gke_cluster" {
# When using these modules in your own templates, you will need to use a Git URL with a ref attribute that pins you
# to a specific version of the modules, such as the following example:
# source = "git::[email protected]:gruntwork-io/gke-cluster.git//modules/gke-cluster?ref=v0.0.1"
source = "../../modules/gke-cluster"

project_id = "${var.project_id}"
region = "${var.region}"
name = "example-cluster"

network = "${google_compute_network.main.name}"
subnetwork = "${google_compute_subnetwork.main.name}"
ip_range_pods = "${var.ip_range_pods}"
ip_range_services = "${var.ip_range_services}"
service_account = "${var.compute_engine_service_account}"
}

# Network

resource "random_string" "suffix" {
length = 4
special = false
upper = false
}

resource "google_compute_network" "main" {
name = "cft-gke-test-${random_string.suffix.result}"
auto_create_subnetworks = "false"
}

resource "google_compute_subnetwork" "main" {
name = "cft-gke-test-${random_string.suffix.result}"
ip_cidr_range = "10.0.0.0/17"
region = "${var.region}"
network = "${google_compute_network.main.self_link}"

secondary_ip_range {
range_name = "cft-gke-test-pods-${random_string.suffix.result}"
ip_cidr_range = "192.168.0.0/18"
}

secondary_ip_range {
range_name = "cft-gke-test-services-${random_string.suffix.result}"
ip_cidr_range = "192.168.64.0/18"
}
}
Empty file.
29 changes: 29 additions & 0 deletions examples/gke-regional-public-cluster/variables.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# ---------------------------------------------------------------------------------------------------------------------
# REQUIRED PARAMETERS
# These parameters must be supplied when consuming this module.
# ---------------------------------------------------------------------------------------------------------------------

variable "project_id" {
robmorgan marked this conversation as resolved.
Show resolved Hide resolved
description = "The name of the GCP Project where all resources will be launched."
}

variable "region" {
description = "The Region in which all GCP resources will be launched."
}

# ---------------------------------------------------------------------------------------------------------------------
# OPTIONAL PARAMETERS
# These parameters have reasonable defaults.
# ---------------------------------------------------------------------------------------------------------------------

variable "ip_range_pods" {
description = "The secondary ip range to use for pods"
}

variable "ip_range_services" {
description = "The secondary ip range to use for pods"
}

variable "compute_engine_service_account" {
description = "Service account to associate to the nodes in the cluster"
}
1 change: 1 addition & 0 deletions examples/gke-zonal-private-cluster/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
TODO
1 change: 1 addition & 0 deletions examples/gke-zonal-public-cluster/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
TODO
149 changes: 149 additions & 0 deletions modules/gke-cluster/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
resource "google_container_cluster" "cluster" {
name = "${var.name}"
description = "${var.description}"
project = "${var.project_id}"

region = "${var.region}"
additional_zones = ["${coalescelist(compact(var.zones), sort(random_shuffle.available_zones.result))}"]

network = "${replace(data.google_compute_network.gke_network.self_link, "https://www.googleapis.com/compute/v1/", "")}"
subnetwork = "${replace(data.google_compute_subnetwork.gke_subnetwork.self_link, "https://www.googleapis.com/compute/v1/", "")}"
min_master_version = "${local.kubernetes_version}"

logging_service = "${var.logging_service}"
monitoring_service = "${var.monitoring_service}"

master_authorized_networks_config = "${var.master_authorized_networks_config}"

addons_config {
http_load_balancing {
disabled = "${var.http_load_balancing ? 0 : 1}"
}

horizontal_pod_autoscaling {
disabled = "${var.horizontal_pod_autoscaling ? 0 : 1}"
}

kubernetes_dashboard {
disabled = "${var.kubernetes_dashboard ? 0 : 1}"
}

network_policy_config {
disabled = "${var.network_policy ? 0 : 1}"
}
}

ip_allocation_policy {
cluster_secondary_range_name = "${var.ip_range_pods}"
services_secondary_range_name = "${var.ip_range_services}"
}

maintenance_policy {
daily_maintenance_window {
start_time = "${var.maintenance_start_time}"
}
}

lifecycle {
ignore_changes = ["node_pool"]
}

# Version 2.0.0 will set the default timeouts to these values.
timeouts {
create = "30m"
update = "30m"
delete = "30m"
}

remove_default_node_pool = true
}

// Node Pool Resource
resource "google_container_node_pool" "pools" {
count = "${length(var.node_pools)}"
name = "${lookup(var.node_pools[count.index], "name")}"
project = "${var.project_id}"
region = "${var.region}"
cluster = "${var.name}"
version = "${lookup(var.node_pools[count.index], "auto_upgrade", false) ? "" : lookup(var.node_pools[count.index], "version", local.node_version)}"
initial_node_count = "${lookup(var.node_pools[count.index], "initial_node_count", lookup(var.node_pools[count.index], "min_count", 1))}"

autoscaling {
min_node_count = "${lookup(var.node_pools[count.index], "min_count", 1)}"
max_node_count = "${lookup(var.node_pools[count.index], "max_count", 100)}"
}

management {
auto_repair = "${lookup(var.node_pools[count.index], "auto_repair", true)}"
auto_upgrade = "${lookup(var.node_pools[count.index], "auto_upgrade", true)}"
}

node_config {
image_type = "${lookup(var.node_pools[count.index], "image_type", "COS")}"
machine_type = "${lookup(var.node_pools[count.index], "machine_type", "n1-standard-2")}"
labels = "${merge(map("cluster_name", var.name), map("node_pool", lookup(var.node_pools[count.index], "name")), var.node_pools_labels["all"], var.node_pools_labels[lookup(var.node_pools[count.index], "name")])}"
metadata = "${merge(map("cluster_name", var.name), map("node_pool", lookup(var.node_pools[count.index], "name")), var.node_pools_metadata["all"], var.node_pools_metadata[lookup(var.node_pools[count.index], "name")])}"
taint = "${concat(var.node_pools_taints["all"], var.node_pools_taints[lookup(var.node_pools[count.index], "name")])}"
tags = ["${concat(list("gke-${var.name}"), list("gke-${var.name}-${lookup(var.node_pools[count.index], "name")}"), var.node_pools_tags["all"], var.node_pools_tags[lookup(var.node_pools[count.index], "name")])}"]

disk_size_gb = "${lookup(var.node_pools[count.index], "disk_size_gb", 100)}"
disk_type = "${lookup(var.node_pools[count.index], "disk_type", "pd-standard")}"
service_account = "${lookup(var.node_pools[count.index], "service_account", var.service_account)}"
preemptible = "${lookup(var.node_pools[count.index], "preemptible", false)}"

oauth_scopes = [
"https://www.googleapis.com/auth/cloud-platform",
]
}

lifecycle {
ignore_changes = ["initial_node_count"]
}

timeouts {
create = "30m"
update = "30m"
delete = "30m"
}

depends_on = ["google_container_cluster.cluster"]
}

// TODO
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This datasource is present at the bottom of the file.

// Add Data Source to get the latest k8s version
// Use this is k8s version is not set.

locals {
kubernetes_version = "${var.kubernetes_version != "latest" ? var.kubernetes_version : data.google_container_engine_versions.region.latest_node_version}"
node_version = "${var.node_version != "" ? var.node_version : local.kubernetes_version}"
network_project_id = "${var.network_project_id != "" ? var.network_project_id : var.project_id}"
}

data "google_compute_zones" "available" {
project = "${var.project_id}"
region = "${var.region}"
}

data "google_compute_network" "gke_network" {
name = "${var.network}"
project = "${local.network_project_id}"
}

data "google_compute_subnetwork" "gke_subnetwork" {
name = "${var.subnetwork}"
region = "${var.region}"
project = "${local.network_project_id}"
}

resource "random_shuffle" "available_zones" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for context, the reason this / the corresponding datasource are useful (https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/blob/3f7527e583ffa07e6a06250844e07c38556a4488/cluster_regional.tf#L29) is because it lets you explicitly define which zones in the region your regional / multi-zonal (afaik regional supersedes multi-zonal) cluster resides in.

input = ["${data.google_compute_zones.available.names}"]
result_count = 3
}

/******************************************
Get available container engine versions
*****************************************/
data "google_container_engine_versions" "region" {
zone = "${data.google_compute_zones.available.names[0]}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A regional cluster doesn't necessarily support the same version(s) as any/all of its corresponding zones. We'll need to supply the region parameter when working with regional clusters.

project = "${var.project_id}"
}
91 changes: 91 additions & 0 deletions modules/gke-cluster/outputs.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
output "name" {
description = "Cluster name"
value = "${local.cluster_name}"
}

output "type" {
description = "Cluster type (regional / zonal)"
value = "${local.cluster_type}"
}

output "location" {
description = "Cluster location (region if regional cluster, zone if zonal cluster)"
value = "${local.cluster_location}"
}

output "region" {
description = "Cluster region"
value = "${local.cluster_region}"
}

output "zones" {
description = "List of zones in which the cluster resides"
value = "${local.cluster_zones}"
}

output "endpoint" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add the outputs from https://github.com/gruntwork-io/terraform-google-gke/pull/10/files#diff-c49d0ae81c71a97455954c7c48bc0eca without the base64decode step? I think they'll be necessary when integrating Tiller.

sensitive = true
description = "Cluster endpoint"
value = "${local.cluster_endpoint}"
}

output "min_master_version" {
description = "Minimum master kubernetes version"
value = "${local.cluster_min_master_version}"
}

output "logging_service" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as dashboard - these are always user supplied (logging, monitoring).

description = "Logging service used"
value = "${local.cluster_logging_service}"
}

output "monitoring_service" {
description = "Monitoring service used"
value = "${local.cluster_monitoring_service}"
}

output "master_authorized_networks_config" {
description = "Networks from which access to master is permitted"
value = "${var.master_authorized_networks_config}"
}

output "master_version" {
description = "Current master kubernetes version"
value = "${local.cluster_master_version}"
}

output "ca_certificate" {
sensitive = true
description = "Cluster ca certificate (base64 encoded)"
value = "${local.cluster_ca_certificate}"
}

output "network_policy_enabled" {
description = "Whether network policy enabled"
value = "${local.cluster_network_policy_enabled}"
}

output "http_load_balancing_enabled" {
description = "Whether http load balancing enabled"
value = "${local.cluster_http_load_balancing_enabled}"
}

output "horizontal_pod_autoscaling_enabled" {
description = "Whether horizontal pod autoscaling enabled"
value = "${local.cluster_horizontal_pod_autoscaling_enabled}"
}

output "kubernetes_dashboard_enabled" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like an output we don't need, since the user will have supplied this variable anyways. Or is that standard convention?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah can probably go, I must have copied it from the other repos.

description = "Whether kubernetes dashboard enabled"
value = "${local.cluster_kubernetes_dashboard_enabled}"
}

output "node_pools_names" {
description = "List of node pools names"
value = "${local.cluster_node_pools_names}"
}

output "node_pools_versions" {
description = "List of node pools versions"
value = "${local.cluster_node_pools_versions}"
}
Loading