Tamedia Kubernetes as a Service (KaaS) Terraform Module (Alpha)

Opinionated batteries included Terraform module to deploy Kubernetes in AWS. Includes:

Managed Addons:

• EBS CSI
• VPC CNI
• CoreDNS
• KubeProxy

Core components (installed by default):

• Karpenter
• Metrics Server
• AWS Load Balancer Controller
• External DNS
• External Secrets
• Prometheus Operator
• Grafana
• Fluent Operator
• Fluentbit for Fargate
• Reloader

Additional components (optional):

• Cert Manager
• Ingress Nginx
• Downscaler

Integrations (optional):

• Okta
• PagerDuty
• Slack

Requirements

The module needs some resources to be deployed in order to operate correctly:

IAM service-linked roles

• AWSServiceRoleForEC2Spot
• AWSServiceRoleForEC2SpotFleet

Usage

module "k8s_platform" {
  source = "tx-pts-dai/kubernetes-platform/aws"
  # Pin this module to a specific version to avoid breaking changes
  # version = "0.0.0"

  name = "example-platform"

  vpc = {
    enabled = true
  }

  tags = {
    Environment = "sandbox"
    GithubRepo  = "terraform-aws-kubernetes-platform"
  }
}

See the Examples below for more use cases

Explanation and description of interesting use-cases

Why this module?

• To provide an AWS account with a K8s cluster with batteries included so that you can start deploying your workloads on a well-built foundation
• To encourage standardization and common practices
• To ease maintenance

Reloader

The Stakater Reloader is a Kubernetes controller that automatically watches for changes in ConfigMaps and Secrets and triggers rolling restarts of the associated deployments, statefulsets, or daemonsets when these configurations are updated. This functionality ensures that applications deployed within a Kubernetes cluster always reflect the latest configuration without manual intervention.

When an application relies on configuration data or sensitive information stored in ConfigMaps or Secrets, and these resources are modified, Reloader automates the process of applying these changes by updating the relevant pods. Without Reloader, such changes would require a manual pod restart or redeployment to take effect.

Reloader is deployed by default on the cluster but is used as on demand via annotations.

Considering this kubernetes deployment and the required annotation:

kind: Deployment
metadata:
  name: foo
  annotations:
    reloader.stakater.com/auto: "true"
spec:
  template:
    metadata:

Reloader will now watch for updates and manage rolling restats of pods for this specific deployment.

Examples

• Complete - Includes creation of VPC, k8s cluster, addons and all the optional features.
• Simple - Simplest EKS deployment with default VPC, addons, ... creation
• Lacework - EKS deployment with Lacework integration
• Datadog - EKS deployment with Datadog Operator integration

Cleanup example deployments

Destroy Workflow - This manual workflow destroys deployed example deployments by selection the branch and the example to destroy.

Contributing

< issues and contribution guidelines for public modules >

Pre-Commit

Installation: install pre-commit and execute pre-commit install. This will generate pre-commit hooks according to the config in .pre-commit-config.yaml

Before submitting a PR be sure to have used the pre-commit hooks or run: pre-commit run -a

The pre-commit command will run:

• Terraform fmt
• Terraform validate
• Terraform docs
• Terraform validate with tflint
• check for merge conflicts
• fix end of files

as described in the .pre-commit-config.yaml file

Requirements

Name	Version
terraform	>= 1.6.0
aws	>= 5.42.0
helm	>= 2.12
kubectl	>= 2.0.2
kubernetes	>= 2.27
time	>= 0.11

Providers

Name	Version
aws	>= 5.42.0
kubernetes	>= 2.27
time	>= 0.11

Modules

Name	Source	Version
acm	terraform-aws-modules/acm/aws	5.1.1
addons	aws-ia/eks-blueprints-addons/aws	1.19.0
amp	terraform-aws-modules/managed-service-prometheus/aws	3.0.0
cluster_secret_store	./modules/addon	n/a
downscaler	tx-pts-dai/downscaler/kubernetes	0.3.1
ebs_csi_driver_irsa	terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks	5.52.1
eks	terraform-aws-modules/eks/aws	20.31.6
fluent_operator	./modules/addon	n/a
grafana	./modules/addon	n/a
karpenter	terraform-aws-modules/eks/aws//modules/karpenter	20.31.6
karpenter_crds	./modules/addon	n/a
karpenter_release	./modules/addon	n/a
karpenter_security_group	./modules/security-group	n/a
network	./modules/network	n/a
okta_secrets	./modules/addon	n/a
pagerduty_secrets	./modules/addon	n/a
prometheus_irsa	terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks	5.52.1
prometheus_operator_crds	./modules/addon	n/a
prometheus_stack	./modules/addon	n/a
reloader	./modules/addon	n/a
slack_secrets	./modules/addon	n/a
ssm	./modules/ssm	n/a
vpc_cni_irsa	terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks	5.52.1

Resources

Name	Type
aws_cloudwatch_log_group.fluentbit	resource
aws_iam_policy.fluentbit	resource
aws_route_table_association.karpenter	resource
aws_security_group_rule.eks_control_plane_ingress	resource
aws_subnet.karpenter	resource
kubernetes_annotations.monitoring	resource
time_sleep.wait_on_destroy	resource
time_static.timestamp_id	resource
aws_availability_zones.available	data source
aws_iam_policy_document.fluentbit	data source
aws_iam_roles.iam_cluster_admins	data source
aws_iam_roles.sso	data source
aws_region.current	data source
aws_route53_zone.base_domain_zone	data source
aws_route_tables.private_route_tables	data source

Inputs

Name	Description	Type	Default	Required
acm_certificate	ACM certificate configuration. If wildcard_certificates is true, all domains will include a wildcard prefix.	object({ domain_name = optional(string) # Overrides base_domain subject_alternative_names = optional(list(string), []) wildcard_certificates = optional(bool, false) wait_for_validation = optional(bool, false) })	{}	no
aws_load_balancer_controller	AWS Load Balancer Controller configurations	any	{}	no
base_domain	Base domain for the platform, used for ingress and ACM certificates	string	"test"	no
cert_manager	Cert Manager configurations	any	{}	no
cluster_admins	Map of IAM roles to add as cluster admins. Only exact matching role names are returned	map(object({ role_name = string kubernetes_groups = optional(list(string)) }))	{}	no
create_addons	Create the platform addons. if set to false, no addons will be created	bool	true	no
downscaler	Downscaler configurations	any	{}	no
eks	Map of EKS configurations	any	{}	no
enable_acm_certificate	Enable ACM certificate	bool	false	no
enable_amp	Enable AWS Managed Prometheus	bool	false	no
enable_aws_load_balancer_controller	Enable AWS Load Balancer Controller	bool	true	no
enable_cert_manager	Enable Cert Manager	bool	false	no
enable_downscaler	Enable Downscaler	bool	false	no
enable_external_dns	Enable External DNS	bool	true	no
enable_external_secrets	Enable External Secrets	bool	true	no
enable_fargate_fluentbit	Enable Fargate Fluentbit	bool	true	no
enable_fluent_operator	Enable fluent operator	bool	true	no
enable_grafana	Enable Grafana	bool	true	no
enable_ingress_nginx	Enable Ingress Nginx	bool	false	no
enable_metrics_server	Enable Metrics Server	bool	true	no
enable_okta	Enable Okta integration	bool	false	no
enable_pagerduty	Enable PagerDuty integration	bool	false	no
enable_prometheus_stack	Enable Prometheus stack	bool	true	no
enable_reloader	Enable Reloader	bool	true	no
enable_slack	Enable Slack integration	bool	false	no
external_dns	External DNS configurations	any	{}	no
external_secrets	External Secrets configurations	any	{}	no
fargate_fluentbit	Fargate Fluentbit configurations	any	{}	no
fluent_cloudwatch_retention_in_days	Number of days to keep logs in cloudwatch	string	"7"	no
fluent_log_annotation	Pod Annotation required to enable fluent bit logging. Setting name to empty string will disable annotation requirement.	object({ name = optional(string, "fluentbit.io/include") value = optional(string, "true") })	{}	no
fluent_operator	Fluent configurations	any	{}	no
grafana	Grafana configurations, used to override default configurations	any	{}	no
ingress_nginx	Ingress Nginx configurations	any	{}	no
karpenter	Karpenter configurations	any	{}	no
metadata	Metadata for the platform	object({ environment = optional(string, "") team = optional(string, "") })	{}	no
metrics_server	Metrics Server configurations	any	{}	no
name	The name of the platform, a timestamp will be appended to this name to make the stack_name. If not provided, the name of the directory will be used.	string	""	no
okta	Okta configurations	object({ base_url = optional(string, "") secrets_manager_secret_name = optional(string, "") kubernetes_secret_name = optional(string, "okta") })	{}	no
pagerduty	PagerDuty configurations	object({ secrets_manager_secret_name = optional(string, "") kubernetes_secret_name = optional(string, "pagerduty") })	{}	no
prometheus_stack	Prometheus stack configurations	any	{}	no
reloader	Reloader configurations	any	{}	no
slack	Slack configurations	object({ secrets_manager_secret_name = optional(string, "") kubernetes_secret_name = optional(string, "slack") })	{}	no
tags	Default tags to apply to all resources	map(string)	{}	no
vpc	Map of VPC configurations	any	{}	no

Outputs

Name	Description
eks	Map of attributes for the EKS cluster
network	Map of attributes for the EKS cluster

Authors

Module is maintained by Alfredo Gottardo, David Beauvererd, Davide Cammarata, Demetrio Carrara, Roland Bapst and Samuel Wibrow

License

Apache 2 Licensed. See LICENSE for full details.