Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add module for EMR container #33

Merged
merged 17 commits into from
Dec 10, 2022
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ module "eks_ack_addons" {
enable_s3 = true
enable_rds = true
enable_amp = true
enable_emrcontainers = true

tags = {
Environment = "dev"
Expand Down Expand Up @@ -51,13 +52,15 @@ Examples codified under the [`examples`](https://github.com/aws-ia/terraform-aws
| <a name="module_amp"></a> [amp](#module\_amp) | github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons/helm-addon | v4.12.2 |
| <a name="module_api_gatewayv2"></a> [api\_gatewayv2](#module\_api\_gatewayv2) | github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons/helm-addon | v4.12.2 |
| <a name="module_dynamodb"></a> [dynamodb](#module\_dynamodb) | github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons/helm-addon | v4.12.2 |
| <a name="module_emrcontainers"></a> [emrcontainers](#module\_emrcontainers) | github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons/helm-addon | v4.18.0 |
| <a name="module_rds"></a> [rds](#module\_rds) | github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons/helm-addon | v4.12.2 |
| <a name="module_s3"></a> [s3](#module\_s3) | github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons/helm-addon | v4.12.2 |

## Resources

| Name | Type |
|------|------|
| [aws_iam_policy.emrcontainers](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [time_sleep.dataplane](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) | resource |
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
| [aws_eks_cluster.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster) | data source |
Expand All @@ -67,6 +70,7 @@ Examples codified under the [`examples`](https://github.com/aws-ia/terraform-aws
| [aws_iam_policy.dynamodb](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy) | data source |
| [aws_iam_policy.rds](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy) | data source |
| [aws_iam_policy.s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy) | data source |
| [aws_iam_policy_document.emrcontainers](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_partition.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) | data source |
| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |

Expand All @@ -79,9 +83,11 @@ Examples codified under the [`examples`](https://github.com/aws-ia/terraform-aws
| <a name="input_cluster_id"></a> [cluster\_id](#input\_cluster\_id) | EKS Cluster Id | `string` | n/a | yes |
| <a name="input_data_plane_wait_arn"></a> [data\_plane\_wait\_arn](#input\_data\_plane\_wait\_arn) | Addon deployment will not proceed until this value is known. Set to node group/Fargate profile ARN to wait for data plane to be ready before provisioning addons | `string` | `""` | no |
| <a name="input_dynamodb_helm_config"></a> [dynamodb\_helm\_config](#input\_dynamodb\_helm\_config) | ACK dynamodb Helm Chart config | `any` | `{}` | no |
| <a name="input_emrcontainers_helm_config"></a> [emrcontainers\_helm\_config](#input\_emrcontainers\_helm\_config) | ACK EMR container Helm Chart config | `any` | `{}` | no |
| <a name="input_enable_amp"></a> [enable\_amp](#input\_enable\_amp) | Enable ACK amp add-on | `bool` | `false` | no |
| <a name="input_enable_api_gatewayv2"></a> [enable\_api\_gatewayv2](#input\_enable\_api\_gatewayv2) | Enable ACK API gateway v2 add-on | `bool` | `false` | no |
| <a name="input_enable_dynamodb"></a> [enable\_dynamodb](#input\_enable\_dynamodb) | Enable ACK dynamodb add-on | `bool` | `false` | no |
| <a name="input_enable_emrcontainers"></a> [enable\_emrcontainers](#input\_enable\_emrcontainers) | Enable ACK EMR container add-on | `bool` | `false` | no |
| <a name="input_enable_rds"></a> [enable\_rds](#input\_enable\_rds) | Enable ACK rds add-on | `bool` | `false` | no |
| <a name="input_enable_s3"></a> [enable\_s3](#input\_enable\_s3) | Enable ACK s3 add-on | `bool` | `false` | no |
| <a name="input_irsa_iam_permissions_boundary"></a> [irsa\_iam\_permissions\_boundary](#input\_irsa\_iam\_permissions\_boundary) | IAM permissions boundary for IRSA roles | `string` | `""` | no |
Expand Down
2 changes: 1 addition & 1 deletion examples/complete/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ To provision this example:

```sh
terraform init
terraform apply
terraform apply -var aws_region=<aws_region> # defaults to us-west-2
```

Enter `yes` at command prompt to apply
Expand Down
5 changes: 4 additions & 1 deletion examples/complete/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ data "aws_eks_cluster_auth" "this" {
name = module.eks_blueprints.eks_cluster_id
}

data "aws_ecr_authorization_token" "token" {}

data "aws_availability_zones" "available" {}
data "aws_caller_identity" "current" {}
data "aws_partition" "current" {}
Expand Down Expand Up @@ -101,7 +103,8 @@ module "eks_ack_addons" {
enable_s3 = true
enable_rds = true
enable_amp = true

enable_emrcontainers = true

tags = local.tags
}

Expand Down
142 changes: 142 additions & 0 deletions main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -372,3 +372,145 @@ data "aws_iam_policy" "amp" {

name = "AmazonPrometheusFullAccess"
}

################################################################################
# EMR Containers
################################################################################

locals {
emr_name = "ack-emrcontainers"
}

module "emrcontainers" {
source = "github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons/helm-addon?ref=v4.18.0"

count = var.enable_emrcontainers ? 1 : 0

helm_config = merge(
{
name = local.emr_name
chart = "emrcontainers-chart"
repository = "oci://public.ecr.aws/aws-controllers-k8s"
version = "v0-stable"
namespace = local.emr_name
description = "Helm Charts for the emrcontainers controller for AWS Controllers for Kubernetes (ACK)"
values = [
# shortens pod name from `ack-emrcontainers-emrcontainers-chart-xxxxxxxxxxxxx` to `ack-emrcontainers-xxxxxxxxxxxxx`
<<-EOT
nameOverride: ack-emrcontainers
EOT
]
},
var.emrcontainers_helm_config
)

set_values = [
{
name = "serviceAccount.name"
value = local.emr_name
},
{
name = "serviceAccount.create"
value = false
},
{
name = "aws.region"
value = local.region
}
]

irsa_config = {
create_kubernetes_namespace = true
kubernetes_namespace = try(var.emrcontainers_helm_config.namespace, local.emr_name)

create_kubernetes_service_account = true
kubernetes_service_account = local.emr_name

irsa_iam_policies = [aws_iam_policy.emrcontainers[0].arn]
}

addon_context = local.addon_context
}

resource "aws_iam_policy" "emrcontainers" {
count = var.enable_emrcontainers ? 1 : 0

name = format("%s-%s", local.emr_name, "controller-iam-policies")
description = "IAM policy for EMRcontainers controller"
path = "/"
policy = data.aws_iam_policy_document.emrcontainers.json
}

// inline policy providered by ack https://raw.githubusercontent.com/aws-controllers-k8s/emrcontainers-controller/main/config/iam/recommended-inline-policy
data "aws_iam_policy_document" "emrcontainers" {
statement {
effect = "Allow"
actions = [
"iam:CreateServiceLinkedRole"
]
resources = ["*"]

condition {
test = "StringLike"
variable = "iam:AWSServiceName"
values = ["emr-containers.amazonaws.com"]
}
}

statement {
effect = "Allow"
actions = [
"emr-containers:CreateVirtualCluster",
"emr-containers:ListVirtualClusters",
"emr-containers:DescribeVirtualCluster",
"emr-containers:DeleteVirtualCluster"
]
resources = ["*"]
}

statement {
effect = "Allow"
actions = [
"emr-containers:StartJobRun",
"emr-containers:ListJobRuns",
"emr-containers:DescribeJobRun",
"emr-containers:CancelJobRun"
]

resources = ["*"]
}

statement {
effect = "Allow"
actions = [
"emr-containers:DescribeJobRun",
"emr-containers:TagResource",
"elasticmapreduce:CreatePersistentAppUI",
"elasticmapreduce:DescribePersistentAppUI",
"elasticmapreduce:GetPersistentAppUIPresignedURL"
]

resources = ["*"]
}

statement {
effect = "Allow"
actions = [
"s3:GetObject",
"s3:ListBucket"
]

resources = ["*"]
}

statement {
effect = "Allow"
actions = [
"logs:Get*",
"logs:DescribeLogGroups",
"logs:DescribeLogStreams"
]
resources = ["*"]
}

}
16 changes: 16 additions & 0 deletions variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,22 @@ variable "rds_helm_config" {
default = {}
}

################################################################################
# EMR Containers
################################################################################

variable "enable_emrcontainers" {
description = "Enable ACK EMR container add-on"
type = bool
default = false
}

variable "emrcontainers_helm_config" {
description = "ACK EMR container Helm Chart config"
type = any
default = {}
}

################################################################################
# AMP
################################################################################
Expand Down