Network rework

AWSCloudSpec.md

@@ -67,6 +67,19 @@
 
 ## <a name="ServiceVirtualPrivateCloud"></a> ![Amazon Virtual Private Cloud](https://raw.githubusercontent.com/awslabs/aws-icons-for-plantuml/main/dist/NetworkingContentDelivery/VirtualPrivateCloud.png) Amazon Virtual Private Cloud
 
+### VPC requirements
+
+| Requirement | Description |  Default value | Mandatory? |
+| ----------- | ----------- | -------------- | ---------- |
+| IPv4 CIDR block | Network size ie. number of available IPs in VPC | 10.1.0.0/18 | yes |
+| Availability zones | How many AZs to spread VPC across | 3 (at least 2 for high availability) | yes |
+| Private subnets | How many private subnets to create | 3 (at least 2 for high availability; one per each AZ) | yes |
+| Public subnets | How many public subnets to create | 3 (at least 2 for high availability; one per each AZ) | yes |
+| NAT gateway | Enable/disable NAT in VPC | enable | yes |
+| Single NAT gateway | Controls how many NAT gateways/Elastic IPs to provision | enable | no |
+| Internet gateway | Enable/disable IGW in VPC | enable | yes |
+| DNS hostnames | Determines whether the VPC supports assigning public DNS hostnames to instances with public IP addresses. | enable | yes |
+
 ### <a name="ResourceInternetGateway"></a>![Internet gateway](https://raw.githubusercontent.com/awslabs/aws-icons-for-plantuml/main/dist/NetworkingContentDelivery/VPCInternetGateway.png) Internet gateway
 | Description |
 | ----------- |
@@ -193,23 +206,44 @@
 
 
 ### <a name="ResourceSubnet"></a>Subnet
-| Name | 
-| ---- | 
-| Public subnet in region 1 zone a | 
-| Public subnet in region 1 zone b | 
+| Name |
+| ---- |
+| Public subnet in region 1 zone a |
+| Public subnet in region 1 zone b |
 | Public subnet in region 1 zone c |
-| Private subnet in region 1 zone a | 
-| Private subnet in region 1 zone b | 
-| Private subnet in region 1 zone c | 
-| Database subnet in region 1 zone a | 
-| Database subnet in region 1 zone b | 
-| Database subnet in region 1 zone c | 
+| Private subnet in region 1 zone a |
+| Private subnet in region 1 zone b |
+| Private subnet in region 1 zone c |
 
+### Private subnets requirements
 
-### <a name="ResourceVirtualPrivateCloud"></a>Virtual Private Cloud
-| Name | Mandatory |
-| ---- | ---------- |
-| Virtual network for SIMPHERA. | Yes |
+| Requirement | Description |  Default value | Mandatory? |
+| ----------- | ----------- | -------------- | ---------- |
+| IPv4 CIDR blocks | Network size, ie number of available IPs per private subnet | 10.1.0.0/22 <br /> 10.1.4.0/22 <br /> 10.1.8.0/22 | yes |
+| Tags | Metadata for organizing your AWS resources | "kubernetes.io/cluster/\<cluster name>" = "shared" <br /> "kubernetes.io/role/elb" = "1" <br /> "purpose" = "private" | yes |
+| Network Access Lists | Allows or denies specific inbound or outbound traffic at the subnet level | Allow all inbound/outbound | yes |
+
+### Public subnets requirements
+
+| Requirement | Description |  Default value | Mandatory? |
+| ----------- | ----------- | -------------- | ---------- |
+| IPv4 CIDR blocks | Network size, ie number of available IPs per public subnet | 10.1.12.0/22 <br /> 10.1.16.0/22 <br /> 10.1.20.0/22 | yes |
+| Tags | Metadata for organizing your AWS resources | "kubernetes.io/cluster/\<cluster name>" = "shared" <br /> "kubernetes.io/role/elb" = "1" <br /> "purpose" = "public" | yes |
+| Network Access Lists | Allows or denies specific inbound or outbound traffic at the subnet level | Allow all inbound/outbound | yes |
+
+### 'Private' route table requirements
+
+| Requirement | Description |  Default value | Mandatory? |
+| ----------- | ----------- | -------------- | ---------- |
+| Routes | Minimum routes for network communication to work | 0.0.0.0/0 to \<NAT gateway> <br /> \<vpcCidrBlock> to local | yes |
+| Subnet associations | Apply route table routes to a particular subnet | Explicit, all private subnets | yes |
+
+### 'Public' route table requirements
+
+| Requirement | Description |  Default value | Mandatory? |
+| ----------- | ----------- | -------------- | ---------- |
+| Routes | Minimum routes for network communication to work | 0.0.0.0/0 to \<Internet gateway> <br /> \<vpcCidrBlock> to local | yes |
+| Subnet associations | Apply route table routes to a particular subnet | Explicit, all public subnets | yes |
 
 ## <a name="ServiceElasticLoadBalancing"></a> ![Elastic Load Balancing](https://raw.githubusercontent.com/awslabs/aws-icons-for-plantuml/main/dist/NetworkingContentDelivery/ElasticLoadBalancing.png) Elastic Load Balancing
 

CHANGELOG.md

@@ -1,7 +1,23 @@
 ## v0.2.0
-- RDS DB subnet group is now using VPC private subnets, instead of
-separate, private DB subnets. Please check MIGRATION.md document
-on procedure how to migrate existing deployments.
+
+- added option to use preconfigured VPC and subnets, in case of customers who
+want to manage network infrastructure themselves. You need to set ID of
+preconfigured VPC: if proper tagging is used on subnets, subnet IDs will be
+filtered out from given VPC ID. If no proper tagging is used on subnets, you
+need to supply IDs of preconfigured subnets. If no VPC ID is given, VPC and
+subnets will be created instead (default behaviour).
+- There is now a separate security group for the license server
+- CodeMeter runtime is now automatically installed and license server is
+configured as network server
+- Disk size is now modifiable for not only RDS but all EKS nodes.
+- Instance type list is now fixed with the selected instance types that are
+compatible with k8s and workload (e.g. cpu type, cpu and memory requests;
+16 Cores, 64 GiB). The list is ordered in such a way, so that the most cost
+efficient EC2 instance type is selected first.
+- RDS DB subnet group is now using VPC private subnets, instead of separate,
+private DB subnets. Please check MIGRATION.md document on procedure how to
+migrate existing deployments.
 
 ## v0.1.0
+
 Initial release of reference architecture

README.md

@@ -520,6 +520,11 @@ Important: During credentials rotation, SIMPHERA will not be available for a sho
 | [aws_eks_node_group.gpuivsnodes](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_node_group) | data source |
 | [aws_iam_policy_document.policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
 | [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
+| [aws_subnet.private_subnet](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnet) | data source |
+| [aws_subnet.public_subnet](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnet) | data source |
+| [aws_subnets.private_subnets](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnets) | data source |
+| [aws_subnets.public_subnets](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnets) | data source |
+| [aws_vpc.preconfigured](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) | data source |
 
 ## Inputs
 
@@ -555,10 +560,13 @@ Important: During credentials rotation, SIMPHERA will not be available for a sho
 | <a name="input_map_accounts"></a> [map\_accounts](#input\_map\_accounts) | Additional AWS account numbers to add to the aws-auth ConfigMap | `list(string)` | `[]` | no |
 | <a name="input_map_roles"></a> [map\_roles](#input\_map\_roles) | Additional IAM roles to add to the aws-auth ConfigMap | <pre>list(object({<br>    rolearn  = string<br>    username = string<br>    groups   = list(string)<br>  }))</pre> | `[]` | no |
 | <a name="input_map_users"></a> [map\_users](#input\_map\_users) | Additional IAM users to add to the aws-auth ConfigMap | <pre>list(object({<br>    userarn  = string<br>    username = string<br>    groups   = list(string)<br>  }))</pre> | `[]` | no |
+| <a name="input_private_subnet_ids"></a> [private\_subnet\_ids](#input\_private\_subnet\_ids) | List of IDs for the private subnets. | `list(any)` | `[]` | no |
+| <a name="input_public_subnet_ids"></a> [public\_subnet\_ids](#input\_public\_subnet\_ids) | List of IDs for the public subnets. | `list(any)` | `[]` | no |
 | <a name="input_scan_schedule"></a> [scan\_schedule](#input\_scan\_schedule) | 6-field Cron expression describing the scan maintenance schedule. Must not overlap with variable install\_schedule. | `string` | `"cron(0 0 * * ? *)"` | no |
 | <a name="input_simpheraInstances"></a> [simpheraInstances](#input\_simpheraInstances) | A list containing the individual SIMPHERA instances, such as 'staging' and 'production'. | <pre>map(object({<br>    name                         = string<br>    postgresqlApplyImmediately   = bool<br>    postgresqlVersion            = string<br>    postgresqlStorage            = number<br>    postgresqlMaxStorage         = number<br>    db_instance_type_simphera    = string<br>    enable_keycloak              = bool<br>    postgresqlStorageKeycloak    = number<br>    postgresqlMaxStorageKeycloak = number<br>    db_instance_type_keycloak    = string<br>    k8s_namespace                = string<br>    secretname                   = string<br>    enable_backup_service        = bool<br>    backup_retention             = number<br>    enable_deletion_protection   = bool<br><br>  }))</pre> | <pre>{<br>  "production": {<br>    "backup_retention": 35,<br>    "db_instance_type_keycloak": "db.t3.large",<br>    "db_instance_type_simphera": "db.t3.large",<br>    "enable_backup_service": true,<br>    "enable_deletion_protection": true,<br>    "enable_keycloak": true,<br>    "k8s_namespace": "simphera",<br>    "name": "production",<br>    "postgresqlApplyImmediately": false,<br>    "postgresqlMaxStorage": 100,<br>    "postgresqlMaxStorageKeycloak": 100,<br>    "postgresqlStorage": 20,<br>    "postgresqlStorageKeycloak": 20,<br>    "postgresqlVersion": "16",<br>    "secretname": "aws-simphera-dev-production"<br>  }<br>}</pre> | no |
 | <a name="input_tags"></a> [tags](#input\_tags) | The tags to be added to all resources. | `map(any)` | `{}` | no |
 | <a name="input_vpcCidr"></a> [vpcCidr](#input\_vpcCidr) | The CIDR for the virtual private cluster. | `string` | `"10.1.0.0/18"` | no |
+| <a name="input_vpcId"></a> [vpcId](#input\_vpcId) | The ID of preconfigured VPC. Change from 'null' to use already existing VPC. | `string` | `null` | no |
 | <a name="input_vpcPrivateSubnets"></a> [vpcPrivateSubnets](#input\_vpcPrivateSubnets) | List of CIDRs for the private subnets. | `list(any)` | <pre>[<br>  "10.1.0.0/22",<br>  "10.1.4.0/22",<br>  "10.1.8.0/22"<br>]</pre> | no |
 | <a name="input_vpcPublicSubnets"></a> [vpcPublicSubnets](#input\_vpcPublicSubnets) | List of CIDRs for the public subnets. | `list(any)` | <pre>[<br>  "10.1.12.0/22",<br>  "10.1.16.0/22",<br>  "10.1.20.0/22"<br>]</pre> | no |
 

data.tf

@@ -5,3 +5,42 @@ data "aws_availability_zones" "available" {
 data "aws_caller_identity" "current" {}
 
 data "aws_region" "current" {}
+
+data "aws_vpc" "preconfigured" {
+  count = local.create_vpc ? 0 : 1
+  id    = var.vpcId
+}
+
+data "aws_subnets" "private_subnets" {
+  count = local.create_vpc ? 0 : 1
+  filter {
+    name   = "vpc-id"
+    values = [var.vpcId]
+  }
+
+  tags = {
+    purpose = "private"
+  }
+}
+
+data "aws_subnet" "private_subnet" {
+  for_each = local.create_vpc ? toset([]) : toset(data.aws_subnets.private_subnets[0].ids)
+  id       = each.value
+}
+
+data "aws_subnets" "public_subnets" {
+  count = local.create_vpc ? 0 : 1
+  filter {
+    name   = "vpc-id"
+    values = [var.vpcId]
+  }
+
+  tags = {
+    purpose = "public"
+  }
+}
+
+data "aws_subnet" "public_subnet" {
+  for_each = local.create_vpc ? toset([]) : toset(data.aws_subnets.public_subnets[0].ids)
+  id       = each.value
+}

k8s.tf

@@ -4,8 +4,8 @@ module "eks" {
   source                                 = "git::https://github.com/aws-ia/terraform-aws-eks-blueprints.git?ref=v4.32.1"
   cluster_version                        = var.kubernetesVersion
   cluster_name                           = var.infrastructurename
-  vpc_id                                 = module.vpc.vpc_id
-  private_subnet_ids                     = module.vpc.private_subnets
+  vpc_id                                 = local.vpc_id
+  private_subnet_ids                     = local.private_subnets
   create_eks                             = true
   map_accounts                           = var.map_accounts
   map_users                              = var.map_users
@@ -41,8 +41,9 @@ module "eks-addons" {
 
   ingress_nginx_helm_config = {
     values = [templatefile("${path.module}/templates/nginx_values.yaml", {
-      internal = "false",
-      scheme   = "internet-facing"
+      internal       = "false",
+      scheme         = "internet-facing",
+      public_subnets = join(", ", local.public_subnets)
     })]
     namespace         = "nginx",
     create_namespace  = true

license-server.tf

@@ -1,9 +1,10 @@
 resource "aws_instance" "license_server" {
-  count                = var.licenseServer ? 1 : 0
-  ami                  = data.aws_ami.amazon_linux_kernel5.id
-  instance_type        = "t3a.large"
-  iam_instance_profile = aws_iam_instance_profile.license_server_profile[0].name
-  subnet_id            = module.vpc.private_subnets[0]
+  count                  = var.licenseServer ? 1 : 0
+  ami                    = data.aws_ami.amazon_linux_kernel5.id
+  instance_type          = "t3a.large"
+  iam_instance_profile   = aws_iam_instance_profile.license_server_profile[0].name
+  subnet_id              = local.private_subnets[0]
+  vpc_security_group_ids = [module.security_group_license_server[0].security_group_id]
 
   metadata_options {
     # [EC2.8] EC2 instances should use IMDSv2

locals.tf

@@ -8,6 +8,10 @@ data "aws_ami" "al2gpu_ami" {
 }
 
 locals {
+  create_vpc                                = var.vpcId == null ? true : false
+  vpc_id                                    = local.create_vpc ? module.vpc[0].vpc_id : var.vpcId
+  use_private_subnets_ids                   = length(var.private_subnet_ids) == 0 ? false : true
+  use_public_subnet_ids                     = length(var.public_subnet_ids) == 0 ? false : true
   infrastructurename                        = var.infrastructurename
   log_group_name                            = "/${module.eks.eks_cluster_id}/worker-fluentbit-logs"
   account_id                                = data.aws_caller_identity.current.account_id
@@ -24,22 +28,24 @@ locals {
   s3_instance_buckets                       = flatten([for name, instance in module.simphera_instance : instance.s3_buckets])
   license_server_bucket                     = var.licenseServer ? [aws_s3_bucket.license_server_bucket[0].bucket] : []
   s3_buckets                                = concat(local.s3_instance_buckets, [aws_s3_bucket.bucket_logs.bucket], local.license_server_bucket)
+  private_subnets                           = local.create_vpc ? module.vpc[0].private_subnets : (local.use_private_subnets_ids ? var.private_subnet_ids : [for s in data.aws_subnet.private_subnet : s.id])
+  public_subnets                            = local.create_vpc ? module.vpc[0].public_subnets : (local.use_public_subnet_ids ? var.public_subnet_ids : [for s in data.aws_subnet.public_subnet : s.id])
   # Using a one-line command for gpuPostUserData to avoid issues due to different line endings between Windows and Linux.
   gpuPostUserData = "curl -fSsl -O https://us.download.nvidia.com/tesla/${var.gpuNvidiaDriverVersion}/NVIDIA-Linux-x86_64-${var.gpuNvidiaDriverVersion}.run \nchmod +x NVIDIA-Linux-x86_64-${var.gpuNvidiaDriverVersion}.run \n./NVIDIA-Linux-x86_64-${var.gpuNvidiaDriverVersion}.run -s --no-dkms --install-libglvnd"
 
   default_managed_node_pools = {
     "default" = {
       node_group_name = "default"
       instance_types  = var.linuxNodeSize
-      subnet_ids      = module.vpc.private_subnets
+      subnet_ids      = local.private_subnets
       desired_size    = var.linuxNodeCountMin
       max_size        = var.linuxNodeCountMax
       min_size        = var.linuxNodeCountMin
     },
     "execnodes" = {
       node_group_name = "execnodes"
       instance_types  = var.linuxExecutionNodeSize
-      subnet_ids      = module.vpc.private_subnets
+      subnet_ids      = local.private_subnets
       desired_size    = var.linuxExecutionNodeCountMin
       max_size        = var.linuxExecutionNodeCountMax
       min_size        = var.linuxExecutionNodeCountMin
@@ -60,7 +66,7 @@ locals {
     "gpuexecnodes" = {
       node_group_name        = "gpuexecnodes"
       instance_types         = var.gpuNodeSize
-      subnet_ids             = module.vpc.private_subnets
+      subnet_ids             = local.private_subnets
       desired_size           = var.gpuNodeCountMin
       max_size               = var.gpuNodeCountMax
       min_size               = var.gpuNodeCountMin
@@ -85,7 +91,7 @@ locals {
     "gpuivsnodes" = {
       node_group_name        = "gpuivsnodes"
       instance_types         = var.ivsGpuNodeSize
-      subnet_ids             = module.vpc.private_subnets
+      subnet_ids             = local.private_subnets
       desired_size           = var.ivsGpuNodeCountMin
       max_size               = var.ivsGpuNodeCountMax
       min_size               = var.ivsGpuNodeCountMin

network.tf

@@ -1,5 +1,6 @@
 
 module "vpc" {
+  count                = local.create_vpc ? 1 : 0
   source               = "terraform-aws-modules/vpc/aws"
   version              = "v3.11.0"
   name                 = "${local.infrastructurename}-vpc"
@@ -29,36 +30,39 @@ module "security_group" {
   version     = "~> 4"
   name        = "${var.infrastructurename}-db-sg"
   description = "PostgreSQL security group"
-  vpc_id      = module.vpc.vpc_id
+  vpc_id      = local.vpc_id
   tags        = var.tags
   ingress_with_cidr_blocks = [
     {
       from_port   = 5432
       to_port     = 5432
       protocol    = "tcp"
       description = "PostgreSQL access from within VPC"
-      cidr_blocks = module.vpc.vpc_cidr_block
+      cidr_blocks = local.create_vpc ? module.vpc[0].vpc_cidr_block : data.aws_vpc.preconfigured[0].cidr_block
     },
   ]
 }
 
 # [EC2.6] VPC flow logging should be enabled in all VPCs
 # https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-standards-fsbp-controls.html#fsbp-ec2-6
 resource "aws_flow_log" "flowlog" {
-  iam_role_arn    = aws_iam_role.flowlogs_role.arn
-  log_destination = aws_cloudwatch_log_group.flowlogs.arn
+  count           = local.create_vpc ? 1 : 0
+  iam_role_arn    = aws_iam_role.flowlogs_role[0].arn
+  log_destination = aws_cloudwatch_log_group.flowlogs[0].arn
   traffic_type    = "ALL"
-  vpc_id          = module.vpc.vpc_id
+  vpc_id          = local.vpc_id
 }
 
 resource "aws_cloudwatch_log_group" "flowlogs" {
+  count             = local.create_vpc ? 1 : 0
   name              = local.flowlogs_cloudwatch_loggroup
   retention_in_days = var.cloudwatch_retention
   kms_key_id        = aws_kms_key.kms_key_cloudwatch_log_group.arn
   tags              = var.tags
 }
 
 resource "aws_iam_role" "flowlogs_role" {
+  count              = local.create_vpc ? 1 : 0
   name               = "${local.infrastructurename}-flowlogs-role"
   assume_role_policy = <<EOF
 {
@@ -78,6 +82,7 @@ EOF
 }
 
 resource "aws_iam_policy" "flowlogs_policy" {
+  count  = local.create_vpc ? 1 : 0
   name   = "${local.infrastructurename}-flowlogs-policy"
   policy = <<EOF
 {
@@ -100,6 +105,7 @@ EOF
 }
 
 resource "aws_iam_role_policy_attachment" "flowlogs_attachment" {
-  role       = aws_iam_role.flowlogs_role.id
-  policy_arn = aws_iam_policy.flowlogs_policy.arn
+  count      = local.create_vpc ? 1 : 0
+  role       = aws_iam_role.flowlogs_role[0].id
+  policy_arn = aws_iam_policy.flowlogs_policy[0].arn
 }