Merge pull request #9 from prabirsekhri/dev-ec2-module

Dev ec2 module
prabirsekhri · Jul 17, 2023 · dbbf8f4 · dbbf8f4
2 parents 92cedb3 + 2de2f30
commit dbbf8f4
Show file tree

Hide file tree

Showing 43 changed files with 1,508 additions and 69 deletions.
diff --git a/.header.md b/.header.md
@@ -2,7 +2,7 @@
 
 This repository contains Terraform code which creates resources required to run Storage Gateway (https://aws.amazon.com/storagegateway/) in AWS and on premises. 
 
-AWS Storage Gateway is available in 4 types : 
+AWS Storage Gateway is available in 4 types: 
 
 - Amazon S3 File Gateway (FILE_S3)
 - Amazon FSx File Gateway (FILE_FSX_SMB)
@@ -15,9 +15,9 @@ The module requires a Gateway type to be declared. The default is configured to
 
 - Link to the S3 SMB Storage Gateway example for VMware: [s3filegateway-vmware](examples/s3filegateway-vmware)
 
-### Prerequisists
+### Prerequisites
 
-- The VMware module requires the vsphere provider to be setup with a service account user name and password that has the necessary permissions in Vcenter to create a VM. This is found in the [settings.tf](examples/s3filegateway-vmware/settings.tf) file. 
+- The VMware module requires the vSphere provider to be setup with a service account user name and password that has the necessary permissions in vCenter to create a VM. This is found in the [settings.tf](examples/s3filegateway-vmware/settings.tf) file. 
 
 ```hcl
 
@@ -32,7 +32,7 @@ provider "vsphere" {
 
 Note that var.allow_unverified_ssl is a boolean that can be set to true to disable SSL certificate verification. This should be used with care as it could allow an attacker to intercept your authentication token. The default value is set to false but can be changed to true for testing purposes only. 
 
-The module also requires connectivity to your vCenter server. Therefore it needs to be deployed from a virtual machine that can reach the vCenter APIs. You may also [Terraform Cloud Agents](https://developer.hashicorp.com/terraform/cloud-docs/agents) if you use already use Terrform Cloud. This allows the modules to be deployed remotely. 
+The module also requires connectivity to your vCenter server. Therefore, it needs to be deployed from a virtual machine that can reach the vCenter APIs. You may also [Terraform Cloud Agents](https://developer.hashicorp.com/terraform/cloud-docs/agents) if you use already use Terraform Cloud. This allows the modules to be deployed remotely. 
 
 ### [vSphere Module](modules/vmware-sgw/)
 
@@ -89,7 +89,45 @@ module "sgw" {
 
 Refer to to the S3 NFS Storage Gateway example for VMware for an end to end example: [s3-nfs-filegateway-vmware](examples/s3-nfs-filegateway-vmware)
 
-## Setting up S3 buckets for S3 File Gateway
+## Usage with Amazon EC2 File Gateway module
+
+- Link to the S3 NFS Storage Gateway example for Amazon EC2: [s3-nfs-filegateway-ec2](examples/s3-nfs-filegateway-ec2)
+
+
+### [EC2 Storage Gateway module](modules/ec2-sgw/)
+
+```hcl
+
+module "ec2-sgw" {
+  source     = "aws-ia/storagegateway/aws//modules/ec2-sgw"
+  vpc_id               = "vpc-abcdef123456"
+  subnet_id            = "subnet-abcdef123456"
+  name                 = "my-storage-gateway"
+  availability_zone    = data.aws_availability_zones.available.names[0]
+  aws_region           = var.aws_region
+  ssh_public_key_path  = var.ssh_public_key_path //optional
+}
+```
+
+Note that the ssh_public_key_path is an optional attribute which takes the absolute path to your public key. The ssh key allows you to administer your Storage Gateway appliance. To create a public key for Amazon EC2, follow this procedure using ssh-keygen. Example path “/Users/user/.ssh/id_rsa.pub”.
+
+### [Storage Gateway Module](modules/aws-sgw/)
+
+Once the EC2 appliance is deployed, the public IP address of the EC2 instance needs to be passed to next module as the gateway IP address. 
+
+```hcl
+
+module "sgw" {
+  depends_on         = [module.ec2_sgw]
+  source             = "aws-ia/storagegateway/aws//modules/aws-sgw"
+  gateway_name       = "my-storage-gateway"
+  gateway_ip_address = module.ec2-sgw.public_ip
+  join_smb_domain    = false
+  gateway_type       = "FILE_S3"
+}
+```
+
+### Setting up S3 buckets for S3 File Gateway
 
 ```hcl
 module "s3_bucket" {
@@ -123,7 +161,7 @@ module "s3_bucket" {
 ```
 Note that versioning is set to false by default for the S3 bucket for the file share for Storage Gateway. Enabling S3 Versioning can increase storage costs within Amazon S3. Please see [here](https://docs.aws.amazon.com/filegateway/latest/files3/CreatingAnSMBFileShare.html) for further information on whether S3 Versioning is right for your workload.
 
-## Setting up SMB File shares
+### Setting up SMB File shares
 
 ```hcl
 module "smb_share" {
@@ -136,7 +174,7 @@ module "smb_share" {
 }
 ```
 
-## Setting up NFS File shares
+### Setting up NFS File shares
 
 ```hcl
 module "nfs_share" {
@@ -152,6 +190,77 @@ module "nfs_share" {
 
 The examples also includes "aws_kms_key" resource block to create a KMS key. For production deployments, you should pass in a key policy that restricts the use of the key based on your access requirements. Refer to this [link](https://docs.aws.amazon.com/kms/latest/developerguide/key-policies.html) for information. 
 
+## Networking Considerations
+
+### Storage Gateway VPC Endpoint configuration for EC2 Gateway
+
+Terraform Storage Gateway module allows you to optionally create an interface VPC Endpoint for Storage Gateway by setting create_vpc_endpoint=true. You can use this connection to activate your gateway and configure it to transfer data to AWS storage services without communicating over the public internet
+
+Example with VPC endpoint configuration : 
+
+
+```hcl
+
+module "ec2-sgw" {
+  source     = "aws-ia/storagegateway/aws//modules/ec2-sgw"
+  gateway_name                       = random_pet.name.id
+  gateway_ip_address                 = module.ec2-sgw.public_ip
+  join_smb_domain                    = false
+  gateway_type                       = "FILE_S3"
+  create_vpc_endpoint                = true
+  create_vpc_endpoint_security_group = true #if false define vpc_endpoint_security_group_id 
+  vpc_id                             = module.vpc.vpc_id
+  vpc_endpoint_subnet_ids            = module.vpc.private_subnets
+  gateway_private_ip_address         = module.ec2-sgw.private_ip
+}
+```
+
+ A security group is also needed for the VPC Endpoint. In the above example, the module handles creation of the security group. However, you may use the vpc_endpoint_security_group_id variable to associate an existing Security group with the VPC endpoint. Please see this [documentation](https://docs.aws.amazon.com/filegateway/latest/files3/gateway-private-link.html) which shows the Security Group requirements for Storage Gateway VPC endpoint. In this module, the security groups are already pre-configured with the required rules with the private IP address of the storage gateway appliance. The configuration can be found in the file [sg.tf](modules/aws-sgw/sg.tf)  file. 
+
+S3 VPC Endpoint configuration
+
+We recommend you configure create a separate VPC endpoint for Amazon S3 File Gateway to transfer data through the VPC rather than a NAT Gateway or NAT Instances. This allows for optimized and private routing to S3 and lower cost. In the S3 NFS File gateway example's [main.tf](examples/s3-nfs-filegateway-ec2/main.tf), we have created a Gateway VPC endpoint as shown below.
+
+```hcl
+resource "aws_vpc_endpoint" "s3" {
+  vpc_id          = module.vpc.vpc_id
+  service_name    = "com.amazonaws.${var.aws_region}.s3"
+  route_table_ids = module.vpc.private_route_table_ids
+} 
+```
+### Storage Gateway Security Group Configuration for EC2 Gateway
+
+You can optionally create the security group and the required rules required for your gateway appliance by setting the variable to create_security_group = true. You can also limit access to range of ingress CIDR blocks in your network from where you require access to the storage gateway by modifying ingress_cidr_blocks attributes as shown in the example below. 
+
+The module also includes the ingress_cidr_block_activation variable specifically to limit access to the CIDR block of the client machine that activates the storage gateway on port 80. This Security Group rule can be optionally removed once the gateway is activated. The source code of the security group configuration can be found in modules/ec2-sgw/sg.tf file. 
+
+```hcl
+
+module "ec2-sgw" {
+  source                        = "aws-ia/storagegateway/aws//modules/ec2-sgw"
+  vpc_id                        = var.vpc_id
+  subnet_id                     = var.subnet_id
+  ingress_cidr_block_activation = "10.0.0.1/32"
+  ingress_cidr_blocks           = ["172.16.0.0/24", "172.16.10.0/24"]
+  create_security_group         = true
+}
+```
+
+To use your own security group, set create_security_group = false and append your own security_group_id attribute as shown in the example below : 
+
+As an example : 
+
+```hcl
+
+module "ec2-sgw" {
+  source                = "aws-ia/storagegateway/aws//modules/ec2-sgw"
+  vpc_id                = var.vpc_id
+  subnet_id             = var.subnet_id
+  create_security_group = false 
+  security_group_id     = "sg-12345678"
+}
+```
+
 ## Support & Feedback
 
 Storage Gateway module for Terraform is maintained by AWS Solution Architects. It is not part of an AWS service and support is provided best-effort by the AWS Storage community.