Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Terraform AWS EC2 VM Example #583

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
24 changes: 24 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,27 @@
bazel-*
compile_commands.json
.gitconfig

# Local .terraform directories
**/.terraform/*

# .tfstate files
*.tfstate
*.tfstate.*
*.private

# Crash log files
crash.log

# Terraform lock files
.terraform.lock.hcl

# Ignore any .tfvars files
*.tfvars

# Ignore override files as they are usually used to override resources locally and so
# are not checked in
override.tf
override.tf.json
*_override.tf
*_override.tf.json
111 changes: 111 additions & 0 deletions cloud-service-provider/aws/ec2-vm/terraform/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
## 🤖 Terraform Example for AWS EC2 VM using the default VPC

This example uses the following Terraform module:

[Terraform Module](https://registry.terraform.io/modules/intel/aws-vm/intel/latest)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we use the standard terraform ec2_instance module instead of Intel one? They seem to be almost the same.

Copy link
Author

@lucasmelogithub lucasmelogithub Nov 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey Sakari, thanks for the review. The module on the PR has already been used to showcase end-to-end OPEA of ChatQna and CodeGen automation in AWS/Azure/GCP. Implements system provisioning plus OPEA deployment in minutes, with no human intervention.
Example here: https://github.com/opea-project/GenAIExamples/blob/main/ChatQnA/README.md#-automated-terraform-deployment-using-intel-optimized-cloud-modules-for-terraform
There are plans to support even more examples.

For consistency, this PR is to backport/enable a generic VM creation using the same module.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My point is that we should use vendor neutral ec2-instance module from the Terraform community instead of Intel specific clone (ec2-vm) of the module. Or am I missing something here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The module being used is also an Open Source Community module.
https://registry.terraform.io/modules/intel/aws-vm/intel/latest

**For additional customization, refer to the module documentation under "Inputs".**

**The Module supports non-default VPCs and much more than what is shown in this example.**

## Overview

This example creates an AWS EC2 in the default VPC. The default region is can be changed in variables.tf.

This example also creates:

- Public IP
- EC2 key pair
- The private key is created in the local system where terraform apply is done
- It also creates a new security groups for network access

## Prerequisites

1. **Install AWS CLI**: Follow the instructions to install the AWS CLI from the [official documentation](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html).

2. **Configure AWS CLI**: Run the following command to configure your AWS credentials and default region.
3. **Install Terraform**: Follow the instructions to install Terraform from the [official documentation](https://learn.hashicorp.com/tutorials/terraform/install-cli).
4. Have your preferred Git client installed. If you don't have one, you can download it from [Git](https://git-scm.com/downloads).

## Configure AWS CLI

```bash
aws configure
```

You will be prompted to enter your AWS Access Key ID, Secret Access Key, default region name, and output format.

## Modify the example to suit your needs

This example is can be customized to your needs by modifying the following files:

```bash
main.tf
variables.tf
```

For additional customization, refer to the module documentation under **"Inputs"** [Terraform Module](https://registry.terraform.io/modules/intel/aws-vm/intel/latest)
.

The module supports much more than what is shown in this example.

## Usage

In variables.tf, replace the below with you own IPV4 CIDR range before running the example.

Use <https://whatismyipaddress.com/> to get your IP address.

```hcl
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = "A.B.C.D/32"
```

**Depending on your use case, you might also need to allow additional.
ports.**

**Modify variable.tf replicating the existing format to add additional ports.**

## Run Terraform

```bash
git clone https://github.com/opea-project/GenAIInfra.git
cd GenAIInfra/cloud-service-provider/aws/ec2-vm/terraform

# Modify the main.tf and variables.tf file to suit your needs (see above)

terraform init
terraform plan
terraform apply
```

## SSH

At this point, the EC2 instance should be up and running. You can SSH into the instance using the private key created in the local system.

```bash
chmod 600 tfkey.private
ssh -i tfkey.private ubuntu@***VM_PUBLIC_IP***

# If in a proxy environment, use the following command
ssh -i tfkey.private -x your-proxy.com.com:PORT ubuntu@***VM_PUBLIC_IP***
```

## OPEA

You can now deploy OPEA components using OPEA instructions.

[OPEA GenAI Examples](https://github.com/opea-project/GenAIExamples)

## Destroy

To destroy the resources created by this example, run the following command:

```bash
terraform destroy
```

## Considerations

The AWS region where this example is run should have a default VPC.
83 changes: 83 additions & 0 deletions cloud-service-provider/aws/ec2-vm/terraform/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Provision EC2 Instance on AWS in default vpc. It is configured to create the EC2 in
# US-East-1 region. The region is provided in variables.tf in this example folder.

# This example also create an EC2 key pair. Associate the public key with the EC2 instance.
# Creates the private key in the local system where terraform apply is done.
# Create a new security group to open up the SSH port 22 to a specific IP CIDR block
# To ssh:
# chmod 600 tfkey.private
# ssh -i tfkey.private ubuntu@<public_ip>

data "aws_ami" "ubuntu-linux-2204" {
most_recent = true
owners = ["099720109477"] # Canonical
filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
}

resource "random_id" "rid" {
byte_length = 5
}

# RSA key of size 4096 bits
resource "tls_private_key" "rsa" {
algorithm = "RSA"
rsa_bits = 4096
}

resource "aws_key_pair" "TF_key" {
key_name = "TF_key-${random_id.rid.dec}"
public_key = tls_private_key.rsa.public_key_openssh
}

resource "local_file" "TF_private_key" {
content = tls_private_key.rsa.private_key_pem
filename = "tfkey.private"
}
resource "aws_security_group" "ssh_security_group" {
description = "security group to configure ports for ssh"
name_prefix = "ssh_security_group"
}

# Modify the `ingress_rules` variable in the variables.tf file to allow the required ports for your CIDR ranges
resource "aws_security_group_rule" "ingress_rules" {
count = length(var.ingress_rules)
type = "ingress"
security_group_id = aws_security_group.ssh_security_group.id
from_port = var.ingress_rules[count.index].from_port
to_port = var.ingress_rules[count.index].to_port
protocol = var.ingress_rules[count.index].protocol
cidr_blocks = [var.ingress_rules[count.index].cidr_blocks]
}

resource "aws_network_interface_sg_attachment" "sg_attachment" {
count = length(module.ec2-vm)
security_group_id = aws_security_group.ssh_security_group.id
network_interface_id = module.ec2-vm[count.index].primary_network_interface_id
}

# Modify the `vm_count` variable in the variables.tf file to create the required number of EC2 instances
module "ec2-vm" {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be module ec2_instance from terraform cummunity.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The module being used is also an Open Source Community module.
https://registry.terraform.io/modules/intel/aws-vm/intel/latest

count = var.vm_count
source = "intel/aws-vm/intel"
version = "1.3.3"
key_name = aws_key_pair.TF_key.key_name
instance_type = var.instance_type # Modify the instance type as required for your AI needs
availability_zone = var.availability_zone
ami = data.aws_ami.ubuntu-linux-2204.id
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be a variable similar to region.

Copy link
Author

@lucasmelogithub lucasmelogithub Dec 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added variables. Thx for the review.


# Size of VM disk in GB
root_block_device = [{
volume_size = var.volume_size
}]

tags = {
Name = "opea-vm-${random_id.rid.dec}"
}
}
113 changes: 113 additions & 0 deletions cloud-service-provider/aws/ec2-vm/terraform/outputs.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
output "id" {
description = "The ID of the instance"
value = try(module.ec2-vm.*.id, module.ec2-vm.*.id, "")
}

output "arn" {
description = "The ARN of the instance"
value = try(module.ec2-vm.*.arn, "")
}

output "capacity_reservation_specification" {
description = "Capacity reservation specification of the instance"
value = try(module.ec2-vm.*.capacity_reservation_specification, "")
}

output "instance_state" {
description = "The state of the instance. One of: `pending`, `running`, `shutting-down`, `terminated`, `stopping`, `stopped`"
value = try(module.ec2-vm.*.instance_state, "")
}

output "outpost_arn" {
description = "The ARN of the Outpost the instance is assigned to"
value = try(module.ec2-vm.*.outpost_arn, "")
}

output "password_data" {
description = "Base-64 encoded encrypted password data for the instance. Useful for getting the administrator password for instances running Microsoft Windows. This attribute is only exported if `get_password_data` is true"
value = try(module.ec2-vm.*.password_data, "")
}

output "primary_network_interface_id" {
description = "The ID of the instance's primary network interface"
value = try(module.ec2-vm.*.primary_network_interface_id, "")
}

output "private_dns" {
description = "The private DNS name assigned to the instance. Can only be used inside the Amazon EC2, and only available if you've enabled DNS hostnames for your VPC"
value = try(module.ec2-vm.*.private_dns, "")
}

output "public_dns" {
description = "The public DNS name assigned to the instance. For EC2-VPC, this is only available if you've enabled DNS hostnames for your VPC"
value = try(module.ec2-vm.*.public_dns, "")
}

output "public_ip" {
description = "The public IP address assigned to the instance, if applicable. NOTE: If you are using an aws_eip with your instance, you should refer to the EIP's address directly and not use `public_ip` as this field will change after the EIP is attached"
value = try(module.ec2-vm.*.public_ip, "")
}

output "private_ip" {
description = "The private IP address assigned to the instance."
value = try(module.ec2-vm.*.private_ip, "")
}

output "ipv6_addresses" {
description = "The IPv6 address assigned to the instance, if applicable."
value = try(module.ec2-vm.*.ipv6_addresses, [])
}

output "tags_all" {
description = "A map of tags assigned to the resource, including those inherited from the provider default_tags configuration block"
value = try(module.ec2-vm.*.tags_all, {})
}

output "spot_bid_status" {
description = "The current bid status of the Spot Instance Request"
value = try(module.ec2-vm.*.spot_bid_status, "")
}

output "spot_request_state" {
description = "The current request state of the Spot Instance Request"
value = try(module.ec2-vm.*.spot_request_state, "")
}

output "spot_instance_id" {
description = "The Instance ID (if any) that is currently fulfilling the Spot Instance request"
value = try(module.ec2-vm.*.spot_instance_id, "")
}

################################################################################
# IAM Role / Instance Profile
################################################################################

output "iam_role_name" {
description = "The name of the IAM role"
value = try(module.ec2-vm.*.aws_iam_role.name, null)
}

output "iam_role_arn" {
description = "The Amazon Resource Name (ARN) specifying the IAM role"
value = try(module.ec2-vm.*.aws_iam_role.arn, null)
}

output "iam_role_unique_id" {
description = "Stable and unique string identifying the IAM role"
value = try(module.ec2-vm.*.aws_iam_role.unique_id, null)
}

output "iam_instance_profile_arn" {
description = "ARN assigned by AWS to the instance profile"
value = try(module.ec2-vm.*.aws_iam_instance_profile.arn, null)
}

output "iam_instance_profile_id" {
description = "Instance profile's ID"
value = try(module.ec2-vm.*.aws_iam_instance_profile.id, null)
}

output "iam_instance_profile_unique" {
description = "Stable and unique string identifying the IAM instance profile"
value = try(module.ec2-vm.*.aws_iam_instance_profile.unique_id, null)
}
4 changes: 4 additions & 0 deletions cloud-service-provider/aws/ec2-vm/terraform/providers.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
provider "aws" {
# Environment Variables used for Authentication
region = var.region
}
Loading