diff --git a/.gitignore b/.gitignore index ea18cd18..db922810 100644 --- a/.gitignore +++ b/.gitignore @@ -23,7 +23,9 @@ cli/semaphores/signal cli/semaphores/unlink api/api_generated.go api/server/executors/executors_generated.go -drivers/storage/ebs/tests/.vagrant +drivers/storage/ebs/tests/.vagrant/ +examples/ec2/vagrant/.vagrant/ + # Created by https://www.gitignore.io diff --git a/examples/ec2/vagrant/README.md b/examples/ec2/vagrant/README.md new file mode 100644 index 00000000..e8741acd --- /dev/null +++ b/examples/ec2/vagrant/README.md @@ -0,0 +1,100 @@ +# EC2 Vagrantfile +This directory includes a Vagrantfile that can be used to bring a libStorage +server/client installation online using REX-Ray and EC2 instances. For example: + +``` +vagrant up --provider=aws --no-parallel +``` + +The following sections outline dependencies, settings, and different execution +scenarios that may be required or useful for using the Vagrantfile. + +### Dependencies +The following dependencies are required in order to bring the EC2 Vagrant +enviornment online: + + * [Vagrant](https://www.vagrantup.com/) 1.8.5+ + * [vagrant-aws](https://github.com/mitchellh/vagrant-aws) + * [vagrant-hostmanager](https://github.com/devopsgroup-io/vagrant-hostmanager) + +Once Vagrant is installed the required plug-ins may be installed with the +following commands: + +```bash +vagrant plugin install vagrant-aws +vagrant plugin install vagrant-hostmanager +``` + +### Settings +The following environment variables may be used to configure the `Vagrantfile`. + +Environment Variable | Description | Required | Default +---------------------|-------------|:--------:|-------- +`AWS_AKEY` | The AWS access key ID. | ✓ | +`AWS_SKEY` | The AWS secret key. | ✓ | +`AWS_KPNM` | The name of the AWS key pair for instances. | ✓ | +`AWS_AMI` | The AMI to use. Defaults to Amazon Linux PV x64 ([AMIs by region](https://aws.amazon.com/amazon-linux-ami/)) | | ami-de347abe +`AWS_REGN` | The AWS region. | | us-west-1 +`AWS_ZONE` | The AWS availability zone. | | a +`AWS_SSHK` | Local SSH key used to access AWS instances. | ✓ | + +### Nodes +The `Vagrantfile` which deploys a libStorage server and two clients onto EC2 +instances named: + + * libstorage-server + * libstorage-client0 + * libstorage-client1 + +### Test Plan Boot Order +The order in which Vagrant brings the nodes online can vary: + +```bash +vagrant up --provider=aws +``` + +The above command will bring all three nodes -- the server and both clients -- +online in parallel since the Vagrant AWS plug-in is configured to support +the parallel option. However, the Vagrantfile will fail if the server +is not brought online first. That is why when bringing the Vagrant nodes online +all at once, the following command should be used: + +```bash +vagrant up --provider=aws --no-parallel +``` + +The above command will bring the nodes online in the following order: + + 1. libstorage-server + 2. libstorage-client0 + 3. libstorage-client1 + +However, another option for starting the environment is to bring up the server +first and then the clients in parallel. Doing so can duplicate a scenario +where two clients are contending for storage resources: + +```bash +vagrant up --provider=aws libstorage-server +vagrant up --provider=aws '/.*client.*/' +``` + +### Scripts +This package includes several test scripts that act as simple ways to execute +random pieces of logic for a node after it is brought online: + + * `server.sh` + * `client0.sh` + * `client1.sh` + +The above files are copied to their respective instances and executed +as soon as the instance is online. That means the `server-tests.sh` script is +executed before the client nodes are even online since the server node is +brought online first. + +### Cleanup +It's important to remember to clean up any EC2, EBS, & EFS resources that may +have been created along the way. To do so simply execute the following command: + +```bash +vagrant destroy -f +``` diff --git a/examples/ec2/vagrant/Vagrantfile b/examples/ec2/vagrant/Vagrantfile new file mode 100644 index 00000000..d3d88d31 --- /dev/null +++ b/examples/ec2/vagrant/Vagrantfile @@ -0,0 +1,590 @@ +# -*- mode: ruby -*- +# vi: set ft=ruby : +require 'fileutils' +require 'shellwords' +require 'fog' + +# general config +$autostart_clients = true # set to false to prevent clients from auto-starting + +# aws config +$aws_akey = ENV['AWS_AKEY'] +$aws_skey = ENV['AWS_SKEY'] +$aws_kpnm = ENV['AWS_KPNM'] +$aws_ami = ENV['AWS_AMI'] ? ENV['AWS_AMI'] : "ami-ef2af28f" +$aws_regn = ENV['AWS_REGN'] ? ENV['AWS_REGN'] : "us-west-2" +$aws_zone = ENV['AWS_ZONE'] ? ENV['AWS_ZONE'] : "a" + +# ssh config +$ssh_user = "centos" +$ssh_pkey = ENV['AWS_SSHK'] # path to SSH private key for AWS + +# node info +$node0_name = "libstorage-server" +$node0_itype = "m4.large" +$node0_texps = "server.sh" + +$node1_name = "libstorage-client0" +$node1_itype = "m4.large" +$node1_texps = "client0.sh" + +$node2_name = "libstorage-client1" +$node2_itype = "t2.micro" +$node2_texps = "client1.sh" + +# Golang information +$goos = "linux" +$goarch = "amd64" +$gover = "1.7.1" +$gotgz = "go#{$gover}.#{$goos}-#{$goarch}.tar.gz" +$gourl = "https://storage.googleapis.com/golang/#{$gotgz}" +$gopath = "/opt/go" + +# project info +$srcs = "#{$gopath}/src/github.com/codedellemc/libstorage" + +# the script to ensure the copied working source is a git repo +$validate_copied_sources = <