Merge pull request #44 from ivallhon/update-asg-workshop

Update "EC2 AutoScaling with multiple instance types and purchase options" workshop

content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/cleanup.md

@@ -14,15 +14,18 @@ If you're running in your own account, make sure you run through these steps to
 	```
 	aws autoscaling delete-auto-scaling-group --auto-scaling-group-name myEC2Workshop --force-delete
 		
-	aws elbv2 delete-load-balancer --load-balancer-arn %loadBalancerArn%
+	aws elbv2 delete-load-balancer --load-balancer-arn $LoadBalancerArn
 
-	aws elbv2 delete-target-group --target-group-arn %targetGroupArn%
+	aws elbv2 delete-target-group --target-group-arn $TargetGroupArn
 
 	aws ec2 delete-launch-template --launch-template-name myEC2Workshop
+
+	aws cloudformation delete-stack --stack-name spotinterruptionhandler
+
 	```
 
 1. Finally, delete the CloudFormation stack itself.
 
 	```
-	aws cloudformation delete-stack --stack-name myEC2Workshop
+	aws cloudformation delete-stack --stack-name $stack_name
 	```

content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/clone_github.md

@@ -18,4 +18,39 @@ In order to execute the steps in the workshop, you'll need to clone the workshop
 	cd ec2-spot-workshops/workshops/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options
 	```
 
-1. Feel free to browse around. You can also browse the directory structure in the **Environment** tab on the left, and even edit files directly there by double clicking on them.
+1. Feel free to browse around. You can also browse the directory structure in the **Environment** tab on the left, and even edit files directly there by double clicking on them.
+
+1. During the workshop, you will neeed to modify the configuration files to refer to the identifiers of the resources created by the CloudFormation stack you deployed. To reduce copy and paste across the CloudFormation console and the Cloud9 environment, we will load the CloudFormation Stack **Outputs** to environment variables. During the workshop the instructions will provide [sed](https://linux.die.net/man/1/sed) commands to populate configuration files. Make sure you open them on the Cloud9 editor to review the files and understand the settings of the resources you will be launching.
+
+	First, configure the stack_name environment variable with the name of your CloudFormation template. For example, if the name of your stack is **myEC2Workshop** run the following command:
+	```bash
+	export stack_name=myEC2Workshop
+
+	```
+
+	Now, load the CloudFormation stack outputs on environment variables running the following command:
+	```bash
+	export AWS_REGION=$(curl --silent http://169.254.169.254/latest/dynamic/instance-identity/document | jq -r .region)
+
+
+	# load outputs to env vars
+	for output in $(aws cloudformation describe-stacks --stack-name $stack_name --query 'Stacks[].Outputs[].OutputKey' --output text)
+	do
+    	export $output=$(aws cloudformation describe-stacks --stack-name $stack_name --query 'Stacks[].Outputs[?OutputKey==`'$output'`].OutputValue' --output text)
+    	eval "echo $output : \"\$$output\""
+	done
+
+	```
+
+	If successfull, the output should be similar to the following:
+
+	```bash
+	awsRegionId : eu-west-1
+	instanceProfile : arn:aws:iam::012345678910:instance-profile/running-workloads-at-scale-instanceProfile-1AWCE0JMHIRI4
+	vpc : vpc-0f0a34a6f7f3f999f
+	instanceSecurityGroup : sg-0ce120b3dde73b545
+	publicSubnet2 : subnet-0278bf57661c1f82b
+	publicSubnet1 : subnet-0f7bec73da5be90c2
+	cloud9Environment : cloud9Environment-C8KgzeALZ6w0
+	loadBalancerSecurityGroup : sg-0b6df7c3ed7c9118d
+	```

content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/create_asg.md

@@ -5,18 +5,36 @@ weight = 100
 
 Amazon EC2 Auto Scaling helps you maintain application availability and allows you to dynamically scale your Amazon EC2 capacity up or down automatically according to conditions you define. You can use Amazon EC2 Auto Scaling for fleet management of EC2 instances to help maintain the health and availability of your fleet and ensure that you are running your desired number of Amazon EC2 instances. You can also use Amazon EC2 Auto Scaling for dynamic scaling of EC2 instances in order to automatically increase the number of Amazon EC2 instances during demand spikes to maintain performance and decrease capacity during lulls to reduce costs. Amazon EC2 Auto Scaling is well suited both to applications that have stable demand patterns or that experience hourly, daily, or weekly variability in usage.
 
-1. Edit **asg.json** update the values of **%TargetGroupArn%** from the previous steps.
+1. Open **asg.json** on the Cloud9 editor and review the configuration. Pay special attention at the **Overrides** and the **InstancesDistribution** configuration blocks and try to guess how many instances of which instance type and which purchase option will be launched. Take a look at our [docs](https://docs.aws.amazon.com/autoscaling/ec2/userguide/asg-purchase-options.html#asg-allocation-strategies) to review how InstancesDistribution and allocation strategies work.
+{{%expand "Help me understand the AutoScaling configuration" %}}
+The **Overrides** configuration block provides EC2 AutoScaling the list of instance types your workload can run on. As Spot instances are **spare** EC2 capacity, your workloads should be flexible to run on multiple instance types and multiple availability zones; hence leveraging multiple *spot capacity pools* and making the most out of the available spare capacity. To select a list of instance types for your workload you can use [Spot Instance Advisor](https://aws.amazon.com/ec2/spot/instance-advisor/) which will help you filtering suitable instances by number of vCPUs and amount of memory and also provide you data about the interruption rate during the last 30 days for each instance type, so you can pick a list of best-suited instance types with low interruption rates.
 
-1. Update the values of **%publicSubnet1%** and **%publicSubnet2%** from the CloudFormat stack outputs.
+Then, the *InstancesDistribution* configuration block determines how EC2 AutoScaling picks the instance types to use, while at the same time it keeps a balanced number of EC2 instances per Availability Zone.
 
-1. Save the file and create the auto scaling group:
+* The **prioritized** allocation strategy for on-demand instances makes AutoScaling try to launch the first instance type of your list and skip to the next instance type if for any reason it's unable to launch it (e.g. temporary unavailability of capacity). This is particularly useful if you have Reserved Instances for your baseline capacity, so AutoScaling launches the instance type matching your reservations. 
+* **OnDemandBaseCapacity** is set to 2, meaning the first two EC2 instances launched by EC2 AutoScaling will be on-demand.
+* **OnDemandPercentageAboveBaseCapacity** is set to 0 so all the additional instances will be launched as Spot Instances
+* **SpotAllocationStrategy** is lowest-price, which instructs AutoScaling to pick the cheapest instance type on each Availability Zone.
+* **SpotInstancePools** is 4, which tells AutoScaling to launch instances across the 4 cheapest instance types on each Availability Zone for the list of instances provided on the overrides; diversifying our fleet acquiring capacity from multiple *Spot pools* and reducing the likelihood of a large portion of our fleet to be interrupted in a short period of time. 
+{{% /expand %}}
+
+1. You will notice there are placeholder values for **%TargetGroupArn%**, **%publicSubnet1%** and **%publicSubnet2%**. To update the configuration file with the values of the Target Group you created previously and the outputs from the CloudFormation template, execute the following command:
+```
+sed -i.bak -e "s#%TargetGroupARN%#$TargetGroupArn#g" -e "s#%publicSubnet1%#$publicSubnet1#g" -e "s#%publicSubnet2%#$publicSubnet2#g" asg.json
+```
 
-	```
-	aws autoscaling create-auto-scaling-group --cli-input-json file://asg.json
-	```
+1. Save the file and create the auto scaling group:
+```
+aws autoscaling create-auto-scaling-group --cli-input-json file://asg.json
+```
 {{% notice note %}}
 This command will not return any output if it is successful.
 {{% /notice %}}
 
 
-1. Browse to the [Auto Scaling console](https://console.aws.amazon.com/ec2/autoscaling/home#AutoScalingGroups:view=details) and check out your newly created auto scaling group. Take a look at the instances it has deployed.
+1. Browse to the [Auto Scaling console](https://console.aws.amazon.com/ec2/autoscaling/home#AutoScalingGroups:view=details) and check out your newly created auto scaling group. Take a look at the instances it has deployed.
+
+
+## Optional exercise
+
+Now that you have deployed an EC2 AutoScaling group with Mixed Instance Types and Purchase Options, take some time to manually scale out and scale in the number of instances of the group and see which instance types AutoScaling launches. Also, modify the SpotInstancePools parameter and experiment with the [capacity-optimized](https://aws.amazon.com/blogs/compute/introducing-the-capacity-optimized-allocation-strategy-for-amazon-ec2-spot-instances/) allocation strategy to get a good grasp of how the different allocation strategies behave. 

content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/create_lt.md

@@ -7,21 +7,28 @@ EC2 Launch Templates reduce the number of steps required to create an instance b
 
 You can create a launch template that contains the configuration information to launch an instance. Launch templates enable you to store launch parameters so that you do not have to specify them every time you launch an instance. For example, a launch template can contain the AMI ID, instance type, and network settings that you typically use to launch instances. When you launch an instance using the Amazon EC2 console, an AWS SDK, or a command line tool, you can specify the launch template to use.
 
+{{% notice note %}}
+You might be wondering how a [Launch Template](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-launch-templates.html) is different from a [Launch Configuration](https://docs.aws.amazon.com/autoscaling/ec2/userguide/LaunchConfiguration.html). They are similar in that they both specify instance configuration information; however Launch Templates provide additional features like versioning and enable you to use the latest features of Amazon EC2 and Auto Scaling Groups with multiple instance types and purchase options.  You can learn more about Launch Templates [here](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-launch-templates.html)
+{{% /notice %}}
+
 You'll use a launch template to specify configuration parameters for launching instances in this workshop.
 
-1. Edit **launch-template-data.json**.
+1. Open **launch-template-data.json** on the Cloud9 editor and examine the configuration, you will notice some of the parameters have a placeholder value **%variableName%**: %instanceProfile%, %instanceSecurityGroup% and %ami-id%.
+![Cloud9 Editor](/images/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/cloud9-editor.jpg)
 
-1. Update the following values from the CloudFormation stack outputs: **%instanceProfile%** and **%instanceSecurityGroup%**.
+1. The variable %ami-id% should contain the latest Amazon Linux 2 AMI, and instanceProfile and instanceSecurityGroup need to be populated with the resources created by your CloudFormation stack; which are available as Stack Outputs. We can pull the latest Amazon Linux 2 AMI with the AWS CLI, and as we have loaded our CloudFormation stack outputs as environment variables on a previous step, for convenience we can use the following commands to update your configuration file:
 
-1. Update **%ami-id%** with the AMI ID for the latest version of Amazon Linux 2 in the AWS region you launched. You can find the AMI ID by running the following command:
+    ```bash
+    # First, this command looks up the latest Amazon Linux 2 AMI
+    export ami_id=$(aws ec2 describe-images --owners amazon --filters 'Name=name,Values=amzn2-ami-hvm-2.0.????????-x86_64-gp2' 'Name=state,Values=available' --output json | jq -r '.Images |   sort_by(.CreationDate) | last(.[]).ImageId')
 
-	```	
-	aws ec2 describe-images --owners amazon --filters 'Name=name,Values=amzn2-ami-hvm-2.0.????????-x86_64-gp2' 'Name=state,Values=available' --output json | jq -r '.Images | sort_by(.CreationDate) | last(.[]).ImageId'
-	```
+    sed -i.bak -e "s#%instanceProfile%#$instanceProfile#g" -e "s/%instanceSecurityGroup%/$instanceSecurityGroup/g" -e "s#%ami-id%#$ami_id#g" -e "s#%UserData%#$(cat user-data.txt | base64 --wrap=0)#g" launch-template-data.json
+
+    ```
 
-1. Save the file.
+1. Your configuration file should now have the variables populated. If you don't see the file updated on the Cloud9 editor, click on it and you will get a message box indicating the file has changed. In that case, click on **Keep remote**.
 
-1. Create the launch template from the launch template config you just saved:
+1. Create the launch template from the launch template config you just updated:
 
 	```
 	aws ec2 create-launch-template --launch-template-name myEC2Workshop --launch-template-data file://launch-template-data.json
@@ -35,7 +42,7 @@ You'll use a launch template to specify configuration parameters for launching i
 	aws ec2 describe-launch-template-versions --launch-template-name myEC2Workshop
 	```
 
-1. Verify that the contents of the launch template user data are correct:
+1. Take a look at the user-data script configured on the launch template to understand what will be installed on the instances while being bootstrapped. 
 
 	```
 	aws ec2 describe-launch-template-versions  --launch-template-name myEC2Workshop --output json | jq -r '.LaunchTemplateVersions[].LaunchTemplateData.UserData' | base64 --decode

content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/deploy_lb.md

@@ -9,30 +9,60 @@ A listener checks for connection requests from clients, using the protocol and p
 
 Each target group routes requests to one or more registered targets, such as EC2 instances, using the protocol and port number that you specify. You can register a target with multiple target groups. You can configure health checks on a per target group basis. Health checks are performed on all targets registered to a target group that is specified in a listener rule for your load balancer.
 
-1. Edit **application-load-balancer.json** and update the values of **%publicSubnet1%**, **%publicSubnet2%**, and **%loadBalancerSecurityGroup%** from the CloudFormation stack outputs. Save the file. Create the application load balancer:
+1. Open **application-load-balancer.json** on the AWS Cloud9 editor and review the configuration. You will notice that some configurations have placeholder values: **%publicSubnet1%**, **%publicSubnet2%**, and **%loadBalancerSecurityGroup%**. 
+
+1. Execute the following command to populate the configuration file with the **Outputs** of your CloudFormation stack. 
+	```
+	sed -i.bak -e "s#%publicSubnet1%#$publicSubnet1#g" -e "s#%publicSubnet2%#$publicSubnet2#g" -e "s#%loadBalancerSecurityGroup%#$loadBalancerSecurityGroup#g" application-load-balancer.json
+	```
+
+1. Create the application load balancer:
 
 	```
 	aws elbv2 create-load-balancer --cli-input-json file://application-load-balancer.json
 	```
-{{% notice note %}}
-Please note the ARN of the application load balancer for use in an upcoming step.
-{{% /notice %}}
 
+1. As on upcoming steps we will need the load balancer ARN, execute the following command to load it into an environment variable:
+	```
+	export LoadBalancerArn=$(aws elbv2 describe-load-balancers --name myEC2Workshop --query LoadBalancers[].LoadBalancerArn --output text)
+	```
 
 1. Browse to the [Load Balancer console](https://console.aws.amazon.com/ec2/v2/home#LoadBalancers:sort=loadBalancerName) to check out your newly created load balancer.
 
-1. 	Edit **target-group.json** and update the value of **%vpc%** from the CloudFormation stack outputs. Save the file. Create the target group:
+1. 	Open the **target-group.json** and review the configuration. You will notice a placeholder value for the VPC configuration **%vpc%**. To populate it from the CloudFormation stack outputs, execute the following command:
+	```
+	sed -i.bak -e "s#%vpc%#$vpc#g" target-group.json
+	```
+
+1. Create the target group:
 
 	```
 	aws elbv2 create-target-group --cli-input-json file://target-group.json
 	```
-{{% notice note %}}
-Please note the ARN of the application target-group for use in an upcoming step.
-{{% /notice %}}
+
+1. As you will need the Target Group ARN of the Target Group you have created on an upcoming step, execute the following command to load it on an environment variable:
+	```
+	export TargetGroupArn=$(aws elbv2 describe-target-groups --names myEC2Workshop --query TargetGroups[].TargetGroupArn --output text)
+	```
+1. Open **modify-target-group.json** on the Cloud9 editor and review its configuration. Then, update the value of **%TargetGroupArn%** with the ARN with the following command:  
+	```
+	sed -i.bak -e "s#%TargetGroupArn%#$TargetGroupArn#g" modify-target-group.json
+	```
+
+1. Modify the target group to set the deregistration_delay_timeout to 2 minutes to match the Spot interruption notification time (default is 5 minutes). You will learn how this setting is used when Spot instances are going to be interrupted on the *Spot resilience* section.
+
+	```
+	aws elbv2 modify-target-group-attributes --cli-input-json file://modify-target-group.json
+	```
 
 1. Browse to the [Target Group console](https://console.aws.amazon.com/ec2/v2/home#TargetGroups:sort=targetGroupName) to check out your newly created target group.
 
-1. Edit **listener.json** and update the values of **%LoadBalancerArn%** and **%TargetGroupArn%** from the previous steps. Save the file. Create the listener:
+1. Open **listener.json** and review the configuration. You will notice there are placeholder values for **%LoadBalancerArn%** and **%TargetGroupArn%**. To populate them with the Load Balancer and Target group created previously, execute the following command.
+	```
+	sed -i.bak -e "s#%LoadBalancerArn%#$LoadBalancerArn#g" -e "s#%TargetGroupArn%#$TargetGroupArn#g" listener.json
+	```
+
+1. Create the listener wiht the following command:
 
 	```
 	aws elbv2 create-listener --cli-input-json file://listener.json