Asg capacity rebalancing & EMR updates

content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/_index.md

@@ -1,7 +1,7 @@
 ---
 title: "EC2 Auto Scaling with multiple instance types and purchase options"
 date: 2019-01-26T00:00:00Z
-weight: 90
+weight: 10
 pre: "<b>⁃ </b>"
 ---
 

content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/cleanup.md

@@ -16,16 +16,23 @@ If you're running in your own account, make sure you run through these steps to
 		
 	aws elbv2 delete-load-balancer --load-balancer-arn $LoadBalancerArn
 
-	aws elbv2 delete-target-group --target-group-arn $TargetGroupArn
-
 	aws ec2 delete-launch-template --launch-template-name myEC2Workshop
 
-	aws cloudformation delete-stack --stack-name spotinterruptionhandler
+	```
 
+1. Delete the Target Group created (you need to wait until the Application Load Balancer has been completely deleted).
+	```
+	aws elbv2 delete-target-group --target-group-arn $TargetGroupArn
 	```
 
 1. Finally, delete the CloudFormation stack itself.
 
 	```
 	aws cloudformation delete-stack --stack-name $stack_name
+	```
+
+1. If you ran the optional Custom Spot Interruption handling exercise, make sure you remove the Spot Interruption handler CloudFormation template deployed by the Serverless Application Repository. 
+
+	```
+	aws cloudformation delete-stack --stack-name serverlessrepo-ec2-spot-interruption-handler
 	```

content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/create_asg.md

@@ -5,17 +5,18 @@ weight = 100
 
 Amazon EC2 Auto Scaling helps you maintain application availability and allows you to dynamically scale your Amazon EC2 capacity up or down automatically according to conditions you define. You can use Amazon EC2 Auto Scaling for fleet management of EC2 instances to help maintain the health and availability of your fleet and ensure that you are running your desired number of Amazon EC2 instances. You can also use Amazon EC2 Auto Scaling for dynamic scaling of EC2 instances in order to automatically increase the number of Amazon EC2 instances during demand spikes to maintain performance and decrease capacity during lulls to reduce costs. Amazon EC2 Auto Scaling is well suited both to applications that have stable demand patterns or that experience hourly, daily, or weekly variability in usage.
 
-1. Open **asg.json** on the Cloud9 editor and review the configuration. Pay special attention at the **Overrides** and the **InstancesDistribution** configuration blocks and try to guess how many instances of which instance type and which purchase option will be launched. Take a look at our [docs](https://docs.aws.amazon.com/autoscaling/ec2/userguide/asg-purchase-options.html#asg-allocation-strategies) to review how InstancesDistribution and allocation strategies work.
+Amazon EC2 Auto Scaling also allows you to combine purchase options and instance types so you can optimize your compute costs. Stateless web applications are a great fit to run on Spot Instances as they can tolerate interruptions and are often flexible to run on multiple instance types. In this section, you will create an Auto Scaling group combining a base of On-Demand instances and scaling out with EC2 Spot instances and save an average of 70% in your compute costs. 
+
+1. Open **asg.json** on the Cloud9 editor and review the configuration. Pay special attention at the **Overrides** and the **InstancesDistribution**. Take a look at our [docs](https://docs.aws.amazon.com/autoscaling/ec2/userguide/asg-purchase-options.html#asg-allocation-strategies) to review how InstancesDistribution and allocation strategies work. You will also notice that the **CapacityRebalance** parameter is set to true, which will proactively attempt to replace Spot Instances at elevated risk of interruption. To learn more about the Capacity Relabancing feature, take a look at the [docs](https://docs.aws.amazon.com/autoscaling/ec2/userguide/capacity-rebalance.html).
 {{%expand "Help me understand the AutoScaling configuration" %}}
-The **Overrides** configuration block provides EC2 AutoScaling the list of instance types your workload can run on. As Spot instances are **spare** EC2 capacity, your workloads should be flexible to run on multiple instance types and multiple availability zones; hence leveraging multiple *spot capacity pools* and making the most out of the available spare capacity. To select a list of instance types for your workload you can use [Spot Instance Advisor](https://aws.amazon.com/ec2/spot/instance-advisor/) which will help you filtering suitable instances by number of vCPUs and amount of memory and also provide you data about the interruption rate during the last 30 days for each instance type, so you can pick a list of best-suited instance types with low interruption rates.
+The **Overrides** configuration block provides EC2 AutoScaling the list of instance types your workload can run on. As Spot instances are **spare** EC2 capacity, your workloads should be flexible to run on multiple instance types and multiple availability zones; hence leveraging multiple *spot capacity pools* and making the most out of the available spare capacity. You can use the [EC2 Instance Types console](https://console.aws.amazon.com/ec2/v2/home?#InstanceTypes:) or the [ec2-instance-selector](https://github.com/aws/amazon-ec2-instance-selector) CLI tool to find suitable instance types. To adhere to best practices and maximize your chances of launching your target Spot capacity, configure a minimum of 6 different instance types across 3 Availability Zones). That would give you the ability to provision Spot capacity from 18 different capacity pools. 
 
-Then, the *InstancesDistribution* configuration block determines how EC2 AutoScaling picks the instance types to use, while at the same time it keeps a balanced number of EC2 instances per Availability Zone.
+Then, the *InstancesDistribution* configuration block determines how EC2 Auto Scaling picks the instance types to use, while at the same time it keeps a balanced number of EC2 instances per Availability Zone.
 
-* The **prioritized** allocation strategy for on-demand instances makes AutoScaling try to launch the first instance type of your list and skip to the next instance type if for any reason it's unable to launch it (e.g. temporary unavailability of capacity). This is particularly useful if you have Reserved Instances for your baseline capacity, so AutoScaling launches the instance type matching your reservations. 
+* The **prioritized** allocation strategy for on-demand instances makes AutoScaling try to launch the first instance type of your list and skip to the next instance type if for any reason it's unable to launch it (e.g. temporary unavailability of capacity). This is particularly useful if you have [Reserved Instances](https://aws.amazon.com/ec2/pricing/reserved-instances/) or [Savings Plans](https://aws.amazon.com/savingsplans/) for your baseline capacity, so Auto Scaling launches the instance type matching your reservations. 
 * **OnDemandBaseCapacity** is set to 2, meaning the first two EC2 instances launched by EC2 AutoScaling will be on-demand.
 * **OnDemandPercentageAboveBaseCapacity** is set to 0 so all the additional instances will be launched as Spot Instances
-* **SpotAllocationStrategy** is lowest-price, which instructs AutoScaling to pick the cheapest instance type on each Availability Zone.
-* **SpotInstancePools** is 4, which tells AutoScaling to launch instances across the 4 cheapest instance types on each Availability Zone for the list of instances provided on the overrides; diversifying our fleet acquiring capacity from multiple *Spot pools* and reducing the likelihood of a large portion of our fleet to be interrupted in a short period of time. 
+* **SpotAllocationStrategy** is capacity-optimized, which instructs AutoScaling to pick the optimal instance type on each Availability Zone based on launch time availability of spare capacity for your instance type selection.
 {{% /expand %}}
 
 1. You will notice there are placeholder values for **%TargetGroupArn%**, **%publicSubnet1%** and **%publicSubnet2%**. To update the configuration file with the values of the Target Group you created previously and the outputs from the CloudFormation template, execute the following command:
@@ -34,7 +35,6 @@ This command will not return any output if it is successful.
 
 1. Browse to the [Auto Scaling console](https://console.aws.amazon.com/ec2/autoscaling/home#AutoScalingGroups:view=details) and check out your newly created auto scaling group. Take a look at the instances it has deployed.
 
+# Optional exercise
 
-## Optional exercise
-
-Now that you have deployed an EC2 AutoScaling group with Mixed Instance Types and Purchase Options, take some time to manually scale out and scale in the number of instances of the group and see which instance types AutoScaling launches. Also, modify the SpotInstancePools parameter and experiment with the [capacity-optimized](https://aws.amazon.com/blogs/compute/introducing-the-capacity-optimized-allocation-strategy-for-amazon-ec2-spot-instances/) allocation strategy to get a good grasp of how the different allocation strategies behave. 
+Now that you have deployed an EC2 Auto Scaling group with Mixed Instance Types and Purchase options, take some time to go through the different configurations in the [console](https://console.aws.amazon.com/ec2autoscaling/home?#/). Click on the **myEC2Workshop** Auto Scaling group and go to the *Purchase options and instance types* section and try to edit the instance types configured on the Auto Scaling group and change the "primary instance type" to see how the Auto Scaling console provides you a recommended list of instance types based on your selected instance type.

content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/create_lt.md

@@ -3,15 +3,11 @@ title = "Create an EC2 launch template"
 weight = 70
 +++
 
-EC2 Launch Templates reduce the number of steps required to create an instance by capturing all launch parameters within one resource. 
+A launch template specifies EC2 instance configuration information: the ID of the Amazon Machine Image (AMI), the instance type, a key pair, security groups, and the other parameters that you use to launch EC2 instances. This configuration can later be used to launch instances from that template via the EC2 API, via EC2 Auto Scaling groups and other AWS services. Launch Templates are similar to Auto Scaling [launch configurations](https://docs.aws.amazon.com/autoscaling/ec2/userguide/LaunchConfiguration.html); however, defining a launch template instead of a launch configuration allows you to have multiple versions of a template. With versioning, you can create a subset of the full set of parameters and then reuse it to create other templates or template versions. For example, you can create a default template that defines common configuration parameters and allow the other parameters to be specified as part of another version of the same template.
 
-You can create a launch template that contains the configuration information to launch an instance. Launch templates enable you to store launch parameters so that you do not have to specify them every time you launch an instance. For example, a launch template can contain the AMI ID, instance type, and network settings that you typically use to launch instances. When you launch an instance using the Amazon EC2 console, an AWS SDK, or a command line tool, you can specify the launch template to use.
+It's recommended that you create Auto Scaling groups from launch templates to ensure that you're accessing the latest features and improvements. Note that not all Auto Scaling group features are available in Launch Configurations. For example, with launch configurations, you cannot create an Auto Scaling group that launches both Spot and On-Demand Instances or that specifies multiple instance types or multiple launch templates. You must use a launch template to configure these features. For more information, see [Auto Scaling groups with multiple instance types and purchase options](https://docs.aws.amazon.com/autoscaling/ec2/userguide/asg-purchase-options.html). 
 
-{{% notice note %}}
-You might be wondering how a [Launch Template](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-launch-templates.html) is different from a [Launch Configuration](https://docs.aws.amazon.com/autoscaling/ec2/userguide/LaunchConfiguration.html). They are similar in that they both specify instance configuration information; however Launch Templates provide additional features like versioning and enable you to use the latest features of Amazon EC2 and Auto Scaling Groups with multiple instance types and purchase options.  You can learn more about Launch Templates [here](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-launch-templates.html)
-{{% /notice %}}
-
-You'll use a launch template to specify configuration parameters for launching instances in this workshop.
+You'll use a Launch Template to specify configuration parameters for launching instances with EC2 Auto Scaling in this workshop.
 
 1. Open **launch-template-data.json** on the Cloud9 editor and examine the configuration, you will notice some of the parameters have a placeholder value **%variableName%**: %instanceProfile%, %instanceSecurityGroup% and %ami-id%.
 ![Cloud9 Editor](/images/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/cloud9-editor.jpg)

content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/setup_cli.md

@@ -3,14 +3,26 @@ title = "Setup AWS CLI and other tools"
 weight = 50
 +++
 
-1. Make sure the latest version of the AWS CLI is installed by running:
+1. Uninstall the AWS CLI 1.x by running:
+	```bash
+	sudo pip uninstall -y awscli
+	```
+
+1. Install the AWS CLI 2.0 by running:
 
+	```bash
+	curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
+	unzip awscliv2.zip
+	sudo ./aws/install
 	```
-	sudo pip install -U awscli
+
+1. Confirm the CLI version with the following command:
+	```bash
+	aws --version
 	```
-	
+
 1. Install dependencies for use in the workshop by running:
 
-	```
+	```bash
 	sudo yum -y install jq
 	```