diff --git a/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/_index.md b/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/_index.md index d9841f42..83c97e68 100644 --- a/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/_index.md +++ b/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/_index.md @@ -1,7 +1,7 @@ --- title: "EC2 Auto Scaling with multiple instance types and purchase options" date: 2019-01-26T00:00:00Z -weight: 90 +weight: 10 pre: "⁃ " --- diff --git a/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/cleanup.md b/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/cleanup.md index 9b20f323..d3459c0c 100644 --- a/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/cleanup.md +++ b/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/cleanup.md @@ -16,16 +16,23 @@ If you're running in your own account, make sure you run through these steps to aws elbv2 delete-load-balancer --load-balancer-arn $LoadBalancerArn - aws elbv2 delete-target-group --target-group-arn $TargetGroupArn - aws ec2 delete-launch-template --launch-template-name myEC2Workshop - aws cloudformation delete-stack --stack-name spotinterruptionhandler + ``` +1. Delete the Target Group created (you need to wait until the Application Load Balancer has been completely deleted). + ``` + aws elbv2 delete-target-group --target-group-arn $TargetGroupArn ``` 1. Finally, delete the CloudFormation stack itself. ``` aws cloudformation delete-stack --stack-name $stack_name + ``` + +1. If you ran the optional Custom Spot Interruption handling exercise, make sure you remove the Spot Interruption handler CloudFormation template deployed by the Serverless Application Repository. + + ``` + aws cloudformation delete-stack --stack-name serverlessrepo-ec2-spot-interruption-handler ``` \ No newline at end of file diff --git a/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/create_asg.md b/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/create_asg.md index 86b045c4..f9e60cad 100644 --- a/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/create_asg.md +++ b/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/create_asg.md @@ -5,17 +5,18 @@ weight = 100 Amazon EC2 Auto Scaling helps you maintain application availability and allows you to dynamically scale your Amazon EC2 capacity up or down automatically according to conditions you define. You can use Amazon EC2 Auto Scaling for fleet management of EC2 instances to help maintain the health and availability of your fleet and ensure that you are running your desired number of Amazon EC2 instances. You can also use Amazon EC2 Auto Scaling for dynamic scaling of EC2 instances in order to automatically increase the number of Amazon EC2 instances during demand spikes to maintain performance and decrease capacity during lulls to reduce costs. Amazon EC2 Auto Scaling is well suited both to applications that have stable demand patterns or that experience hourly, daily, or weekly variability in usage. -1. Open **asg.json** on the Cloud9 editor and review the configuration. Pay special attention at the **Overrides** and the **InstancesDistribution** configuration blocks and try to guess how many instances of which instance type and which purchase option will be launched. Take a look at our [docs](https://docs.aws.amazon.com/autoscaling/ec2/userguide/asg-purchase-options.html#asg-allocation-strategies) to review how InstancesDistribution and allocation strategies work. +Amazon EC2 Auto Scaling also allows you to combine purchase options and instance types so you can optimize your compute costs. Stateless web applications are a great fit to run on Spot Instances as they can tolerate interruptions and are often flexible to run on multiple instance types. In this section, you will create an Auto Scaling group combining a base of On-Demand instances and scaling out with EC2 Spot instances and save an average of 70% in your compute costs. + +1. Open **asg.json** on the Cloud9 editor and review the configuration. Pay special attention at the **Overrides** and the **InstancesDistribution**. Take a look at our [docs](https://docs.aws.amazon.com/autoscaling/ec2/userguide/asg-purchase-options.html#asg-allocation-strategies) to review how InstancesDistribution and allocation strategies work. You will also notice that the **CapacityRebalance** parameter is set to true, which will proactively attempt to replace Spot Instances at elevated risk of interruption. To learn more about the Capacity Relabancing feature, take a look at the [docs](https://docs.aws.amazon.com/autoscaling/ec2/userguide/capacity-rebalance.html). {{%expand "Help me understand the AutoScaling configuration" %}} -The **Overrides** configuration block provides EC2 AutoScaling the list of instance types your workload can run on. As Spot instances are **spare** EC2 capacity, your workloads should be flexible to run on multiple instance types and multiple availability zones; hence leveraging multiple *spot capacity pools* and making the most out of the available spare capacity. To select a list of instance types for your workload you can use [Spot Instance Advisor](https://aws.amazon.com/ec2/spot/instance-advisor/) which will help you filtering suitable instances by number of vCPUs and amount of memory and also provide you data about the interruption rate during the last 30 days for each instance type, so you can pick a list of best-suited instance types with low interruption rates. +The **Overrides** configuration block provides EC2 AutoScaling the list of instance types your workload can run on. As Spot instances are **spare** EC2 capacity, your workloads should be flexible to run on multiple instance types and multiple availability zones; hence leveraging multiple *spot capacity pools* and making the most out of the available spare capacity. You can use the [EC2 Instance Types console](https://console.aws.amazon.com/ec2/v2/home?#InstanceTypes:) or the [ec2-instance-selector](https://github.com/aws/amazon-ec2-instance-selector) CLI tool to find suitable instance types. To adhere to best practices and maximize your chances of launching your target Spot capacity, configure a minimum of 6 different instance types across 3 Availability Zones). That would give you the ability to provision Spot capacity from 18 different capacity pools. -Then, the *InstancesDistribution* configuration block determines how EC2 AutoScaling picks the instance types to use, while at the same time it keeps a balanced number of EC2 instances per Availability Zone. +Then, the *InstancesDistribution* configuration block determines how EC2 Auto Scaling picks the instance types to use, while at the same time it keeps a balanced number of EC2 instances per Availability Zone. -* The **prioritized** allocation strategy for on-demand instances makes AutoScaling try to launch the first instance type of your list and skip to the next instance type if for any reason it's unable to launch it (e.g. temporary unavailability of capacity). This is particularly useful if you have Reserved Instances for your baseline capacity, so AutoScaling launches the instance type matching your reservations. +* The **prioritized** allocation strategy for on-demand instances makes AutoScaling try to launch the first instance type of your list and skip to the next instance type if for any reason it's unable to launch it (e.g. temporary unavailability of capacity). This is particularly useful if you have [Reserved Instances](https://aws.amazon.com/ec2/pricing/reserved-instances/) or [Savings Plans](https://aws.amazon.com/savingsplans/) for your baseline capacity, so Auto Scaling launches the instance type matching your reservations. * **OnDemandBaseCapacity** is set to 2, meaning the first two EC2 instances launched by EC2 AutoScaling will be on-demand. * **OnDemandPercentageAboveBaseCapacity** is set to 0 so all the additional instances will be launched as Spot Instances -* **SpotAllocationStrategy** is lowest-price, which instructs AutoScaling to pick the cheapest instance type on each Availability Zone. -* **SpotInstancePools** is 4, which tells AutoScaling to launch instances across the 4 cheapest instance types on each Availability Zone for the list of instances provided on the overrides; diversifying our fleet acquiring capacity from multiple *Spot pools* and reducing the likelihood of a large portion of our fleet to be interrupted in a short period of time. +* **SpotAllocationStrategy** is capacity-optimized, which instructs AutoScaling to pick the optimal instance type on each Availability Zone based on launch time availability of spare capacity for your instance type selection. {{% /expand %}} 1. You will notice there are placeholder values for **%TargetGroupArn%**, **%publicSubnet1%** and **%publicSubnet2%**. To update the configuration file with the values of the Target Group you created previously and the outputs from the CloudFormation template, execute the following command: @@ -34,7 +35,6 @@ This command will not return any output if it is successful. 1. Browse to the [Auto Scaling console](https://console.aws.amazon.com/ec2/autoscaling/home#AutoScalingGroups:view=details) and check out your newly created auto scaling group. Take a look at the instances it has deployed. +# Optional exercise -## Optional exercise - -Now that you have deployed an EC2 AutoScaling group with Mixed Instance Types and Purchase Options, take some time to manually scale out and scale in the number of instances of the group and see which instance types AutoScaling launches. Also, modify the SpotInstancePools parameter and experiment with the [capacity-optimized](https://aws.amazon.com/blogs/compute/introducing-the-capacity-optimized-allocation-strategy-for-amazon-ec2-spot-instances/) allocation strategy to get a good grasp of how the different allocation strategies behave. \ No newline at end of file +Now that you have deployed an EC2 Auto Scaling group with Mixed Instance Types and Purchase options, take some time to go through the different configurations in the [console](https://console.aws.amazon.com/ec2autoscaling/home?#/). Click on the **myEC2Workshop** Auto Scaling group and go to the *Purchase options and instance types* section and try to edit the instance types configured on the Auto Scaling group and change the "primary instance type" to see how the Auto Scaling console provides you a recommended list of instance types based on your selected instance type. \ No newline at end of file diff --git a/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/create_lt.md b/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/create_lt.md index 72967167..a69bc8d3 100644 --- a/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/create_lt.md +++ b/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/create_lt.md @@ -3,15 +3,11 @@ title = "Create an EC2 launch template" weight = 70 +++ -EC2 Launch Templates reduce the number of steps required to create an instance by capturing all launch parameters within one resource. +A launch template specifies EC2 instance configuration information: the ID of the Amazon Machine Image (AMI), the instance type, a key pair, security groups, and the other parameters that you use to launch EC2 instances. This configuration can later be used to launch instances from that template via the EC2 API, via EC2 Auto Scaling groups and other AWS services. Launch Templates are similar to Auto Scaling [launch configurations](https://docs.aws.amazon.com/autoscaling/ec2/userguide/LaunchConfiguration.html); however, defining a launch template instead of a launch configuration allows you to have multiple versions of a template. With versioning, you can create a subset of the full set of parameters and then reuse it to create other templates or template versions. For example, you can create a default template that defines common configuration parameters and allow the other parameters to be specified as part of another version of the same template. -You can create a launch template that contains the configuration information to launch an instance. Launch templates enable you to store launch parameters so that you do not have to specify them every time you launch an instance. For example, a launch template can contain the AMI ID, instance type, and network settings that you typically use to launch instances. When you launch an instance using the Amazon EC2 console, an AWS SDK, or a command line tool, you can specify the launch template to use. +It's recommended that you create Auto Scaling groups from launch templates to ensure that you're accessing the latest features and improvements. Note that not all Auto Scaling group features are available in Launch Configurations. For example, with launch configurations, you cannot create an Auto Scaling group that launches both Spot and On-Demand Instances or that specifies multiple instance types or multiple launch templates. You must use a launch template to configure these features. For more information, see [Auto Scaling groups with multiple instance types and purchase options](https://docs.aws.amazon.com/autoscaling/ec2/userguide/asg-purchase-options.html). -{{% notice note %}} -You might be wondering how a [Launch Template](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-launch-templates.html) is different from a [Launch Configuration](https://docs.aws.amazon.com/autoscaling/ec2/userguide/LaunchConfiguration.html). They are similar in that they both specify instance configuration information; however Launch Templates provide additional features like versioning and enable you to use the latest features of Amazon EC2 and Auto Scaling Groups with multiple instance types and purchase options. You can learn more about Launch Templates [here](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-launch-templates.html) -{{% /notice %}} - -You'll use a launch template to specify configuration parameters for launching instances in this workshop. +You'll use a Launch Template to specify configuration parameters for launching instances with EC2 Auto Scaling in this workshop. 1. Open **launch-template-data.json** on the Cloud9 editor and examine the configuration, you will notice some of the parameters have a placeholder value **%variableName%**: %instanceProfile%, %instanceSecurityGroup% and %ami-id%. ![Cloud9 Editor](/images/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/cloud9-editor.jpg) diff --git a/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/setup_cli.md b/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/setup_cli.md index 7c0be744..23010441 100644 --- a/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/setup_cli.md +++ b/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/setup_cli.md @@ -3,14 +3,26 @@ title = "Setup AWS CLI and other tools" weight = 50 +++ -1. Make sure the latest version of the AWS CLI is installed by running: +1. Uninstall the AWS CLI 1.x by running: + ```bash + sudo pip uninstall -y awscli + ``` + +1. Install the AWS CLI 2.0 by running: + ```bash + curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" + unzip awscliv2.zip + sudo ./aws/install ``` - sudo pip install -U awscli + +1. Confirm the CLI version with the following command: + ```bash + aws --version ``` - + 1. Install dependencies for use in the workshop by running: - ``` + ```bash sudo yum -y install jq ``` \ No newline at end of file diff --git a/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/spot_resilience.md b/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/spot_resilience.md index e8d20f6d..8846dab2 100644 --- a/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/spot_resilience.md +++ b/content/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/spot_resilience.md @@ -4,34 +4,75 @@ weight = 155 +++ ### Handling Spot Interruptions -When EC2 needs the capacity back in a specific capacity pool (a combination of an instance type in an Availability Zone) it could start interrupting the Spot Instances that are running in that AZ, by sending a 2 minute interruption notification, and then terminating the instance. The 2 minute interruption notification is delivered via [EC2 instance meta-data](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html#spot-instance-termination-notices) as well as CloudWatch Events. -Let's deploy a Lambda function that would catch the CloudWatch event for `EC2 Spot Instance Interruption Warning` and automatically detach the soon-to-be-terminated instance from the EC2 Auto Scaling group. +When EC2 needs the capacity back in a specific capacity pool (a combination of an instance type in an Availability Zone) it can start interrupting Spot Instances that are running in that AZ, by sending a 2 minute interruption notification, and then terminating the instance. The 2 minute interruption notification is delivered via the [EC2 instance meta-data service](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html#spot-instance-termination-notices) as well as [Amazon EventBridge](https://aws.amazon.com/eventbridge/). + +Since November 5th, 2020, EC2 Spot instances also get a notification when they are at elevated risk of interruption via a new signal called [EC2 Rebalance Recommendation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/rebalance-recommendations.html). This signal can arrive sooner than the two-minute Spot Instance interruption notice, giving you more headroom to react to that upcoming interruption. You can decide to rebalance your workload to new or existing Spot Instances that are not at an elevated risk of interruption. + +With the release of EC2 Rebalance Recommendations, EC2 Auto Scaling released a feature called Capacity Rebalaning that, if enabled, whenever a Spot instance in your Auto Scaling group is at an elevated risk of interruption, it will proactively attempt to launch a replacement Spot Instance. For the feature to work as expected it's highly recommended that you configure multiple instance types in your Auto Scaling group, and that you use the capacity-optimized allocation strategy. In this case, Auto Scaling will launch the optimal instance type out of your selection of instances based on spare capacity availability at launch time. Once the replacement instance is running and passing health checks, Auto Scaling will then terminate the Spot Instance at an elevated risk of interruption, triggering the configured [deregistration delay](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-target-groups.html#deregistration-delay) from the load balancer (if you have a [Target Group](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-target-groups.html) configured on your Auto Scaling group), and executing termination [lifecycle hooks](https://docs.aws.amazon.com/autoscaling/ec2/userguide/lifecycle-hooks.html) if they are set up (Auto Scaling lifecycle hooks allows you to execute actions before an instance is put in service, and/or before it's terminated). The diagram below reflects the timeline of a Capacity Rebalancing activity: + +![Capacity Rebalancing Diagram](/images/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/capacity-rebalancing-diagram.png) + +During the Auto Scaling group configuration, we have already enabled Capacity Rebalancing, configured multiple instance types and Availability Zones and the capacity-optimized allocation strategy, so we are all set. If you want to review these configurations again, inspect the asg.json file or go to the Auto Scaling console and check the *Purchase options and instance types* section of your Auto Scaling group. + +![Auto Scaling console](/images/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/asg-purchase-options-and-instance-types.png) + +Note that when we have configured the Application Load Balancer, we have also configured the deregistration delay of the target group to 90 seconds, so the deregistration of the instance from the ALB completes within the 2 minutes notice. This is because it is not always possible for Amazon EC2 to send the rebalance recommendation signal before the two-minute Spot Instance interruption notice. Therefore, the rebalance recommendation signal can arrive along with the two-minute interruption notice. + +#### Knowledge check +How can you increase the resilience of the web application that you deployed in this workshop, when using EC2 Spot Instances? + +{{%expand "Click here for the answer" %}} +1. Add an Availability Zone - the EC2 Auto Scaling group is currently deployed in two AZs. By adding an AZ to your application, you will tap into more EC2 Spot capacity pools. +2. Add Instance Types - the 6 instance types that are configured in the Auto Scaling group have small performance variability, so it's possible to run all these instance types in a single ASG and scale on the same dynamic scaling policy. Are there any other instance types that can be added? +{{% /expand %}} + +#### Challenges + +* By default, a Target Group linked to an Application Load Balancer distributes the requests across its registered instances using a Round Robin load balancing algorithm. Is there anything you could do to spread the load more efficiently if you have backend instances from different instance families that may have slight differences on processing power? Take a look at [this](https://aws.amazon.com/about-aws/whats-new/2019/11/application-load-balancer-now-supports-least-outstanding-requests-algorithm-for-load-balancing-requests/) article. + +{{%expand "Click here for the answer" %}} +If your web application is sensitive to differences in processing power of different instance types you can use the Least Outstanding Requests load balancing algorithm. With this algorithm, as the new request comes in, the load balancer will send it to the target with least number of outstanding requests. Targets processing long-standing requests or having lower processing capabilities are not burdened with more requests and the load is evenly spread across targets. This also helps the new targets to effectively take load off of overloaded targets. You can configure the routing algorithm on the [Target Group section](https://console.aws.amazon.com/ec2/v2/home?#TargetGroups:sort=targetGroupName) within the EC2 console selecting your target group and clicking *Actions* -> *Edit Attributes* +{{% /expand %}} + +### Optional exercise: Custom Spot interruption notification handling + +In this workshop, you deployed a simple stateless web application and leveraged Capacity Rebalancing for handling the Spot Instance lifecycle. Capacity Rebalancing works great for this use case, as it will automatically take care of bringing up replacement capacity when Spot instances are at an elevated risk of interruption and gracefully attempt to finish in-flight requests coming from the Application Load Balancer. + +For other EC2 Auto Scaling workloads, like queue workers or other similar batch processes, you may prefer to execute actions when the EC2 Rebalance Recommendation signal is issued (like stop consuming new jobs) but to not terminate the instance until in-flight job processing has finished and/or act only when the two-minute instance termination warning arrives. For these cases, you can build your own handling logic by handling Amazon EventBridge --> AWS Lambda and / or a local script that consumes the EC2 metadata service notifications. Take some time to explore options or take a look at [this sample solution](https://github.com/awslabs/ec2-spot-labs/tree/master/ec2-spot-interruption-handler). + +{{%expand "Click here for instructions to deploy the custom EC2 Spot Interruption Handler" %}} +In this exercise, you will deploy an Amazon EventBridge rule to catch the `EC2 Spot Instance Interruption warning` events and an AWS Lambda function to automatically detach the soon-to-be-terminated instance from the EC2 Auto Scaling group. + By calling the [DetachInstances](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_DetachInstances.html) API you achieve two things: 1. You can specify on the API call whether to keep the current desired capacity on the Auto Scaling group, or decrement it by the number of instances being detached. By keeping the same desired capacity, Auto Scaling will immediately launch a replacement instance. - 1. If the Auto Scaling group has a Load Balancer or a Target Group attached (as we have in this workshop), the instance is deregistered from it. Also, if connection draining is enabled for your Load Balancer (or Target Group), the Auto Scaling group waits for in-flight requests to complete (up to the configured timeout, which we have set up to 120 sec). + 1. If the Auto Scaling group has a Load Balancer or a Target Group attached (as we have in this workshop), the instance is deregistered from it. Also, if connection draining is enabled for your Load Balancer (or Target Group), the Auto Scaling group waits for in-flight requests to complete (up to the configured timeout, which we have set up to 90 sec). You can learn more about the Detaching EC2 Instances from an Auto Scaling group [here](https://docs.aws.amazon.com/autoscaling/ec2/userguide/detach-instance-asg.html). +We provide you with a sample EC2 Spot Interruption handler [here](https://github.com/awslabs/ec2-spot-labs/tree/master/ec2-spot-interruption-handler) that you can easily deploy via the Serverless Application Repository. -To save time, we will use a CloudFormation template to deploy the Lambda Function that will handle EC2 Spot interruptions, and the CloudWatch event rule to catch the Spot Interruption notifications, and subscribe the Lambda Function to it. + 1. Go to the `Available applications` section of the [Serverless Application Repository console](https://console.aws.amazon.com/serverlessrepo/home#/available-applications) - 1. Take some time to review the CloudFormation template and understand what will be launched. Then, execute the following command to deploy the template: + 1. Under `Public applications` section, mark the checkbox *Show apps that create custom IAM roles or resource policies* and type `ec2-spot-interruption-handler`, then click on the application. You can also access the application directly clicking [this link](https://console.aws.amazon.com/lambda/home?#/create/app?applicationId=arn:aws:serverlessrepo:eu-west-1:310006123715:applications/ec2-spot-interruption-handler) - ``` - aws cloudformation deploy --template-file spot-interruption-handler.yaml --stack-name spotinterruptionhandler --capabilities CAPABILITY_IAM - ``` + 1. Scroll to the bottom and on the `Application settings` section, leave the default settings and mark the checkbox *I acknowledge that this app creates custom IAM roles and resource policies*. Then click `Deploy`. - 1. When the CloudFormation deployment completes (under 2 minutes), open the [AWS Lambda console](https://console.aws.amazon.com/lambda/home) and click on the newly deployed Function name. - - 1. Feel free to examine the code in the Inline code editor. + 1. Allow a couple of minutes for the application to be deployed. Take this time to browse the solution details on [GitHub](https://github.com/awslabs/ec2-spot-labs/tree/master/ec2-spot-interruption-handler). +The serverless application has configured the following resources: -Now our infrastructure is ready to respond to Spot Interruptions by detaching Spot Instances from the Auto Scaling group when they receive a Spot interruption notification. We can't simulate an EC2 Spot Interruption, but we can invoke the Lambda Function with a simulation of a CloudWatch event for an EC2 Spot Instance Interruption Warning, and see the result. +* An AWS Lambda function that receives Spot Interruption notifications, checks if the instance is part of an Auto Scaling group (and if it's been tagged with Key: `SpotInterruptionHandler/enabled` Value: `true`) and to then call the [DetachInstances API](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_DetachInstances.html) of EC2 Auto Scaling to start connection draining and launch a replacement instance. Feel free to go to the [AWS Lambda console](https://console.aws.amazon.com/lambda/home#/functions) and inspect the `SpotInterruptionHandler` function. +* An Amazon EventBridge event rule to capture Spot Instance interruption notifications and trigger the above Lambda function. Feel free to go to the [Amazon EventBridge console](https://console.aws.amazon.com/events/home#/rules) to inspect the rule. +* An IAM role for the Lambda function to be able to call the AWS API. - 1. In the top right corner of the AWS Lambda console, click the dropdown menu **Select a test event** -> **Configure test events** +If you check the Auto Scaling group instances, you will see they've been already tagged appropriately for SpotInterruptionHandler to take actions if one of its Spot Instance is interrupted (check the asg.json file and you'll see the tag there). At this stage, our infrastructure is ready to respond to Spot Interruptions. + +We can't simulate an actual EC2 Spot Interruption, but we can invoke the Lambda Function with a simulated EC2 Spot Instance Interruption Warning event for one of or Auto Scaling group instances, and see the result. + + 1. Go to the [AWS Lambda console](https://console.aws.amazon.com/lambda/home#/functions) and open the SpotInterruptionHandler function. In the top right corner, click the dropdown menu **Select a test event** -> **Configure test events** 1. With **Create a new test event** selected, provide an Event name (i.e TestSpotInterruption). In the event text box, paste the following: ```json @@ -53,12 +94,12 @@ Now our infrastructure is ready to respond to Spot Interruptions by detaching Sp } ``` - 1. Replace both occurrences of **"\"** with the instance-id of one of the Spot Instances that are currently running in your EC2 Auto Scaling group (you can get an instance-id from the Instances tab in the bottom pane of the [EC2 Auto Scaling groups console](https://console.aws.amazon.com/ec2/autoscaling/home#AutoScalingGroups:view=details) ). You don't need to change any of the other parameters in the event json. + 1. Replace both occurrences of **"\"** with the instance-id of one of the Spot Instances that are currently running in your EC2 Auto Scaling group (you can get an instance-id from the `Instance management` tab in the bottom pane of the [EC2 Auto Scaling groups console](https://console.aws.amazon.com/ec2autoscaling/home) ). You don't need to change any of the other parameters in the event json. 1. Click **Create** 1. With your new test name (i.e TestSpotInterruption) selected in the dropdown menu, click the **Test** button. - 1. The execution result should be **succeeded** and you can expand the details to see the successful log message: "Instance i-01234567890123456 belongs to AutoScaling Group runningAmazonEC2WorkloadsAtScale. Detaching instance..." - 1. Go back to the [EC2 Auto Scaling groups console](https://console.aws.amazon.com/ec2/autoscaling/home#AutoScalingGroups:view=details), and under the **Activity History** tab in the bottom pane, you should see a **Detaching EC2 instance** activity, followed shortly after by a **Launching a new EC2 instance** activity. - 1. Go to the [EC2 ELB Target Groups console](https://console.aws.amazon.com/ec2/v2/home?1#TargetGroups:sort=targetGroupName) and click on the **runningAmazonEC2WorkloadsAtScale** Target Group, go to the Targets tab in the bottom pane, you should see the instance in `draining` mode. + 1. The execution result should be **succeeded** and you can expand the details to see the successful log message: "Interruption response actions completed for instance i-0156bedcef61187e8 belonging to runningAmazonEC2WorkloadsAtScale" + 1. Go back to the [EC2 Auto Scaling groups console](https://console.aws.amazon.com/ec2autoscaling/home), click on the myEC2Workshop Auto Scaling group and under the **Activity** tab in the bottom pane, you should see a **Detaching EC2 instance** activity, followed shortly after by a **Launching a new EC2 instance** activity. Notice the status `WaitingForELBConnectionDraining` on the detached instance. You will find the deregistration timeout configured on the `modify-target-group.json` file. + 1. Go to the [EC2 ELB Target Groups console](https://console.aws.amazon.com/ec2/v2/home#TargetGroups:sort=targetGroupName) and click on the **myEC2Workshop** Target Group, go to the Targets tab in the bottom pane, you should see the instance in `draining` status. Great result! by leveraging the EC2 Spot Instance Interruption Warning, the Lambda Function detached the instance from the Auto Scaling group and the ELB Target Group, thus draining existing connections, and launching a replacement instance before the current instance is terminated. @@ -66,30 +107,4 @@ Great result! by leveraging the EC2 Spot Instance Interruption Warning, the Lamb In a real scenario, EC2 would terminate the instance after two minutes, however in this case we simply mocked up the interruption so the EC2 instance will keep running outside the Auto Scaling group. Go to the EC2 console and terminate the instance that you used on the mock up event. {{% /notice %}} - -### Increasing the application's resilience when using Spot Instances - -In a previous step in this workshop, you learned that the EC2 Auto Scaling group is configured to fulfill the 4 lowest-priced instance types, out of a list of 9 types, in each Availability Zone. Since Spot is spare EC2 capacity, its supply and demand vary. By diversifying your usage of capacity pools (which are a combination of an instance type in an Availability Zone), you increase your chances of getting the desired capacity, and decrease the potential number of interruptions in case Spot Instances are interrupted when EC2 needs the capacity back for On-Demand. - -#### Knowledge check -How can you increase the resilience of the web application that you deployed in this workshop, when using EC2 Spot Instances? - -{{%expand "Click here for the answer" %}} -1. Add an Availability Zone - the EC2 Auto Scaling group is currently deployed in two AZs. By adding an AZ to your application, you will tap into more EC2 Spot capacity pools, further diversifying your usage and decreasing the blast radius in case a Spot interruption occurs in one of the Spot capacity pools. -2. Add Instance Types - the 6 instance types that are configured in the Auto Scaling group have small performance variability, so it's possible to run all these instance types in a single ASG and scale on the same dynamic scaling policy. Are there any other instance types that can be added? -3. Increase SpotInstancePools - The ASG is configured to fulfill the 4 lowest-priced instance types. Increase this number to further diversify the usage of capacity pools and decrease the blast radius. -{{% /expand %}} - -#### Challenges -* What other Spot allocation strategy can you choose, would it be suitable for this workload? if not, when will you use it?\ -Hint: read through the following [article] (https://aws.amazon.com/blogs/compute/introducing-the-capacity-optimized-allocation-strategy-for-amazon-ec2-spot-instances/) - -{{%expand "Click here for the answer" %}} -If you have workloads that are not stateless and fault-tolerant like the Web application that you deployed in this workshop, you can use the capacity-optimized allocation strategy, in order to instruct the ASG to launch instances in the capacity pools which are least likely to be interrupted. -{{% /expand %}} - -* By default, a Target Group linked to an Application Load Balancer distributes the requests across its registered instances using a Round Robin load balancing algorithm. Is there anything you could do to spread the load more efficiently if you have backend instances from different instance families that may have slight differences on processing power? Take a look at [this](https://aws.amazon.com/about-aws/whats-new/2019/11/application-load-balancer-now-supports-least-outstanding-requests-algorithm-for-load-balancing-requests/) article. - -{{%expand "Click here for the answer" %}} -If your web application is sensitive to differences in processing power of different instance types you can use the Least Outstanding Requests load balancing algorithm. With this algorithm, as the new request comes in, the load balancer will send it to the target with least number of outstanding requests. Targets processing long-standing requests or having lower processing capabilities are not burdened with more requests and the load is evenly spread across targets. This also helps the new targets to effectively take load off of overloaded targets. You can configure the routing algorithm on the [Target Group section](https://console.aws.amazon.com/ec2/v2/home?#TargetGroups:sort=targetGroupName) within the EC2 console selecting your target group and clicking *Actions* -> *Edit Attributes* {{% /expand %}} \ No newline at end of file diff --git a/content/running-amazon-ec2-workloads-at-scale/_index.md b/content/running-amazon-ec2-workloads-at-scale/_index.md index 70603587..ae1e399d 100644 --- a/content/running-amazon-ec2-workloads-at-scale/_index.md +++ b/content/running-amazon-ec2-workloads-at-scale/_index.md @@ -1,7 +1,7 @@ --- title: "Running EC2 Workloads at Scale with EC2 Auto Scaling" date: 2019-01-24T09:05:54Z -weight: 10 +weight: 90 pre: "⁃ " --- diff --git a/content/running_spark_apps_with_emr_on_spot_instances/cloud9-awscli.md b/content/running_spark_apps_with_emr_on_spot_instances/cloud9-awscli.md index 95d0eafb..4893acdc 100644 --- a/content/running_spark_apps_with_emr_on_spot_instances/cloud9-awscli.md +++ b/content/running_spark_apps_with_emr_on_spot_instances/cloud9-awscli.md @@ -1,5 +1,5 @@ --- -title: "Update to the latest AWS CLI" +title: "Configure your Cloud9 workspace" chapter: false weight: 20 comment: default install now includes aws-cli/1.15.83 @@ -9,12 +9,12 @@ comment: default install now includes aws-cli/1.15.83 For this workshop, please ignore warnings about the version of pip being used. {{% /notice %}} -1. Run the following command to view the current version of aws-cli: -``` -aws --version +1. Uninstall the AWS CLI 1.x by running: +```bash +sudo pip uninstall -y awscli ``` -1. Update to the latest version: +1. Install the AWS CLI 2.x by running the following command: ``` curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" unzip awscliv2.zip @@ -26,3 +26,10 @@ sudo ./aws/install ``` aws --version ``` + +1. Create an SSH Key pair so you can then SSH into the EMR cluster + +```bash +aws ec2 create-key-pair --key-name emr-workshop-key-pair --query "KeyMaterial" --output text > emr-workshop-key-pair.pem +chmod 400 emr-workshop-key-pair.pem +``` \ No newline at end of file diff --git a/content/running_spark_apps_with_emr_on_spot_instances/examining_cluster.md b/content/running_spark_apps_with_emr_on_spot_instances/examining_cluster.md index b4405e36..5247ac83 100644 --- a/content/running_spark_apps_with_emr_on_spot_instances/examining_cluster.md +++ b/content/running_spark_apps_with_emr_on_spot_instances/examining_cluster.md @@ -15,35 +15,65 @@ In this step, when you look at the utilization of the EMR cluster, do not expect {{% /notice %}} ### Using Ganglia, YARN ResourceManager and Spark History Server -The recommended approach to connect to the web interfaces running on our EMR cluster is to use SSH tunneling. [Click here] (https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-web-interfaces.html) to learn more about connecting to EMR interfaces. -For the purpose of this workshop, and since we started our EMR cluster in a VPC public subnet, we can allow access in the EC2 Security Group in order to reach the TCP ports on which the web interfaces are listening on. +To connect to the web interfaces running on our EMR cluster you need to use SSH tunneling. [Click here] (https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-web-interfaces.html) to learn more about connecting to EMR interfaces. -{{% notice warning %}} -Normally you would not run EMR in a public subnet and open TCP access to the master instance, this faster approach is just used for the purpose of the workshop. -{{% /notice %}} - -To allow access to your IP address to reach the EMR web interfaces via EC2 Security Groups: +First, we need to grant SSH access from the Cloud9 environment to the EMR cluster master node: 1. In your EMR cluster page, in the AWS Management Console, go to the **Summary** tab -2. Click on the ID of the security under **Security groups for Master** +2. Click on the ID of the security group in **Security groups for Master** 3. Check the Security Group with the name **ElasticMapReduce-master** -4. In the lower pane, click the **Inbound tab** and click the **Edit** -5. Click **Add Rule**. Under Type, select **All Traffic**, under Source, select **My IP** +4. In the lower pane, click the **Inbound tab** and click the **Edit inbound rules** +5. Click **Add Rule**. Under Type, select **SSH**, under Source, select **Custom**. As the Cloud9 environment and the EMR cluster are on the default VPC, introduce the CIDR of your Default VPC (e.g. 172.16.0.0/16). To check your VPC CIDR, go to the [VPC console](https://console.aws.amazon.com/vpc/home?#) and look for the CIDR of the **Default VPC**. 6. Click **Save** -{{% notice note %}} -While the Ganglia web interface uses TCP port 80, the YARN ResourceManager web interface uses TCP port 8088 and the Spark History Server uses TCP port 18080, which might not allowed for outbound traffic on your Internet connection. If you are using a network connection that blocks these ports (or in other words, doesn't allow non-well known ports) then you will not be able to reach the YARN ResourceManager web interface and Spark History Server. You can either skip that part of the workshop, use a different Internet connection (i.e mobile hotspot) or consider using the more complex method of SSH tunneling described [here] (https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-web-interfaces.html) -{{% /notice %}} +At this stage, we'll be able to ssh into the EMR master node. First we will access the Ganglia web interface to look at cluster metrics: -Go back to the Summary tab in your EMR cluster page, and you will see links to tools in the **Connections** section (you might need to refresh the page). -1. Click **Ganglia** to open the web interface. -2. Have a look around. Take notice of the heatmap (**Server Load Distribution**). Notable graphs are: +1. Go to the EMR Management Console, click on your cluster, and open the **Application user interfaces** tab. You'll see the list of on-cluster application interfaces. +2. Copy the master node DNS name from one of the interface urls, it will look like ec2.xx-xxx-xxx-xxx..compute.amazonaws.com +3. Establish an SSH tunnel to port 80, where Ganglia is bound, executing the below command on your Cloud9 environment (update the command with your master node DNS name): + + ``` + ssh -i ~/environment/emr-workshop-key-pair.pem -N -L 8080:ec2-###-##-##-###.compute-1.amazonaws.com:80 hadoop@ec2-###-##-##-###.compute-1.amazonaws.com + ``` + + You'll get a message saying the authenticity of the host can't be established. Type 'yes' and hit enter. The message will look similar to the following: + + ``` + The authenticity of host 'ec2-54-195-131-148.eu-west-1.compute.amazonaws.com (172.31.36.62)' can't be established. + ECDSA key fingerprint is SHA256:Cmv0qkh+e4nm5qir6a9fPN5DlgTUEaCGBN42txhShoI. + ECDSA key fingerprint is MD5:ee:63:d0:4a:a2:29:8a:c9:41:1b:a1:f0:f6:8e:68:4a. + Are you sure you want to continue connecting (yes/no)? + ``` + +4. Now, on your Cloud9 environment, click on the "Preview" menu on the top and then click on "Preview Running Application". You'll see a browser window opening on the environment with an Apache test page. on the URL, append /ganglia/ to access the Ganglia Interface. The url will look like https://xxxxxx.vfs.cloud9.eu-west-1.amazonaws.com/ganglia/. +![Cloud9-Ganglia](/images/running-emr-spark-apps-on-spot/cloud9-ganglia.png) +5. Click on the button next to "Browser" (arrow inside a box) to open Ganglia in a dedicated browser page.Have a look around. Take notice of the heatmap (**Server Load Distribution**). Notable graphs are: * **Cluster CPU last hour** - this will show you the CPU utilization that our Spark application consumed on our EMR cluster. you should see that utilization varied and reached around 70%. * **Cluster Memory last hour** - this will show you how much memory we started the cluster with, and how much Spark actually consumed. -3. Go back to the Summary page and click the **Resource Manager** link. -4. On the left pane, click **Nodes**, and in the node table, you should see the number of containers that each node ran. -5. Go back to the Summary page and click the **Spark History Server** link. -6. Click on the App ID in the table (where App Name = Amazon reviews word count) and go to the **Executors** tab -7. You can again see the number of executors that are running in your EMR cluster under the **Executors table** + +Now, let's look at the **Resource Manager** application user interface. + +1. Go to the Cloud9 terminal where you have established the ssh connection, and press ctrl+c to close it. +1. Create an SSH tunnel to the cluster master node on port 8088 by running this command (update the command with your master node DNS name): + + ``` + ssh -i ~/environment/emr-workshop-key-pair.pem -N -L 8080:ec2-###-##-##-###.compute-1.amazonaws.com:8088 hadoop@ec2-###-##-##-###.compute-1.amazonaws.com + ``` + +1. Now, on your browser, update the URL to "/cluster" i.e. https://xxxxxx.vfs.cloud9.eu-west-1.amazonaws.com/cluster +1. On the left pane, click Nodes, and in the node table, you should see the number of containers that each node ran. + +Now, let's look at **Spark History Server** application user interface: + +1. Go to the Cloud9 terminal where you have established the ssh connection, and press ctrl+c to close it. +1. Create an SSH tunnel to the cluster master node on port 18080 by running this command (update the command with your master node DNS name): + + ``` + ssh -i ~/environment/emr-workshop-key-pair.pem -N -L 8080:ec2-###-##-##-###.compute-1.amazonaws.com:18080 hadoop@ec2-###-##-##-###.compute-1.amazonaws.com + ``` + +1. Now, on your browser, go to the base URL of your Cloud9 environment i.e. https://xxxxxx.vfs.cloud9.eu-west-1.amazonaws.com/ +1. Click on the App ID in the table (where App Name = Amazon reviews word count) and go to the **Executors** tab +1. You can again see the number of executors that are running in your EMR cluster under the **Executors table** ### Using CloudWatch Metrics diff --git a/content/running_spark_apps_with_emr_on_spot_instances/fleet_config_options.md b/content/running_spark_apps_with_emr_on_spot_instances/fleet_config_options.md index d3d50f12..1db04746 100644 --- a/content/running_spark_apps_with_emr_on_spot_instances/fleet_config_options.md +++ b/content/running_spark_apps_with_emr_on_spot_instances/fleet_config_options.md @@ -8,14 +8,14 @@ While our cluster is starting (7-8 minutes) and the step is running (4-10 minute ![fleetconfigs](/images/running-emr-spark-apps-on-spot/emrinstancefleets-core1.png) #### Maximum Spot price -Since Amazon EC2 Spot Instances [changed the pricing model and bidding is no longer required] (https://aws.amazon.com/blogs/compute/new-amazon-ec2-spot-pricing/), we have an optional "Max-price" field for our Spot requests, which would limit how much we're willing to pay for the instance. It is recommended to leave this value at 100% of the On-Demand price, in order to avoid limiting our instance diversification. We are going to pay the Spot market price regardless of the Maximum price that we can specify, and setting a higher max price does not increase the chance of getting Spot capacity nor does it decrease the chance of getting your Spot Instances interrupted when EC2 needs the capacity back. You can see the current Spot price in the AWS Management Console under EC2 -> Spot Requests -> **Pricing History**. +Since Nov 2017, Amazon EC2 Spot Instances [changed the pricing model and bidding was eliminated] (https://aws.amazon.com/blogs/compute/new-amazon-ec2-spot-pricing/). We have an optional "Max-price" field for our Spot requests, which would limit how much we're willing to pay for the instance. It is recommended to leave this value at 100% of the On-Demand price, in order to avoid limiting our instance diversification. We are going to pay the Spot market price regardless of the Maximum price that we can specify, and setting a higher max price does not increase the chance of getting Spot capacity nor does it decrease the chance of getting your Spot Instances interrupted when EC2 needs the capacity back. You can see the current Spot price in the AWS Management Console under EC2 -> Spot Requests -> **Pricing History**. #### Each instance counts as X units This configuration allows us to give each instance type in our diversified fleet a weight that will count towards our Total units. By default, this weight is configured as the number of YARN VCores that the instance type has by default (this would typically equate to the number of EC2 vCPUs) - this way it's easy to set the Total units to the number of vCPUs we want our cluster to run with, and EMR will select the best instances while taking into account the required number of instances to run. For example, if r4.xlarge is the instance type that EMR found to be the least likely to be interrupted, its weight is 4 and our total units (only Spot) is 32, then 8 * r4.xlarge instances will be launched by EMR in the fleet. If my Spark application is memory driven, I can set the total units to the total amount of memory I want my cluster to run with, and change the "Each instance counts as" field to the total memory of the instance, leaving aside some memory for the operating system and other processes. For example, for the r4.xlarge I can set its weight to 25. If I then set up the Total units to 500 then EMR will bring up 20 * r4.xlarge instances in the fleet. Since our executor size is 18 GB, one executor will run on this instance type. #### Defined duration -This option will allow you run your EMR Instance Fleet on Spot Blocks, which are uninterrupted Spot Instances, available for 1-6 hours, at a lower discount compared to Spot Instances. +This option will allow you run your EMR Instance Fleet on [Spot Blocks](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-requests.html#fixed-duration-spot-instances), which are uninterrupted Spot Instances, available for 1-6 hours, at a lower discount compared to Spot Instances. #### Provisioning timeout You can determine that after a set amount of minutes, if EMR is unable to provision your selected Spot Instances due to lack of capacity, it will either start On-Demand instances instead, or terminate the cluster. This can be determined according to the business definition of the cluster or Spark application - if it is SLA bound and should complete even at On-Demand price, then the "Switch to On-Demand" option might be suitable. However, make sure you diversify the instance types in the fleet when looking to use Spot Instances, before you look into failing over to On-Demand. \ No newline at end of file diff --git a/content/running_spark_apps_with_emr_on_spot_instances/launching_emr_cluster-1.md b/content/running_spark_apps_with_emr_on_spot_instances/launching_emr_cluster-1.md index 3871810c..51c9a25d 100644 --- a/content/running_spark_apps_with_emr_on_spot_instances/launching_emr_cluster-1.md +++ b/content/running_spark_apps_with_emr_on_spot_instances/launching_emr_cluster-1.md @@ -13,7 +13,7 @@ To launch the cluster, follow these steps: 1. [Open the EMR console] (https://console.aws.amazon.com/elasticmapreduce/home) in the region where you are looking to launch your cluster. 1. Click "**Create Cluster**" 1. Click "**Go to advanced options**" -1. Select the latest EMR release, and in the list of components, only leave **Hadoop** checked and also check **Spark** and **Ganglia** (we will use it later to monitor our cluster) +1. Select the latest EMR 5.x.x release (the console will default to it), and in the list of components, only leave **Hadoop** checked and also check **Spark** and **Ganglia** (we will use it later to monitor our cluster) 1. Under "**Steps (Optional)**" -> Step type drop down menu, select "**Spark application**" and click **Add step**, then add the following details in the Add step dialog window: * **Spark-submit options**: here we will configure the memory and core count for each executor, as described in the previous section. Use these settings (make sure you have two '-' chars): diff --git a/content/running_spark_apps_with_emr_on_spot_instances/launching_emr_cluster-2.md b/content/running_spark_apps_with_emr_on_spot_instances/launching_emr_cluster-2.md index e3f8b807..96219879 100644 --- a/content/running_spark_apps_with_emr_on_spot_instances/launching_emr_cluster-2.md +++ b/content/running_spark_apps_with_emr_on_spot_instances/launching_emr_cluster-2.md @@ -3,7 +3,7 @@ title: "Launch a cluster - Step 2" weight: 70 --- -Under "**Instance group configuration**", select Instance Fleets. Under Network, select the VPC that you deployed using the CloudFormation template earlier in the workshop (or the default VPC if you're running the workshop in an AWS event), and select all subnets in the VPC. When you select multiple subnets, the EMR cluster will still be started in a single Availability Zone, but EMR Instance Fleets will make the best instance type selection based on available capacity and price across the multiple availability zones that you specified. +Under "**Instance group configuration**", select Instance Fleets. Under Network, select the VPC that you deployed using the CloudFormation template earlier in the workshop (or the default VPC if you're running the workshop in an AWS event), and select all subnets in the VPC. When you select multiple subnets, the EMR cluster will still be started in a single Availability Zone, but EMR Instance Fleets will make the best instance type selection based on available capacity and price across the multiple availability zones that you specified. Aslo, click on the checkbox "Apply allocation strategy" to leverage lowest-price allocation for On-Demand Instances and Capacity-Optimized allocation for Spot Instances; this will also allow you to configure up to 15 instance types on the Task Instance fleet. ![FleetSelection1](/images/running-emr-spark-apps-on-spot/emrinstancefleetsnetwork.png) diff --git a/content/running_spark_apps_with_emr_on_spot_instances/launching_emr_cluster-3.md b/content/running_spark_apps_with_emr_on_spot_instances/launching_emr_cluster-3.md index 32c77804..964d2a13 100644 --- a/content/running_spark_apps_with_emr_on_spot_instances/launching_emr_cluster-3.md +++ b/content/running_spark_apps_with_emr_on_spot_instances/launching_emr_cluster-3.md @@ -8,6 +8,7 @@ Key=Name Value (Optional)=EMRTransientCluster1 ![tags](/images/running-emr-spark-apps-on-spot/emrtags.png) -Click **Next** to go to **Step 4: Security** +Click **Next** to go to **Step 4: Security**. On the **EC2 key pair** drop-down, select *emr-workshop-key-pair*. +![key_pair](/images/running-emr-spark-apps-on-spot/keypair.png) -Leave all the settings as-is and click "**Create cluster**" \ No newline at end of file +Leave all the other settings as-is and click "**Create cluster**" \ No newline at end of file diff --git a/static/images/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/asg-purchase-options-and-instance-types.png b/static/images/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/asg-purchase-options-and-instance-types.png new file mode 100644 index 00000000..bbc91666 Binary files /dev/null and b/static/images/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/asg-purchase-options-and-instance-types.png differ diff --git a/static/images/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/capacity-rebalancing-diagram.png b/static/images/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/capacity-rebalancing-diagram.png new file mode 100644 index 00000000..4e903f88 Binary files /dev/null and b/static/images/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/capacity-rebalancing-diagram.png differ diff --git a/static/images/running-emr-spark-apps-on-spot/cloud9-ganglia.png b/static/images/running-emr-spark-apps-on-spot/cloud9-ganglia.png new file mode 100644 index 00000000..ce3154d5 Binary files /dev/null and b/static/images/running-emr-spark-apps-on-spot/cloud9-ganglia.png differ diff --git a/static/images/running-emr-spark-apps-on-spot/emrinstancefleetsnetwork.png b/static/images/running-emr-spark-apps-on-spot/emrinstancefleetsnetwork.png index 36df46f2..375bd312 100644 Binary files a/static/images/running-emr-spark-apps-on-spot/emrinstancefleetsnetwork.png and b/static/images/running-emr-spark-apps-on-spot/emrinstancefleetsnetwork.png differ diff --git a/static/images/running-emr-spark-apps-on-spot/keypair.png b/static/images/running-emr-spark-apps-on-spot/keypair.png new file mode 100644 index 00000000..e6f0fcd9 Binary files /dev/null and b/static/images/running-emr-spark-apps-on-spot/keypair.png differ diff --git a/workshops/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/asg.json b/workshops/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/asg.json index 43ef64e2..ccfefd44 100644 --- a/workshops/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/asg.json +++ b/workshops/ec2-auto-scaling-with-multiple-instance-types-and-purchase-options/asg.json @@ -31,16 +31,23 @@ "OnDemandAllocationStrategy": "prioritized", "OnDemandBaseCapacity": 2, "OnDemandPercentageAboveBaseCapacity": 0, - "SpotAllocationStrategy": "lowest-price", - "SpotInstancePools": 4 + "SpotAllocationStrategy": "capacity-optimized" } }, "MinSize": 2, "MaxSize": 12, "DesiredCapacity": 4, + "CapacityRebalance": true, "TargetGroupARNs": [ "%TargetGroupARN%" ], "HealthCheckType": "EC2", - "VPCZoneIdentifier": "%publicSubnet1%,%publicSubnet2%" + "VPCZoneIdentifier": "%publicSubnet1%,%publicSubnet2%", + "Tags": [ + { + "Key": "SpotInterruptionHandler/enabled", + "Value": "true", + "PropagateAtLaunch": true + } + ] }