diff --git a/content/ecs-spot-capacity-providers/WorkshopSetup/cli_setup.md b/content/ecs-spot-capacity-providers/WorkshopSetup/cli_setup.md index 1ab56055..5f69d41e 100644 --- a/content/ecs-spot-capacity-providers/WorkshopSetup/cli_setup.md +++ b/content/ecs-spot-capacity-providers/WorkshopSetup/cli_setup.md @@ -38,10 +38,6 @@ In order to execute the steps in the workshop, you'll need to clone the workshop In the Cloud9 IDE terminal, run the following command: -(Remove before prod) -``` -git clone https://github.com/jalawala/ec2-spot-workshops.git -``` ``` git clone https://github.com/awslabs/ec2-spot-workshops.git ``` diff --git a/content/ecs-spot-capacity-providers/WorkshopSetup/launch_cloudformation.md b/content/ecs-spot-capacity-providers/WorkshopSetup/launch_cloudformation.md index ba5d813b..371f92cb 100644 --- a/content/ecs-spot-capacity-providers/WorkshopSetup/launch_cloudformation.md +++ b/content/ecs-spot-capacity-providers/WorkshopSetup/launch_cloudformation.md @@ -8,7 +8,7 @@ Launch the CloudFormation stack To save time on the initial setup, a CloudFormation template will be used to create the required resources needed for the workshop. -1. You can view and download the CloudFormation template from GitHub [here, Change location before making it live] (https://github.com/ec2-spot-workshops/workshops/ecs-spot-capacity-providers/ecs-spot-workshop-cfn-for-aws-console.yaml). +1. You can view and download the CloudFormation template from GitHub [here, Change location before making it live] (https://github.com/ec2-spot-workshops/workshops/ecs-spot-capacity-providers/ecs-spot-workshop-cfn.yaml). 2. Take a moment to review the CloudFormation template so you understand the resources it will be creating. 3. Browse to the [AWS CloudFormation console] (https://console.aws.amazon.com/cloudformation). Make sure you are in AWS region designated by the facilitators of the workshop 4. Click **Create stack**. diff --git a/content/ecs-spot-capacity-providers/module-1/asg_with_od.md b/content/ecs-spot-capacity-providers/module-1/asg_with_od.md index 5417c519..44cdf416 100644 --- a/content/ecs-spot-capacity-providers/module-1/asg_with_od.md +++ b/content/ecs-spot-capacity-providers/module-1/asg_with_od.md @@ -23,7 +23,7 @@ export OD_PERCENTAGE=100 # Note that ASG will have 100% On-Demand, 0% Spot Run the following command to substitute the template with actual values from the global variables ```bash -sed -i -e "s#%ASG_NAME%#$ASG_NAME#g" -e "s#%OD_PERCENTAGE%#$OD_PERCENTAGE#g" -e "s#%PUBLIC_SUBNET_LIST%#$VPCPublicSubnets#g" -e "s#%LT_ID%#$LaunchTemplateId#g" asg.json +sed -i -e "s#%ASG_NAME%#$ASG_NAME#g" -e "s#%OD_PERCENTAGE%#$OD_PERCENTAGE#g" -e "s#%PUBLIC_SUBNET_LIST%#$VPCPublicSubnets#g" asg.json ``` Create the ASG for the On-Demand Instances diff --git a/content/ecs-spot-capacity-providers/module-1/asg_with_spot.md b/content/ecs-spot-capacity-providers/module-1/asg_with_spot.md index 00004403..2c1d8f0f 100644 --- a/content/ecs-spot-capacity-providers/module-1/asg_with_spot.md +++ b/content/ecs-spot-capacity-providers/module-1/asg_with_spot.md @@ -5,23 +5,8 @@ weight: 15 In this section, you create an Auto Scaling group for EC2 Spot Instances using the Launch Template created in previous section. This procedure is exactly same as the previous section except for a few changes specific to the configuration for Spot Instances. -One of the best practices for adoption of Spot Instances is to diversify the EC2 instances across different instance types and availability zones, in order to tap into multiple spare capacity pools. -One key criteria for choosing the instance size can be based on the ECS Task vCPU and Memory limit configuration. For example, look at the ECS task resource limits in the file **ec2-task.json**: - -```plaintext -"cpu": "256", "memory": "1024" -``` - -This means the ratio for vCPU:Memory in our ECS task that would run in the cluster is **1:4**. Ideally, we should select instance types with similar vCPU:Memory ratio, in order to have good utilization of the resources in the EC2 instances. The smallest instance type which would satisfy this critera from the latest generation of x86_64 EC2 instance types is m5.large. To learn more about EC2 instance types click [here](https://aws.amazon.com/ec2/instance-types/) - -In order to adhere to EC2 Spot best practices and diversify our use of instance types (in order to tap into multiple spare capacity pools), we can use the EC2 Instance Types console to find instance types which have similar hardware characteristics to the m5.large. The t2 & t3 instance types are burstable instance types, which also fit our objective in this workshop. To learn more about EC2 burstable instance types, click [here](https://aws.amazon.com/ec2/instance-types/t3/) - -![OD ASG](/images/ecs-spot-capacity-providers/ec1.png) -![OD ASG](/images/ecs-spot-capacity-providers/ec2.png) -![OD ASG](/images/ecs-spot-capacity-providers/ec3.png) - -We selected 10 different instance types as can seen asg.json, but you can configure up to 20 different instance types in an ASG. We chose instance types with similar hardware characteristics in order to have a consistent auto scaling experience. +We have configured the instance diversification in asg.json. We chose instance types with similar hardware characteristics in order to have a consistent auto scaling experience. Copy the file **templates/asg.json** for the Auto Scaling group configuration. @@ -41,7 +26,7 @@ export OD_PERCENTAGE=0 # Note that ASG will have 0% On-Demand, 100% Spot Run the following command to substitute the template with actual values from the global variables ```bash -sed -i -e "s#%ASG_NAME%#$ASG_NAME#g" -e "s#%OD_PERCENTAGE%#$OD_PERCENTAGE#g" -e "s#%PUBLIC_SUBNET_LIST%#$VPCPublicSubnets#g" -e "s#%LT_ID%#$LaunchTemplateId#g" asg.json +sed -i -e "s#%ASG_NAME%#$ASG_NAME#g" -e "s#%OD_PERCENTAGE%#$OD_PERCENTAGE#g" -e "s#%PUBLIC_SUBNET_LIST%#$VPCPublicSubnets#g" asg.json ``` Create the Auto scaling group that will run Spot Instances in our cluster diff --git a/content/ecs-spot-capacity-providers/module-1/selecting_spot_instance_types.md b/content/ecs-spot-capacity-providers/module-1/selecting_spot_instance_types.md new file mode 100644 index 00000000..f0c74d4b --- /dev/null +++ b/content/ecs-spot-capacity-providers/module-1/selecting_spot_instance_types.md @@ -0,0 +1,108 @@ +--- +title: "Selecting Instance Types" +date: 2018-08-07T11:05:19-07:00 +weight: 13 +draft: false +--- + +[Amazon EC2 Spot Instances](https://aws.amazon.com/ec2/spot/) offer spare compute capacity available in the AWS Cloud at steep discounts compared to On-Demand prices. EC2 can interrupt Spot Instances with two minutes of notification when EC2 needs the capacity back. You can use Spot Instances for various fault-tolerant and flexible applications. Some examples are analytics, containerized workloads, high-performance computing (HPC), stateless web servers, rendering, CI/CD, and other test and development workloads. + +One of the best practices to successfully adopt Spot instances is to implement **Spot instance diversification** as part of your configuration. Spot instance diversification helps to procure +capacity from multiple Spot Instance pools, both for scaling up and for replacing spot instances that may receive a spot instance termination notification. A Spot instance pool is a set of unused EC2 instances with the same instance type (for example, m5.large), operating system, Availability Zone. + +We can diversify Spot instance pools using below strategy: + + - By Implementing instance diversification within the nodegroups, by selecting a mix of instances types and families from different Spot instance pools that meet the same vCPU's and memory criteria. + +One key criteria for choosing the instance size can be based on the ECS Task vCPU and Memory limit configuration. For example, look at the ECS task resource limits in the file **ec2-task.json**: + +```plaintext +"cpu": "256", "memory": "1024" +``` + +This means the ratio for vCPU:Memory in our ECS task that would run in the cluster is **1:4**. Ideally, we should select instance types with similar vCPU:Memory ratio, in order to have good utilization of the resources in the EC2 instances. The smallest instance type which would satisfy this critera from the latest generation of x86_64 EC2 instance types is m5.large. To learn more about EC2 instance types click [here](https://aws.amazon.com/ec2/instance-types/) + +We will use **[amazon-ec2-instance-selector](https://github.com/aws/amazon-ec2-instance-selector)** to help us select the relevant instance +types and familes with sufficient number of vCPUs and RAM. + +There are over 270 different instance types available on EC2 which can make the process of selecting appropriate instance types difficult. **[amazon-ec2-instance-selector](https://github.com/aws/amazon-ec2-instance-selector)** helps you select compatible instance types for your application to run on. The command line interface can be passed resource criteria like vcpus, memory, network performance, and much more and then return the available, matching instance types. + +Let's first install **amazon-ec2-instance-selector** : + +``` +curl -Lo ec2-instance-selector https://github.com/aws/amazon-ec2-instance-selector/releases/download/v1.3.0/ec2-instance-selector-`uname | tr '[:upper:]' '[:lower:]'`-amd64 && chmod +x ec2-instance-selector +sudo mv ec2-instance-selector /usr/local/bin/ +ec2-instance-selector --version +``` + +Now that you have ec2-instance-selector installed, you can run +`ec2-instance-selector --help` to understand how you could use it for selecting +instances that match your workload requirements. For the purpose of this workshop +we need to first get a group of instances that meet the 4vCPUs and 16GB of RAM. +Run the following command to get the list of instances. + +```bash +ec2-instance-selector --vcpus 2 --memory 8192 --gpus 0 --current-generation -a x86_64 --deny-list '.*n.*' +``` + +This should display a list like the one that follows (note results might differ depending on the region). We will use this instances as part of one of our node groups. + +``` +m4.large +m5.large +m5a.large +m5ad.large +m5d.large +t2.large +t3.large +t3a.large +``` + +Internally ec2-instance-selector is making calls to the [DescribeInstanceTypes](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstanceTypes.html) for the specific region and filtering +the intstances based on the criteria selected in the command line, in our case +we did filter for instances that meet the following criteria: + + * Instances with no GPUs + * of x86_64 Architecture (no ARM instances like A1 or m6g instances for example) + * Instances that have 4 vCPUs and 16GB of Ram + * Instances of current generation (4th gen onwards) + * Instances that don't meet the regular expresion `.*n.*`, so effectively m5n, m5dn. + +{{% notice warning %}} +Your workload may have other constraints that you should consider when selecting instances types. For example. **t2** and **t3** instance types are [burstable instances](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/burstable-performance-instances.html) and might not be appropriate for CPU bound workloads that require CPU execution determinism. Instances such as m5**a** are [AMD Instances](https://aws.amazon.com/ec2/amd/), if your workload is sensitive to numerical differences (i.e: financial risk calculations, industrial simulations) mixing these instance types might not be appropriate. +{{% /notice %}} + +{{% notice note %}} +You are encouraged to test what are the options that `ec2-instance-selector` provides and run a few commands with it to familiarize yourself with the tool. +For example, try running the same commands as you did before with the extra parameter **`--output table-wide`**. +{{% /notice %}} + +### Challenge + +Find out another group that adheres to a 1vCPU:4GB ratio, this time using instances with 8vCPU's and 32GB of RAM. + +{{%expand "Expand this for an example on the list of instances" %}} + +That should be easy. You can run the command: + +```bash +ec2-instance-selector --vcpus 8 --memory 32768 --gpus 0 --current-generation -a x86_64 --deny-list '.*n.*|.*h.*' +``` + +which should yield a list as follows + +``` +m4.2xlarge +m5.2xlarge +m5a.2xlarge +m5d.2xlarge +t2.2xlarge +t3.2xlarge +t3a.2xlarge +``` + +{{% /expand %}} + + + + diff --git a/content/ecs-spot-capacity-providers/module-2/_index.md b/content/ecs-spot-capacity-providers/module-2/_index.md index 0bca5176..57c21272 100644 --- a/content/ecs-spot-capacity-providers/module-2/_index.md +++ b/content/ecs-spot-capacity-providers/module-2/_index.md @@ -1,5 +1,5 @@ --- -title: "Module-2 (Optional): Optimizing costs using ECS Fargate Spot Capacity Providers" +title: "Module-2 : Optimizing costs using ECS Fargate Spot Capacity Providers" weight: 40 --- diff --git a/workshops/ecs-spot-capacity-providers/ecs-spot-workshop-cfn-for-eventengine.yaml b/workshops/ecs-spot-capacity-providers/ecs-spot-workshop-cfn-for-eventengine.yaml deleted file mode 100644 index 857ff7d5..00000000 --- a/workshops/ecs-spot-capacity-providers/ecs-spot-workshop-cfn-for-eventengine.yaml +++ /dev/null @@ -1,571 +0,0 @@ ---- -AWSTemplateFormatVersion: 2010-09-09 -Description: ECS Spot Workshop CFN Template - -Metadata: - Author: - Description: Jayaprakash Alawala - License: - Description: 'Copyright 2020 Amazon.com, Inc. and its affiliates. All Rights Reserved. - - Licensed under the Amazon Software License (the "License"). You may not use this file - except in compliance with the License. A copy of the License is located at - - http://aws.amazon.com/asl/ - - or in the "license" file accompanying this file. This file is distributed on an "AS IS" - BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations under the License.' - -Parameters: - sourceCidr: - Default: 0.0.0.0/0 - Description: Optional - CIDR/IP range for instance ssh/http access and load balancer http - access - Type: String - - ECSAMI: - Description: AMI ID - Type: AWS::SSM::Parameter::Value - Default: /aws/service/ecs/optimized-ami/amazon-linux-2/recommended/image_id - -Resources: - - vpc: - Type: AWS::EC2::VPC - Properties: - CidrBlock: 10.0.0.0/16 - EnableDnsHostnames: true - EnableDnsSupport: true - Tags: - - Key: Name - Value: !Sub "${AWS::StackName}" - - internetGateway: - Type: AWS::EC2::InternetGateway - DependsOn: - - vpc - - attachGateway: - Type: AWS::EC2::VPCGatewayAttachment - DependsOn: - - vpc - - internetGateway - Properties: - InternetGatewayId: !Ref internetGateway - VpcId: !Ref vpc - - - publicSubnet1: - Type: AWS::EC2::Subnet - DependsOn: attachGateway - Properties: - AvailabilityZone: - Fn::Select: - - 0 - - Fn::GetAZs: - Ref: AWS::Region - CidrBlock: 10.0.0.0/24 - MapPublicIpOnLaunch: true - Tags: - - Key: Name - Value: !Sub "${AWS::StackName}-publicSubnet1" - VpcId: !Ref vpc - - publicSubnet2: - Type: AWS::EC2::Subnet - DependsOn: attachGateway - Properties: - AvailabilityZone: - Fn::Select: - - 1 - - Fn::GetAZs: - Ref: AWS::Region - CidrBlock: 10.0.1.0/24 - MapPublicIpOnLaunch: true - Tags: - - Key: Name - Value: !Sub "${AWS::StackName}-publicSubnet2" - VpcId: !Ref vpc - - publicSubnet3: - Type: AWS::EC2::Subnet - DependsOn: attachGateway - Properties: - AvailabilityZone: - Fn::Select: - - 2 - - Fn::GetAZs: - Ref: AWS::Region - CidrBlock: 10.0.2.0/24 - MapPublicIpOnLaunch: true - Tags: - - Key: Name - Value: !Sub "${AWS::StackName}-publicSubnet3" - VpcId: !Ref vpc - - - privateSubnet1: - Type: AWS::EC2::Subnet - Properties: - AvailabilityZone: - Fn::Select: - - 0 - - Fn::GetAZs: - Ref: AWS::Region - CidrBlock: 10.0.3.0/24 - Tags: - - Key: Name - Value: !Sub "${AWS::StackName}-privateSubnet1" - VpcId: !Ref vpc - - - privateSubnet2: - Type: AWS::EC2::Subnet - Properties: - AvailabilityZone: - Fn::Select: - - 1 - - Fn::GetAZs: - Ref: AWS::Region - CidrBlock: 10.0.4.0/24 - Tags: - - Key: Name - Value: !Sub "${AWS::StackName}-privateSubnet2" - VpcId: !Ref vpc - - privateSubnet3: - Type: AWS::EC2::Subnet - Properties: - AvailabilityZone: - Fn::Select: - - 2 - - Fn::GetAZs: - Ref: AWS::Region - CidrBlock: 10.0.5.0/24 - Tags: - - Key: Name - Value: !Sub "${AWS::StackName}-privateSubnet3" - VpcId: !Ref vpc - - - publicRouteTable: - Type: AWS::EC2::RouteTable - DependsOn: - - vpc - - attachGateway - Properties: - Tags: - - Key: Name - Value: !Sub "${AWS::StackName} Public Route Table" - VpcId: !Ref vpc - - RouteTablePrivate: - Type: AWS::EC2::RouteTable - DependsOn: vpc - Properties: - VpcId: !Ref vpc - Tags: - - Key: Name - Value: !Sub "${AWS::StackName} Private Route Table" - - publicSubnet1RouteTableAssociation: - Type: AWS::EC2::SubnetRouteTableAssociation - DependsOn: - - publicRouteTable - - publicSubnet1 - - attachGateway - Properties: - RouteTableId: !Ref publicRouteTable - SubnetId: !Ref publicSubnet1 - - publicSubnet2RouteTableAssociation: - Type: AWS::EC2::SubnetRouteTableAssociation - DependsOn: - - publicRouteTable - - publicSubnet2 - - attachGateway - Properties: - RouteTableId: !Ref publicRouteTable - SubnetId: !Ref publicSubnet2 - - publicSubnet3RouteTableAssociation: - Type: AWS::EC2::SubnetRouteTableAssociation - DependsOn: - - publicRouteTable - - publicSubnet3 - - attachGateway - Properties: - RouteTableId: !Ref publicRouteTable - SubnetId: !Ref publicSubnet3 - - - privateSubnet1RouteTableAssociation: - Type: AWS::EC2::SubnetRouteTableAssociation - DependsOn: - - RouteTablePrivate - - privateSubnet1 - Properties: - RouteTableId: !Ref RouteTablePrivate - SubnetId: !Ref privateSubnet1 - - privateSubnet2RouteTableAssociation: - Type: AWS::EC2::SubnetRouteTableAssociation - DependsOn: - - RouteTablePrivate - - privateSubnet2 - Properties: - RouteTableId: !Ref RouteTablePrivate - SubnetId: !Ref privateSubnet2 - - privateSubnet3RouteTableAssociation: - Type: AWS::EC2::SubnetRouteTableAssociation - DependsOn: - - RouteTablePrivate - - privateSubnet3 - Properties: - RouteTableId: !Ref RouteTablePrivate - SubnetId: !Ref privateSubnet3 - - - EIPNATGateway: - Type: AWS::EC2::EIP - DependsOn: - - attachGateway - - vpc - Properties: - Domain: vpc - - NATGateway: - Type: AWS::EC2::NatGateway - DependsOn: - - publicSubnet1 - Properties: - AllocationId: !GetAtt EIPNATGateway.AllocationId - SubnetId: !Ref publicSubnet1 - - publicRoute: - Type: AWS::EC2::Route - DependsOn: - - publicRouteTable - - internetGateway - - attachGateway - Properties: - DestinationCidrBlock: 0.0.0.0/0 - GatewayId: !Ref internetGateway - RouteTableId: !Ref publicRouteTable - - privateRoute: - Type: AWS::EC2::Route - DependsOn: - - RouteTablePrivate - - NATGateway - Properties: - DestinationCidrBlock: 0.0.0.0/0 - NatGatewayId: !Ref NATGateway - RouteTableId: !Ref RouteTablePrivate - - autoScalingServiceLinkedRole: - Type: AWS::IAM::ServiceLinkedRole - Properties: - AWSServiceName: autoscaling.amazonaws.com - Description: Default Service-Linked Role enables access to AWS Services and Resources - used or managed by Auto Scaling - - loadBalancerSecurityGroup: - Type: AWS::EC2::SecurityGroup - DependsOn: - - vpc - Properties: - GroupDescription: Allow all traffic from internet - SecurityGroupIngress: - - CidrIp: !Ref sourceCidr - IpProtocol: -1 - VpcId: !Ref vpc - - instanceSecurityGroup: - Type: AWS::EC2::SecurityGroup - DependsOn: - - vpc - Properties: - GroupDescription: Allow traffic from ALB - VpcId: !Ref vpc - - instanceSecurityGroupIngress: - Type: AWS::EC2::SecurityGroupIngress - DependsOn: - - instanceSecurityGroup - - loadBalancerSecurityGroup - Properties: - GroupId: !Ref instanceSecurityGroup - IpProtocol: -1 - SourceSecurityGroupId: !Ref loadBalancerSecurityGroup - - ECSServiceALB: - Type: AWS::ElasticLoadBalancingV2::LoadBalancer - DependsOn: - - loadBalancerSecurityGroup - - publicSubnet1 - - publicSubnet2 - - publicSubnet3 - Properties: - Name: !Sub "${AWS::StackName}" - Scheme: internet-facing - SecurityGroups: - - !Ref loadBalancerSecurityGroup - Subnets: - - !Ref publicSubnet1 - - !Ref publicSubnet2 - - !Ref publicSubnet3 - - ECSServiceTG: - Type: AWS::ElasticLoadBalancingV2::TargetGroup - DependsOn: - - vpc - Properties: - HealthCheckIntervalSeconds: 50 - HealthCheckPath: / - HealthCheckPort: traffic-port - HealthCheckProtocol: HTTP - HealthCheckTimeoutSeconds: 45 - HealthyThresholdCount: 2 - Matcher: - HttpCode: 200 - Name: !Sub "${AWS::StackName}" - Port: 80 - Protocol: HTTP - UnhealthyThresholdCount: 4 - VpcId: !Ref vpc - - ECSServiceALBListener: - Type: AWS::ElasticLoadBalancingV2::Listener - DependsOn: - - ECSServiceALB - - ECSServiceTG - Properties: - DefaultActions: - - Type: forward - TargetGroupArn: !Ref ECSServiceTG - LoadBalancerArn: !Ref ECSServiceALB - Port: 80 - Protocol: HTTP - - instanceRole: - Type: AWS::IAM::Role - Properties: - RoleName: !Sub "${AWS::StackName}-EcsInstanceRole" - AssumeRolePolicyDocument: - Statement: - - Action: - - sts:AssumeRole - Effect: Allow - Principal: - Service: - - ec2.amazonaws.com - Version: 2012-10-17 - ManagedPolicyArns: - - arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role - - arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceAutoscaleRole - - arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess - - Cloud9instanceRole: - Type: AWS::IAM::Role - Properties: - RoleName: !Sub "${AWS::StackName}-Cloud9InstanceRole" - AssumeRolePolicyDocument: - Statement: - - Action: - - sts:AssumeRole - Effect: Allow - Principal: - Service: - - ec2.amazonaws.com - Version: 2012-10-17 - ManagedPolicyArns: - - arn:aws:iam::aws:policy/AdministratorAccess - - cloud9Environment: - DependsOn: - - publicSubnet1 - Properties: - Name : EcsSpotWorkshop - OwnerArn : !Sub arn:aws:sts::${AWS::AccountId}:assumed-role/TeamRole/MasterKey - Description: ECS Spot Workshop - Cloud9 environment - InstanceType: t3.large - SubnetId: - Ref: publicSubnet1 - Type: AWS::Cloud9::EnvironmentEC2 - - Cloud9instanceProfile: - Type: AWS::IAM::InstanceProfile - DependsOn: - - Cloud9instanceRole - Properties: - InstanceProfileName: !Sub "${AWS::StackName}-Cloud9InstanceProfile" - Path: / - Roles: - - Ref: Cloud9instanceRole - - instanceProfile: - Type: AWS::IAM::InstanceProfile - DependsOn: - - instanceRole - Properties: - InstanceProfileName: !Sub "${AWS::StackName}-InstanceProfile" - Path: / - Roles: - - Ref: instanceRole - - - ECSInstanceLaunchTemplate: - DependsOn: - - instanceProfile - Type: AWS::EC2::LaunchTemplate - Properties: - LaunchTemplateData: - InstanceType: t3.large - ImageId: !Ref ECSAMI - IamInstanceProfile: - Arn: - Fn::GetAtt: - - instanceProfile - - Arn - - SecurityGroupIds: - - !Ref instanceSecurityGroup - - TagSpecifications: - - ResourceType: instance - Tags: - - Key: Name - Value: !Sub "${AWS::StackName}" - UserData: - Fn::Base64: !Sub | - #!/bin/bash - echo "ECS_CLUSTER=EcsSpotWorkshop" >> /etc/ecs/ecs.config - echo "ECS_ENABLE_SPOT_INSTANCE_DRAINING=true" >> /etc/ecs/ecs.config - echo "ECS_CONTAINER_STOP_TIMEOUT=120s" >> /etc/ecs/ecs.config - echo "ECS_ENABLE_CONTAINER_METADATA=true" >> /etc/ecs/ecs.config - LaunchTemplateName: !Sub "${AWS::StackName}" - - ecrRepository: - Type: AWS::ECR::Repository - Properties: - RepositoryName: ecs-spot-workshop/webapp - -Outputs: - awsRegionId: - Description: The AWS Region ID your template was launched in - Value: !Ref AWS::Region - Export: - Name: !Sub "${AWS::StackName}-awsRegionId" - - instanceRole: - Description: Instance Role - Value: - Fn::GetAtt: - - instanceRole - - Arn - Export: - Name: !Sub "${AWS::StackName}-instanceRole" - - instanceProfile: - Description: Instance profile ARN - Value: - Fn::GetAtt: - - instanceProfile - - Arn - Export: - Name: !Sub "${AWS::StackName}-instanceProfile" - - Cloud9instanceProfile: - Description: Cloud9 Instance profile ARN - Value: - Fn::GetAtt: - - Cloud9instanceProfile - - Arn - Export: - Name: !Sub "${AWS::StackName}-Cloud9instanceProfile" - - - instanceSecurityGroup: - Description: Instance security group - Value: !Ref instanceSecurityGroup - Export: - Name: !Sub "${AWS::StackName}-instanceSecurityGroup" - - loadBalancerSecurityGroup: - Description: Load Balancer security group - Value: !Ref loadBalancerSecurityGroup - - - publicSubnet1: - Description: Public subnet 1 - Value: !Ref publicSubnet1 - - publicSubnet2: - Description: Public subnet 2 - Value: !Ref publicSubnet2 - - publicSubnet3: - Description: Public subnet 3 - Value: !Ref publicSubnet3 - - privateSubnet1: - Description: Private subnet 1 - Value: !Ref privateSubnet1 - - privateSubnet2: - Description: Private subnet 2 - Value: !Ref privateSubnet2 - - privateSubnet3: - Description: Private subnet 3 - Value: !Ref privateSubnet3 - - vpc: - Description: The VPC - Value: !Ref vpc - - LaunchTemplateId: - Description: The Launch Template - Value: !Ref ECSInstanceLaunchTemplate - - ALBDNSName: - Description: The Application Load Balancer - Value: !GetAtt ECSServiceALB.DNSName - - TargetGroup: - Description: The Target Group - Value: !Ref ECSServiceTG - - VPCPublicSubnets: - Description: The list of public subnets in the VPC - Value: !Join [",", [!Ref publicSubnet1, !Ref publicSubnet2]] - - VPCPrivateSubnets: - Description: The list of private subnets in the VPC - Value: !Join [",", [!Ref privateSubnet1, !Ref privateSubnet2]] - - ecrRepository: - Description: The ecrRepository - Value: !Ref ecrRepository - - Cloud9instanceRole: - Description: The Cloud9instanceRole - Value: - Fn::GetAtt: - - Cloud9instanceRole - - Arn - - autoScalingServiceLinkedRole: - Description: The AWS Region ID your template was launched in - Value: !Ref autoScalingServiceLinkedRole - - cloud9Environment: - Description: Cloud9 environment - Value: - Fn::GetAtt: - - cloud9Environment - - Name -... diff --git a/workshops/ecs-spot-capacity-providers/ecs-spot-workshop-cfn-for-aws-console.yaml b/workshops/ecs-spot-capacity-providers/ecs-spot-workshop-cfn.yaml similarity index 91% rename from workshops/ecs-spot-capacity-providers/ecs-spot-workshop-cfn-for-aws-console.yaml rename to workshops/ecs-spot-capacity-providers/ecs-spot-workshop-cfn.yaml index 743247f7..a0b72116 100644 --- a/workshops/ecs-spot-capacity-providers/ecs-spot-workshop-cfn-for-aws-console.yaml +++ b/workshops/ecs-spot-capacity-providers/ecs-spot-workshop-cfn.yaml @@ -29,6 +29,14 @@ Parameters: Type: AWS::SSM::Parameter::Value Default: /aws/service/ecs/optimized-ami/amazon-linux-2/recommended/image_id + EETeamRoleArn: + Type: String + Default: '' + +Conditions: + EventEngine: + !Not [!Equals [!Ref EETeamRoleArn, '']] + Resources: vpc: @@ -39,7 +47,7 @@ Resources: EnableDnsSupport: true Tags: - Key: Name - Value: !Sub "${AWS::StackName}" + Value: "EcsSpotWorkshop" internetGateway: Type: AWS::EC2::InternetGateway @@ -69,7 +77,7 @@ Resources: MapPublicIpOnLaunch: true Tags: - Key: Name - Value: !Sub "${AWS::StackName}-publicSubnet1" + Value: "EcsSpotWorkshop publicSubnet1" VpcId: !Ref vpc publicSubnet2: @@ -85,7 +93,7 @@ Resources: MapPublicIpOnLaunch: true Tags: - Key: Name - Value: !Sub "${AWS::StackName}-publicSubnet2" + Value: "EcsSpotWorkshop publicSubnet2" VpcId: !Ref vpc publicSubnet3: @@ -101,7 +109,7 @@ Resources: MapPublicIpOnLaunch: true Tags: - Key: Name - Value: !Sub "${AWS::StackName}-publicSubnet3" + Value: "EcsSpotWorkshop publicSubnet3" VpcId: !Ref vpc @@ -116,7 +124,7 @@ Resources: CidrBlock: 10.0.3.0/24 Tags: - Key: Name - Value: !Sub "${AWS::StackName}-privateSubnet1" + Value: "EcsSpotWorkshop privateSubnet1" VpcId: !Ref vpc @@ -131,7 +139,7 @@ Resources: CidrBlock: 10.0.4.0/24 Tags: - Key: Name - Value: !Sub "${AWS::StackName}-privateSubnet2" + Value: "EcsSpotWorkshop privateSubnet2" VpcId: !Ref vpc privateSubnet3: @@ -145,7 +153,7 @@ Resources: CidrBlock: 10.0.5.0/24 Tags: - Key: Name - Value: !Sub "${AWS::StackName}-privateSubnet3" + Value: "EcsSpotWorkshop privateSubnet3" VpcId: !Ref vpc @@ -157,7 +165,7 @@ Resources: Properties: Tags: - Key: Name - Value: !Sub "${AWS::StackName} Public Route Table" + Value: "EcsSpotWorkshop Public Route Table" VpcId: !Ref vpc RouteTablePrivate: @@ -167,7 +175,7 @@ Resources: VpcId: !Ref vpc Tags: - Key: Name - Value: !Sub "${AWS::StackName} Private Route Table" + Value: "EcsSpotWorkshop Private Route Table" publicSubnet1RouteTableAssociation: Type: AWS::EC2::SubnetRouteTableAssociation @@ -309,7 +317,7 @@ Resources: - publicSubnet2 - publicSubnet3 Properties: - Name: !Sub "${AWS::StackName}" + Name: "EcsSpotWorkshop" Scheme: internet-facing SecurityGroups: - !Ref loadBalancerSecurityGroup @@ -331,7 +339,7 @@ Resources: HealthyThresholdCount: 2 Matcher: HttpCode: 200 - Name: !Sub "${AWS::StackName}" + Name: "EcsSpotWorkshop" Port: 80 Protocol: HTTP UnhealthyThresholdCount: 4 @@ -353,7 +361,7 @@ Resources: instanceRole: Type: AWS::IAM::Role Properties: - RoleName: !Sub "${AWS::StackName}-EcsInstanceRole" + RoleName: "EcsSpotWorkshop-EcsInstanceRole" AssumeRolePolicyDocument: Statement: - Action: @@ -371,7 +379,7 @@ Resources: Cloud9instanceRole: Type: AWS::IAM::Role Properties: - RoleName: !Sub "${AWS::StackName}-Cloud9InstanceRole" + RoleName: "EcsSpotWorkshop-Cloud9InstanceRole" AssumeRolePolicyDocument: Statement: - Action: @@ -390,6 +398,8 @@ Resources: Properties: Name : EcsSpotWorkshop Description: ECS Spot Workshop - Cloud9 environment + OwnerArn: + !If [EventEngine, !Sub "arn:aws:sts::${AWS::AccountId}:assumed-role/TeamRole/MasterKey", !Ref "AWS::NoValue"] InstanceType: t3.large SubnetId: Ref: publicSubnet1 @@ -400,7 +410,7 @@ Resources: DependsOn: - Cloud9instanceRole Properties: - InstanceProfileName: !Sub "${AWS::StackName}-Cloud9InstanceProfile" + InstanceProfileName: "EcsSpotWorkshop-Cloud9InstanceProfile" Path: / Roles: - Ref: Cloud9instanceRole @@ -410,7 +420,7 @@ Resources: DependsOn: - instanceRole Properties: - InstanceProfileName: !Sub "${AWS::StackName}-InstanceProfile" + InstanceProfileName: "EcsSpotWorkshop-InstanceProfile" Path: / Roles: - Ref: instanceRole @@ -437,7 +447,7 @@ Resources: - ResourceType: instance Tags: - Key: Name - Value: !Sub "${AWS::StackName}" + Value: "EcsSpotWorkshop" UserData: Fn::Base64: !Sub | #!/bin/bash @@ -445,7 +455,7 @@ Resources: echo "ECS_ENABLE_SPOT_INSTANCE_DRAINING=true" >> /etc/ecs/ecs.config echo "ECS_CONTAINER_STOP_TIMEOUT=120s" >> /etc/ecs/ecs.config echo "ECS_ENABLE_CONTAINER_METADATA=true" >> /etc/ecs/ecs.config - LaunchTemplateName: !Sub "${AWS::StackName}" + LaunchTemplateName: "EcsSpotWorkshop" ecrRepository: Type: AWS::ECR::Repository @@ -456,8 +466,6 @@ Outputs: awsRegionId: Description: The AWS Region ID your template was launched in Value: !Ref AWS::Region - Export: - Name: !Sub "${AWS::StackName}-awsRegionId" instanceRole: Description: Instance Role @@ -465,8 +473,6 @@ Outputs: Fn::GetAtt: - instanceRole - Arn - Export: - Name: !Sub "${AWS::StackName}-instanceRole" instanceProfile: Description: Instance profile ARN @@ -474,8 +480,6 @@ Outputs: Fn::GetAtt: - instanceProfile - Arn - Export: - Name: !Sub "${AWS::StackName}-instanceProfile" Cloud9instanceProfile: Description: Cloud9 Instance profile ARN @@ -483,15 +487,10 @@ Outputs: Fn::GetAtt: - Cloud9instanceProfile - Arn - Export: - Name: !Sub "${AWS::StackName}-Cloud9instanceProfile" - instanceSecurityGroup: Description: Instance security group Value: !Ref instanceSecurityGroup - Export: - Name: !Sub "${AWS::StackName}-instanceSecurityGroup" loadBalancerSecurityGroup: Description: Load Balancer security group @@ -561,6 +560,10 @@ Outputs: Description: The AWS Region ID your template was launched in Value: !Ref autoScalingServiceLinkedRole + EETeamRoleArn: + Description: EETeamRoleArn + Value: !Ref EETeamRoleArn + cloud9Environment: Description: Cloud9 environment Value: diff --git a/workshops/ecs-spot-capacity-providers/templates/asg.json b/workshops/ecs-spot-capacity-providers/templates/asg.json index 1dad70cb..275a2c98 100755 --- a/workshops/ecs-spot-capacity-providers/templates/asg.json +++ b/workshops/ecs-spot-capacity-providers/templates/asg.json @@ -3,10 +3,13 @@ "MixedInstancesPolicy": { "LaunchTemplate": { "LaunchTemplateSpecification": { - "LaunchTemplateId": "%LT_ID%", + "LaunchTemplateName": "EcsSpotWorkshop", "Version": "1" }, "Overrides": [ + { + "InstanceType": "m4.large" + }, { "InstanceType": "m5.large" }, @@ -20,22 +23,13 @@ "InstanceType": "m5d.large" }, { - "InstanceType": "m5dn.large" - }, - { - "InstanceType": "m5n.large" + "InstanceType": "t2.large" }, { "InstanceType": "t3.large" }, { "InstanceType": "t3a.large" - }, - { - "InstanceType": "m4.large" - }, - { - "InstanceType": "t2.large" } ] },