Merge pull request #2 from ruecarlo/kvrajesh_ecs

Conflicts resolved

content/ecs-spot-capacity-providers/architecture.md

@@ -24,4 +24,4 @@ Here is the overall architecture of what you will be building throughout this wo
 
 
 #### Here is a diagram of the resulting architecture:
-![Overall Architecture](/images/ecs-spot-capacity-providers/architecture1.png)
+![Overall Architecture](/images/ecs-spot-capacity-providers/architecture1.png)

content/ecs-spot-capacity-providers/module-1/_index.md

@@ -41,4 +41,5 @@ The strategy sets FARGATE as the default capacity provider. That means if there
 
 Click  _***Update Cluster***_ on the top right corner to see default Capacity Provider Strategy. As shown base=1 is set for FARGATE Capacity Provider.
 
-![ECS Cluster](/images/ecs-spot-capacity-providers/c2.png)
+![ECS Cluster](/images/ecs-spot-capacity-providers/c2.png)
+

content/ecs-spot-capacity-providers/module-1/attach_iam_role.md

@@ -0,0 +1,54 @@
+---
+title: "Attach the IAM role to your Workspace"
+chapter: true
+weight: 20
+---
+
+### Attach the IAM role to your Workspace
+
+- Follow [this deep link to find your Cloud9 EC2 instance](https://console.aws.amazon.com/ec2/v2/home?#Instances:tag:Name=aws-cloud9-.*workshop.*;sort=desc:launchTime)
+- Select the instance, then choose **Actions / Instance Settings / Attach/Replace IAM Role**
+- Choose **ecsspotworkshop-admin** from the **IAM Role** drop down, and select **Apply**
+- Return to your workspace and click the sprocket, or launch a new tab to open the Preferences tab
+- Select **AWS SETTINGS**
+- Turn off **AWS managed temporary credentials**
+- Close the Preferences tab
+- To ensure temporary credentials aren't already in place we will also remove any existing credentials file:
+```
+rm -vf ${HOME}/.aws/credentials
+```
+
+- We should configure our aws cli with our current region as default:
+```
+export ACCOUNT\_ID=$(aws sts get-caller-identity  --output text --query Account)
+ export AWS\_REGION=$(curl -s  169.254.169.254/latest/dynamic/instance-identity/document | jq -r '.region')
+ echo "export  ACCOUNT\_ID=${ACCOUNT\_ID}" \>\> ~/.bash\_profile
+ echo "export  AWS\_REGION=${AWS\_REGION}" \>\> ~/.bash\_profile
+ aws configure set default.region ${AWS\_REGION}
+ aws configure get default.region
+```
+
+- Use the [GetCallerIdentity](https://docs.aws.amazon.com/cli/latest/reference/sts/get-caller-identity.html) CLI command to validate that the Cloud9 IDE is using the correct IAM role.
+
+```
+aws sts get-caller-identity
+```
+- The output assumed-role name should contain:
+
+```
+{
+      "Account": "000474600478",
+      "UserId": "AROAQAHCJ2QPAONSHPAXY:i-01ad7d6cd53ba8945",
+      "Arn": "arn:aws:sts::000474600478:assumed-role/ecsspotworkshop-admin/i-01ad7d6cd53ba8945"
+ }
+```
+
+
+
+#### Attach IAM role to your Cloud 9 Environment:
+![Cloud 9 Environment](/images/ecs-spot-capacity-providers/iam_attach_role.png)
+
+
+
+
+Now you are done with Module-1, Proceed to Module-2 of this workshop.

content/ecs-spot-capacity-providers/module-1/cli_setup.md

@@ -0,0 +1,40 @@
+---
+title: "CLI Setup"
+chapter: true
+weight: 10
+---
+
+### Setup AWS CLI and other tools
+
+Make sure the latest version of the AWS CLI is installed by running:
+
+```
+sudo pip install -U awscli  
+```
+Install dependencies for use in the workshop by running:
+
+```
+sudo yum -y install jq gettext
+```
+
+### Clone the GitHub repo
+
+In order to execute the steps in the workshop, you'll need to clone the workshop GitHub repo.
+
+In the Cloud9 IDE terminal, run the following command:
+
+(remove before prod0
+```
+git clone https://github.com/jalawala/ec2-spot-workshops.git 
+```
+```
+git clone https://github.com/awslabs/ec2-spot-workshops.git
+```
+Change into the workshop directory:
+
+```
+cd ec2-spot-workshops/workshops/ecs-spot-capacity-providers
+```
+
+Feel free to browse around. You can also browse the directory structure in the **Environment** tab on the left, and even edit files directly there by double clicking on them.
+

content/ecs-spot-capacity-providers/module-1/create_iam_role.md

@@ -0,0 +1,43 @@
+---
+title: "Create IAM roles for your Workspace"
+chapter: true
+weight: 15
+---
+
+### Create IAM roles for your Workspace
+
+
+In order to work with ECS from our workstation, we will need the appropriate permissions for our developer workstation instance.
+
+1. Go to the [IAM Console](https://console.aws.amazon.com/iam/home), **Roles** > **Create New Role > AWS Service > EC2.** We will later assign this role to our workstation instance.
+1. Click **Next: Permissions.** Confirm that **AdministratorAccess** is checked (TBD: to restrict needed permissions only)
+1. Click **Next:Tags** Take the defaults, and click **Next: Review** to review.
+1. Enter **ecsspotworkshop-admin** for the Name, and click **Create role**.
+
+<div align="left">
+Use the same process to create another new role so that EC2 instances in the ECS cluster have appropriate permissions to access the container registry, auto-scale, etc. We will later assign this role to the EC2 instances in our ECS cluster.
+</div>
+
+In the Create Role screen, enter below two roles in the text field and select the two policies.
+
+```
+AmazonEC2ContainerServiceforEC2Role AmazonEC2ContainerServiceAutoscaleRole
+``` 
+
+In the Review screen, enter **ecslabinstanceprofile** for the Role name and click **Create Role**.
+
+**Note** : By default, ECS wizard creates ecsInstanceRole for you to use. However, it's a best practice to create a specific role for your use so that we can add more policies in the future when we need to.  
+
+Use the same process to create another new role so that EC2 Auto scaling will have necessary permissions to launch/terminate resources on your behalf.  
+
+
+Under the section  **Or select a service to view its use cases**, select 'EC2 Auto scaling' for the service which will use this role.  
+
+Under the section  **Select your use case**, select 'EC2 Auto scaling' and click on Next: Permissions  
+We will later use this role when we create auto scaling groups. 
+
+
+In the Create Role screen, enter[**AutoScalingServiceRolePolicy**](https://console.aws.amazon.com/iam/home?region=us-east-1#/policies/arn%3Aaws%3Aiam%3A%3Aaws%3Apolicy%2Faws-service-role%2FAutoScalingServiceRolePolicy)
+
+In the optional suffix, enter ec2 as shown below so that role become [**AWSServiceRoleForAutoScaling_ec2**](https://console.aws.amazon.com/iam/home?region=us-east-1#/roles/AWSServiceRoleForAutoScaling_ec2)
+

content/ecs-spot-capacity-providers/module-1/workspace1.md

@@ -0,0 +1,24 @@
+---
+title: "Create a Workspace"
+chapter: true
+weight: 5
+---
+
+## Create a Workspace
+
+_If you are running the workshop on your own, the Cloud9 workspace should be built by an IAM user with Administrator privileges, not the root account user. Please ensure you are logged in as an IAM user, not the root account user._
+
+We will create a Cloud9 environment first to execute all the commands needed for this workshop.
+
+1. Login into AWS console with your account credentials
+1. When working on AWS provided account your facilitator provides which Region to choose.
+1. On your own AWS account, select any region of your choice
+1. Select **Services** and type cloud9
+1. Select **Create environment**
+1. Name it **ecsspotworkshop**. Click " **Next Step**", keep all other defaults and click " **Next Step**".  keep all other defaults and click " **Create Environment**"
+1. When it comes up, customize the environment by closing the **welcome tab** and **lower work area** , and opening a new **terminal** tab in the main work area:
+1. If you like this theme, you can choose it yourself by selecting **View / Themes / Solarized / Solarized Dark** in the Cloud9 workspace menu.
+
+
+#### Your workspace should now look like this:
+![Cloud 9 Environment](/images/ecs-spot-capacity-providers/cloud9_4.png)

content/ecs-spot-capacity-providers/module-2/_index.md

@@ -26,4 +26,5 @@ Amazon ECS Capacity Provider - Managed Scaling
 
 <p style="text-align: justify;"> 
 When creating a capacity provider, you can optionally enable managed scaling. When managed scaling is enabled, Amazon ECS manages the scale-in and scale-out actions of the Auto Scaling group. On your behalf, Amazon ECS creates an AWS Auto Scaling scaling plan with a target tracking scaling policy based on the target capacity value you specify. Amazon ECS then associates this scaling plan with your Auto Scaling group. For each of the capacity providers with managed scaling enabled, an Amazon ECS managed CloudWatch metric with the prefix AWS/ECS/ManagedScaling is created along with two CloudWatch alarms. The CloudWatch metrics and alarms are used to monitor the container instance capacity in your Auto Scaling groups and will trigger the Auto Scaling group to scale in and scale out as needed.
-</p>
+</p>
+

content/ecs-spot-capacity-providers/module-2/fargate_service.md

@@ -0,0 +1,123 @@
+---
+title: "Create ECS Fargate Services"
+chapter: true
+weight: 2
+---
+
+###  Create ECS Fargate Services
+
+In this section, we will create 3 ECS Services to show how tasks can be deployed across FARGATE and FARGATE\_SPOT capacity providers(CP).
+
+
+| **Service Name** | **No. of Tasks** | **No. of Tasks on FARGATE CP** | **Number of Tasks on FARGATE_SPOT CP** | **CP Strategy** |
+| --- | --- |--- |--- |--- |
+| **webapp-fargate-service-fargate** | 2 | 2 | 0 | FARGATE Capacity Provider weight =1 |
+| **fargate-service-fargate-spot** | 2 | 0 | 2 | FARGATE_SPOT Capacity Provider weight =1 |
+| **fargate-service-fargate-mix** | 4 | 3 | 1 | FARGATE Capacity Provider weight =3 FARGATE_SPOT Capacity Provider weight =1 |
+
+We will be creating the ECS services and tasks in the new VPC we created in the Module-1 i.e. **Quick-Start-VPC**
+
+So let's first find the default public subnets created in this VPC. You can find the subnet IDs in this VPC in the  AWS console as shown below, under the VPC service.
+
+Alternatively you can run the below command to list all the subnets in this VPC
+
+```
+aws ec2 describe-subnets --filters "Name=tag:aws:cloudformation:stack-name,Values=Quick-Start-VPC" | jq -r '.Subnets[].SubnetId'
+```
+
+The output from above command looks like below.
+
+```
+subnet-07a877ee28959daa3
+subnet-015fc3e06f653980a
+subnet-003ef0ebc04c89b2d
+```
+
+Run the below command to set a variable for the subnets. We will use this variable in other steps.
+
+```
+export PUBLIC\_SUBNET\_LIST="subnet-07a877ee28959daa3,subnet-015fc3e06f653980a,subnet-003ef0ebc04c89b2d"
+```
+
+Now let's find the default security group created in this VPC. You can find it in the AWS console as follows.
+
+You can also run the below command to list the default security group in this VPC
+
+```
+export VPC\_ID=$(aws ec2 describe-vpcs --filters "Name=tag:aws:cloudformation:stack-name,Values=Quick-Start-VPC" | jq -r '.Vpcs[0].VpcId')
+ echo "Quick Start VPC ID is $VPC\_ID"
+```
+
+The output from above command looks like below.
+
+```
+Quick Start VPC ID is vpc-0a2fc4f24cbfab696
+```
+
+```
+export SECURITY\_GROUP=$( aws ec2 describe-security-groups --filters "Name=vpc-id,Values=$VPC\_ID" | jq -r '.SecurityGroups[0].GroupId')
+ echo "Default Security group is $SECURITY\_GROUP"
+```
+
+The output from above command looks like below.
+
+```
+Default Security group is sg-03ccfca80f9fddf4d
+```
+
+Deploy the service **webapp-fargate-service-fargate** using below command.
+
+```
+aws ecs create-service \
+     --capacity-provider-strategy capacityProvider=FARGATE,weight=1 \
+     --cluster EcsSpotWorkshopCluster \
+     --service-name webapp-fargate-service-fargate \
+     --task-definition webapp-fargate-task:1 \
+     --desired-count 2 \
+     --region $AWS\_REGION \
+     --network-configuration "awsvpcConfiguration={subnets=[$PUBLIC\_SUBNET\_LIST],securityGroups=[$SECURITY\_GROUP],assignPublicIp="ENABLED"}"
+
+```
+Note the capacity provider strategy used for this service.  It provides weight only for FARGATE capacity provider. This strategy overrides the default capacity provider strategy which is set to FARGATE capacity provider.
+
+That means ECS schedules all of the tasks (2 in this case) in service on the FARGATE Capacity providers.
+
+Deploy the service **webapp-fargate-service-fargate-spot** using below command
+
+```
+aws ecs create-service \
+	 --capacity-provider-strategy capacityProvider=FARGATE\_SPOT,weight=1 \
+     --cluster EcsSpotWorkshopCluster \
+     --service-name webapp-fargate-service-fargate-spot \
+     --task-definition webapp-fargate-task:1 \
+     --desired-count 2\
+     --region $AWS\_REGION \
+     --network-configuration "awsvpcConfiguration={subnets=[$PUBLIC\_SUBNET\_LIST],securityGroups=[$SECURITY\_GROUP],assignPublicIp="ENABLED"}"
+```
+
+Note the capacity provider strategy used for this service. It provides weight only for FARGATE\_SPOT capacity provider. This strategy overrides the default capacity provider strategy which is set to FARGATE capacity provider.
+
+That means ECS schedules all of the tasks (2 in this case) in service on the FARGATE\_SPOT Capacity providers.
+
+Deploy the service **webapp-fargate-service-fargate-mix** using below command
+
+```
+aws ecs create-service \
+	 --capacity-provider-strategy capacityProvider=FARGATE,weight=3 capacityProvider=FARGATE\_SPOT,weight=1 \
+     --cluster EcsSpotWorkshopCluster \
+     --service-name webapp-fargate-service-fargate-mix \
+     --task-definition webapp-fargate-task:1 \
+     --desired-count 4\
+     --region $AWS\_REGION \
+     --network-configuration "awsvpcConfiguration={subnets=[$PUBLIC\_SUBNET\_LIST],securityGroups=[$SECURITY\_GROUP],assignPublicIp="ENABLED"}"
+```
+
+Note the capacity provider strategy used for this service.  It provides a weight of 3 to FARGATE and 1 to FARGATE\_SPOT capacity provider. This strategy overrides the default capacity provider strategy which is set to FARGATE capacity provider.
+
+That means ECS schedules splits the total tasks (4 in this case) in 3:1 ratio between FARGATE and FARGATE\_SPOT Capacity providers.
+
+But how do you verify if ECS really scheduled the tasks in this way?
+
+Click on the service  **webapp-fargate-service-fargate-mix** and select Tasks Tab
+
+Click on each task and note the Capacity Provider

content/ecs-spot-capacity-providers/module-2/fargate_task.md

@@ -0,0 +1,19 @@
+---
+title: "Create ECS Fargate Tasks"
+chapter: true
+weight: 1
+---
+
+###  Create ECS Fargate Tasks
+
+In this section, we will create a task definition for for tasks to be launched on the Fargate Capacity Providers.
+
+Run the below command to create the task definition
+
+```
+aws ecs register-task-definition --cli-input-json file://webapp-fargate-task.jso
+```
+
+The task will look like this in console
+
+PIC: TBD

content/ecs-spot-capacity-providers/module-3/_index.md

@@ -83,3 +83,4 @@ To ensure that your containers exit gracefully before the task stops, the follow
 is using. Specifying a stopTimeout value gives you time between the moment the task state change event is received and the point at which the container is forcefully stopped. 
 
 • The **SIGTERM** signal must be received from within the container to perform any cleanup actions.
+