awslabs · ruecarlo · Dec 6, 2020 · Jul 27, 2020 · Jul 28, 2020 · Jul 31, 2020
diff --git a/README.md b/README.md
@@ -1,4 +1,4 @@
-## Ec2 Spot Workshops
+## EC2 Spot Workshops
 
 Collection of workshops to demonstrate best practices in using Amazon EC2 Spot Instances. https://aws.amazon.com/ec2/spot/
 
@@ -11,7 +11,7 @@ The content of the workshops is built using [hugo](https://gohugo.io/).
 ### Local Build
 To build the content
  * clone this repository
- * [install hugo](https://gohugo.io/getting-started/installing/)
+ * [install hugo](https://gohugo.io/getting-started/installing/). The website is currently running on Hugo 0.53, since we have some markdown issues with the latest versions. You can download the exact version here: https://github.com/gohugoio/hugo/releases/download/v0.53/hugo_0.53_Linux-64bit.tar.gz
  * The project uses [hugo learn](https://github.com/matcornic/hugo-theme-learn/) template as a git submodule. To update the content, execute the following code
 ```bash
 pushd themes/learn

diff --git a/config.toml b/config.toml
@@ -5,8 +5,6 @@ theme = "learn"
 uglyurls = true
 sectionPagesMenu = "main"
 pygmentsCodeFences = true
-googleAnalytics = "UA-136840914-1"
-
 
 [blackfriday]
 hrefTargetBlank = true
@@ -25,28 +23,30 @@ disableInlineCopyToClipBoard = true
 home = [ "HTML", "AMP", "RSS", "JSON"]
 page = [ "HTML", "AMP"]
 
-
+# All privacy options enabled. There should not be 
+# any cookies or privacy  
 [privacy]
   [privacy.disqus]
     disable = false
   [privacy.googleAnalytics]
-    anonymizeIP = false
-    disable = false
+    anonymizeIP = true
+    disable = true
     respectDoNotTrack = true
     useSessionStorage = false
   [privacy.instagram]
-    disable = false
+    disable = true
     simple = false
   [privacy.twitter]
-    disable = false
+    disable = true
     enableDNT = false
     simple = false
   [privacy.vimeo]
-    disable = false
+    disable = true
     simple = false
   [privacy.youtube]
     disable = false
-    privacyEnhanced = false
+    # This is required to avoid cookies on youtube integration
+    privacyEnhanced = true
 
 
 [[menu.shortcuts]]

diff --git a/content/nextflow-on-aws-batch/10_prerequisites/nf_aws_event.md b/content/nextflow-on-aws-batch/10_prerequisites/nf_aws_event.md
@@ -8,7 +8,7 @@ weight: 20
 
 {{% notice warning %}}
 Only complete this section if you are at an AWS hosted event (such as re:Invent,
-Kubecon, Immersion Day, or any other event hosted by an AWS employee). If you are running the workshop on your own, go to: [Start the workshop on your own]({{< relref "self_paced.md" >}}).
+Kubecon, Immersion Day, or any other event hosted by an AWS employee). If you are running the workshop on your own, go to: [Start the workshop on your own]({{< relref "nf_self_paced.md" >}}).
 {{% /notice %}}
 
 ### Login to the AWS Workshop Portal

diff --git a/content/nextflow-on-aws-batch/10_prerequisites/nf_self_paced.md b/content/nextflow-on-aws-batch/10_prerequisites/nf_self_paced.md
@@ -5,7 +5,7 @@ weight: 10
 ---
 
 {{% notice warning %}}
-Only complete this section if you are running the workshop on your own. If you are at an AWS hosted event (such as re:Invent, Kubecon, Immersion Day, etc), go to [Start the workshop at an AWS event]({{< relref "aws_event.md" >}}).
+Only complete this section if you are running the workshop on your own. If you are at an AWS hosted event (such as re:Invent, Kubecon, Immersion Day, etc), go to [Start the workshop at an AWS event]({{< relref "nf_aws_event.md" >}}).
 {{% /notice %}}
 
 ### Prepare for CleanUp

diff --git a/content/resources.md b/content/resources.md
@@ -16,9 +16,9 @@ disableToc: true
 
 ##### [ - [Blog post] New – EC2 Auto Scaling Groups With Multiple Instance Types & Purchase Options](https://aws.amazon.com/blogs/aws/new-ec2-auto-scaling-groups-with-multiple-instance-types-purchase-options/)
 
-
 ##### [ - [Blog post] Capacity-Optimized Spot Instance Allocation in Action at Mobileye and Skyscanner](https://aws.amazon.com/blogs/aws/capacity-optimized-spot-instance-allocation-in-action-at-mobileye-and-skyscanner/)
 
+##### [ - [Blog post] Proactively manage the Spot Instance lifecycle using the new Capacity Rebalancing feature for EC2 Auto Scaling](https://aws.amazon.com/blogs/compute/proactively-manage-spot-instance-lifecycle-using-the-new-capacity-rebalancing-feature-for-ec2-auto-scaling/)
 
 ##### [ - [AWS Online Tech Talks]: Optimize AI/ML, Kubernetes and Big Data Workloads at Your Startup](https://www.youtube.com/watch?v=2donGftFSIY)
 
@@ -33,6 +33,8 @@ disableToc: true
 
 ## Big-data and analytics
 
+##### [ - [Blog post] Optimizing Amazon EMR for resilience and cost with capacity-optimized Spot Instances](https://aws.amazon.com/blogs/big-data/optimizing-amazon-emr-for-resilience-and-cost-with-capacity-optimized-spot-instances/)
+
 ##### [ - [Blog post] Best practices for running Apache Spark applications using Amazon EC2 Spot Instances with Amazon EMR](https://aws.amazon.com/blogs/big-data/best-practices-for-running-apache-spark-applications-using-amazon-ec2-spot-instances-with-amazon-emr/)
 
 ##### [ - [Tutorial] Getting started tutorial: Optimizing Amazon EMR clusters for cost and scale with EC2 Spot Instances and Amazon EMR](https://aws.amazon.com/getting-started/hands-on/optimize-amazon-emr-clusters-with-ec2-spot/)

diff --git a/content/running-amazon-ec2-workloads-at-scale/create_asg.md b/content/running-amazon-ec2-workloads-at-scale/create_asg.md
@@ -12,7 +12,7 @@ sed -i.bak -e "s#%TargetGroupARN%#$tg_arn#g" -e "s/%publicSubnet1%/$publicSubnet
 
 **Challenge**
 
-The EC2 Auto Scaling group that you are going to deploy supports [multiple purchase options (On-Demand and Spot Instances) and EC2 instance types](https://docs.aws.amazon.com/autoscaling/ec2/serguide/asg-purchase-options.html). 
+The EC2 Auto Scaling group that you are going to deploy supports [multiple purchase options (On-Demand and Spot Instances) and EC2 instance types](https://docs.aws.amazon.com/autoscaling/ec2/userguide/asg-purchase-options.html). 
 
 * Examining the asg.json configuration file, can you determine what would be the different configuration options in the deployed ASG?
 * How many On-Demand and Spot Instances would be deployed?

diff --git a/content/using_ec2_spot_instances_with_eks/_index.md b/content/using_ec2_spot_instances_with_eks/_index.md
@@ -5,10 +5,10 @@ weight: 50
 pre: "<b>⁃ </b>"
 ---
 
-In this workshop, you learn how to provision, manage, and maintain your Amazon Kubernetes 
-clusters with Amazon EKS at any scale on Spot Instances to architect for optimizations on cost and scale. 
-We dive deep using hands-on material to provision and scale worker nodes, handle 
-interruptions, and design for fault tolerance.
+In this workshop, you will learn how to provision, manage, and maintain your Kubernetes 
+clusters with Amazon Elastic Kubernetes Service (Amazon EKS) at any scale on Spot Instances to architect for optimizations on cost and scale. 
+We will dive deep using hands-on material to provision and scale worker nodes, handle 
+Spot interruptions, and design for fault tolerance.
 
 This workshop is originally based on AWS [EKS Workshop](https://eksworkshop.com/). You can find 
 there more modules and learn about other Amazon Elastic Kubernetes Service best practices.

diff --git a/...t/using_ec2_spot_instances_with_eks/eksctl/create_eks_cluster_eksctl_command.md b/...t/using_ec2_spot_instances_with_eks/eksctl/create_eks_cluster_eksctl_command.md
@@ -0,0 +1,13 @@
+---
+title: "Create EKS cluster Command"
+chapter: false
+disableToc: true
+hidden: true
+---
+<!--
+This markdown file is used as part of another file using 'insert-md-from-file' shortcode
+-->
+
+```
+eksctl create cluster --version=1.18 --name=eksworkshop-eksctl --node-private-networking  --managed --nodes=2 --alb-ingress-access --region=${AWS_REGION} --node-labels="lifecycle=OnDemand,intent=control-apps" --asg-access
+```
diff --git a/content/using_ec2_spot_instances_with_eks/eksctl/launcheks.md b/content/using_ec2_spot_instances_with_eks/eksctl/launcheks.md
@@ -14,20 +14,7 @@ weight: 20
 
 {{%expand "Expand here to see the solution" %}}
 
-Use the [GetCallerIdentity](https://docs.aws.amazon.com/cli/latest/reference/sts/get-caller-identity.html) CLI command to validate that the Cloud9 IDE is using the correct IAM role.
-
-```
-aws sts get-caller-identity
-
-```
-
-{{% notice note %}}
-**Select the tab** and and validate the assumed role…
-{{% /notice %}}
-{{< tabs name="Region" >}}
-    {{< tab name="...ON YOUR OWN" include="../prerequisites/on_your_own_validaterole.md" />}}
-    {{< tab name="...AT AN AWS EVENT" include="../prerequisites/at_an_aws_validaterole.md" />}}
-{{< /tabs >}}
+{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/prerequisites/validate_workspace_role.md" %}}
 
 If you do not see the correct role, please go back and **[validate the IAM role]({{< relref "../prerequisites/update_workspaceiam.md" >}})** for troubleshooting.
 
@@ -37,12 +24,9 @@ If you do see the correct role, proceed to next step to create an EKS cluster.
 
 ### Create an EKS cluster
 
-The following command will create an eks cluster with the name `eksworkshop-eksctl`
-.It will also create a nodegroup with 2 on-demand instances.
+The following command will create an eks cluster with the name `eksworkshop-eksctl`. It will also create a nodegroup with 2 on-demand instances.
 
-```
-eksctl create cluster --version=1.16 --name=eksworkshop-eksctl --node-private-networking  --managed --nodes=2 --alb-ingress-access --region=${AWS_REGION} --node-labels="lifecycle=OnDemand,intent=control-apps" --asg-access
-```
+{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/eksctl/create_eks_cluster_eksctl_command.md" %}}
 
 eksctl allows us to pass parameters to initialize the cluster. While initializing the cluster, eksctl does also allow us to create nodegroups.
 
@@ -52,12 +36,4 @@ The managed nodegroup will have two m5.large nodes and it will bootstrap with th
 Launching EKS and all the dependencies will take approximately **15 minutes**
 {{% /notice %}}
 
-The command above, created a **Managed Nodegroup**. [Amazon EKS managed node groups](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html) automate the provisioning and lifecycle management of nodes. Managed Nodegroups use the latest [EKS-optimized AMIs](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html). The node run in your AWS account provisioned as apart of an EC2 Auto Scaling group that is managed for you by Amazon EKS. This means EKS takes care of the lifecycle management and undifferentiated heavy lifting on operations such as node updates, handling of terminations, gracefully drain of nodes to ensure that your applications stay available.
-
-
-
-
-
-
-
-
+The command above, created a **Managed Nodegroup**. [Amazon EKS managed node groups](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html) automate the provisioning and lifecycle management of nodes. Managed Nodegroups use the latest [EKS-optimized AMIs](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html). The node run in your AWS account provisioned as apart of an EC2 Auto Scaling group that is managed for you by Amazon EKS. This means EKS takes care of the lifecycle management and undifferentiated heavy lifting on operations such as node updates, handling of terminations, gracefully drain of nodes to ensure that your applications stay available.
diff --git a/content/using_ec2_spot_instances_with_eks/eksctl/prerequisites.md b/content/using_ec2_spot_instances_with_eks/eksctl/prerequisites.md
@@ -6,8 +6,9 @@ weight: 10
 
 For this module, we need to download the [eksctl](https://eksctl.io/) binary:
 ```
-export EKSCTL_VERSION=0.23.0
-curl --silent --location "https://github.com/weaveworks/eksctl/releases/download/${EKSCTL_VERSION}/eksctl_Linux_amd64.tar.gz" | tar xz -C /tmp
+export EKSCTL_VERSION=0.31.0
+curl --silent --location "https://github.com/weaveworks/eksctl/releases/download/${EKSCTL_VERSION}/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
+
 sudo mv -v /tmp/eksctl /usr/local/bin
 ```
 

diff --git a/content/using_ec2_spot_instances_with_eks/helm_root/deploy_metric_server.md b/content/using_ec2_spot_instances_with_eks/helm_root/deploy_metric_server.md
@@ -11,7 +11,7 @@ Metrics Server is a cluster-wide aggregator of resource usage data. These metric
 kubectl create namespace metrics
 helm install metrics-server \
     stable/metrics-server \
-    --version 2.10.0 \
+    --version 2.11.4 \
     --namespace metrics
 ```
 

diff --git a/content/using_ec2_spot_instances_with_eks/helm_root/helm_deploy.md b/content/using_ec2_spot_instances_with_eks/helm_root/helm_deploy.md
@@ -26,7 +26,7 @@ Homebrew on macOS.
 Download the `stable` repository so we have something to start with:
 
 ```sh
-helm repo add stable https://kubernetes-charts.storage.googleapis.com/
+helm repo add stable https://charts.helm.sh/stable/
 helm repo update
 ```
 

diff --git a/content/using_ec2_spot_instances_with_eks/helm_root/install_kube_ops_view.md b/content/using_ec2_spot_instances_with_eks/helm_root/install_kube_ops_view.md
@@ -14,6 +14,7 @@ helm install kube-ops-view \
 stable/kube-ops-view \
 --set service.type=LoadBalancer \
 --set nodeSelector.intent=control-apps \
+--version 1.2.4 \
 --set rbac.create=True
 ```
 
@@ -32,8 +33,8 @@ helm list
 
 should display : 
 ```
-NAME            REVISION        UPDATED                         STATUS          CHART                   APP VERSION     NAMESPACE
-kube-ops-view   1               Sun Sep 22 11:47:31 2019        DEPLOYED        kube-ops-view-1.1.0     0.11            default  
+NAME            NAMESPACE   REVISION   UPDATED               STATUS     CHART              
+kube-ops-view   default     1          2020-11-20 05:16:47   deployed   kube-ops-view-1.2.4
 ```
 
 With this we can explore kube-ops-view output by checking the details about the newly service created. 

diff --git a/content/using_ec2_spot_instances_with_eks/jenkins/setup_agents.md b/content/using_ec2_spot_instances_with_eks/jenkins/setup_agents.md
@@ -34,6 +34,9 @@ nodeGroups:
           k8s.io/cluster-autoscaler/node-template/label/lifecycle: Ec2Spot
           k8s.io/cluster-autoscaler/node-template/label/intent: jenkins-agents
           k8s.io/cluster-autoscaler/node-template/label/aws.amazon.com/spot: "true"
+      iam:
+        withAddonPolicies:
+          autoScaler: true            
 EoF
 ```
 
@@ -53,10 +56,10 @@ The creation of the workers will take about 3 minutes.
 #### Instructing Jenkins to run jobs on the new, Spot dedicated nodegroup
 1. In  the Jenkins dashboard, browse to **Manage Jenkins** -> **Manage Node and Clouds**
 1. On the left hand side click on the **Configure Clouds** link. That will take you to the cloud configuration where Kubernetes.
-1. Click on the **Pod Templates...** button to expand the default pod template definition  and then click again on **Pod Template Details...*** 
-1. Change the default pod name, attribute **Name** from `defualt` to `jenkins-agent`. We want to be able to identify the pods that are running in our clusters by name. 
+1. Click on the **Pod Templates...** button to expand the default pod template definition and then click again on **Pod Template Details...*** 
+1. Change the default pod name, attribute **Name** from `defualt` to `jenkins-agent`. We want to be able to identify the pods that are running in our clusters by name.
 ![Jenkins Pod Setup 1](/images/using_ec2_spot_instances_with_eks/jenkins/jenkinslabels-1.png)
-1. At the bottom of the page, near the end of the Pod template section, In the **Node Selector** Pod , add the following: `intent=jenkins-agents,lifecycle=Ec2Spot`
+1. At the bottom of the page, near the end of the Pod template section, for the **Node Selector** parameter , add the following: `intent=jenkins-agents,lifecycle=Ec2Spot` in order to instruct the Jenkins agent pods to run on the dedicated node group.
 ![Jenkins Pod Setup 2](/images/using_ec2_spot_instances_with_eks/jenkins/jenkinslabels-2.png)
 1. Click **Save**
 

diff --git a/content/using_ec2_spot_instances_with_eks/jenkins/setup_jenkins.md b/content/using_ec2_spot_instances_with_eks/jenkins/setup_jenkins.md
@@ -7,7 +7,7 @@ weight: 30
 #### Install Jenkins
 
 ```
-helm install cicd stable/jenkins --set rbac.create=true,master.servicePort=80,master.serviceType=LoadBalancer
+helm install cicd jenkinsci/jenkins --set rbac.create=true,master.servicePort=80,master.serviceType=LoadBalancer,master.JCasC.enabled=false,master.enableXmlConfig=true
 ```
 
 The output of this command will give you some additional information such as the

diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/at_an_aws_updateiam.md b/content/using_ec2_spot_instances_with_eks/prerequisites/at_an_aws_updateiam.md
@@ -7,8 +7,8 @@ hidden: true
 
 ## Attach the IAM role to your Workspace
 
-1. Follow [this deep link to find your Cloud9 EC2 instance](https://console.aws.amazon.com/ec2/v2/home?#Instances:tag:Name=aws-cloud9-.*workshop.*;sort=desc:launchTime)
-1. Select the instance, then choose **Actions / Instance Settings / Attach/Replace IAM Role**
+1. Follow this [deep link to find your Cloud9 EC2 instance](https://console.aws.amazon.com/ec2/v2/home?#Instances:tag:Name=aws-cloud9-eksworkshop;sort=desc:launchTime).
+1. Select the instance, then choose **Actions / Security / Modify IAM role**
 ![c9instancerole](/images/using_ec2_spot_instances_with_eks/prerequisites/c9instancerole.png)
-1. Choose **TeamRoleInstance** from the **IAM Role** drop down, and select **Apply**
+1. Choose **TeamRoleInstance** from the **IAM role** drop down, and select **Save**
 ![c9attachrole](/images/using_ec2_spot_instances_with_eks/prerequisites/c9attachroleee.png)
diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/aws_event.md b/content/using_ec2_spot_instances_with_eks/prerequisites/aws_event.md
@@ -23,4 +23,57 @@ If you are at an AWS event, an AWS account was created for you to use throughout
 
 You are now logged in to the AWS console in an account that was created for you, and will be available only throughout the workshop run time.
 
-Once you have completed the step above, **you can head straight to [Create a Workspace]({{< relref "workspace.md" >}})**
+{{% notice info %}}
+In the interest of time for shorter events we sometimes deploy the resources required as a prerequisite for you. If you were told so, please review the cloudformation outputs of the stack that was deployed by **expanding the instructions below**.
+{{% /notice %}}
+
+{{%expand "Click to reveal detailed instructions" %}}
+
+#### What resources are already deployed {#resources_deployed}
+
+We have deployed the below resources required to get started with the workshop using a CloudFormation Template (**[eks-spot-workshop-quickstarter-cnf.yml] (prerequisites.files/eks-spot-workshop-quickstart-cnf.yml)**), Please reference the below  resources created by the stack.
+
++ An [AWS Cloud9](https://console.aws.amazon.com/cloud9) workspace with
+    - An IAM role created and attached to the workspace with Administrator access
+    - Kubernetes tools installed (kubectl, jq and envsubst)
+    - awscli upgraded to v2
+    - Created and imported a key pair to Amazon EC2
+    - [eksctl](https://eksctl.io/) installed, The official CLI for Amazon EKS 
+
++ An EKS cluster with the name `eksworkshop-eksctl` and a [EKS managed node group](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html)  with 2 on-demand instances.
+
+{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/eksctl/create_eks_cluster_eksctl_command.md" %}}
+
+
+#### Use your resources 
+
+In this workshop, you'll need to reference the resources created by the CloudFormation stack that we setup for you.
+
+1. On the [AWS CloudFormation console](https://console.aws.amazon.com/cloudformation), select the stack name that starts with **mod-** in the list.
+
+1. In the stack details pane, click the **Outputs** tab.
+
+![cnf_output](/images/using_ec2_spot_instances_with_eks/prerequisites/cnf_output.png)
+
+It is recommended that you keep this window open so you can easily refer to the outputs and resources throughout the workshop.
+
+{{% notice info %}}
+you will notice additional Cloudformation stacks were also deployed which is the result of the stack that starts with **mod-**. One to deploy the Cloud9 Workspace and two other to create the EKS cluster and managed nodegroup.
+{{% /notice %}}
+
+#### Launch your Cloud9 workspace
+
+- Click on the url against `Cloud9IDE` from the outputs
+
+{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/prerequisites/workspace_at_launch.md" %}}
+
+{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/prerequisites/update_workspace_settings.md" %}}
+
+{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/prerequisites/validate_workspace_role.md" %}}
+
+{{% notice note %}}
+Since we have already setup the prerequisites, **you can head straight to [Test the Cluster]({{<  relref "../eksctl/test.md"  >}})**
+{{% /notice %}}
+
+{{% /expand%}}
+
diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/k8stools.md b/content/using_ec2_spot_instances_with_eks/prerequisites/k8stools.md
@@ -14,8 +14,9 @@ for the download links.](https://docs.aws.amazon.com/eks/latest/userguide/gettin
 {{% /notice %}}
 
 #### Install kubectl
+
 ```
-export KUBECTL_VERSION=v1.16.12
+export KUBECTL_VERSION=v1.18.12
 sudo curl --silent --location -o /usr/local/bin/kubectl https://storage.googleapis.com/kubernetes-release/release/${KUBECTL_VERSION}/bin/linux/amd64/kubectl
 sudo chmod +x /usr/local/bin/kubectl
 ```
@@ -31,4 +32,4 @@ for command in kubectl jq envsubst
   do
     which $command &>/dev/null && echo "$command in path" || echo "$command NOT FOUND"
   done
-```
+```