diff --git a/content/using_ec2_spot_instances_with_eks/introduction/_index.md b/content/using_ec2_spot_instances_with_eks/005_introduction/_index.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/introduction/_index.md rename to content/using_ec2_spot_instances_with_eks/005_introduction/_index.md diff --git a/content/using_ec2_spot_instances_with_eks/introduction/architecture/_index.md b/content/using_ec2_spot_instances_with_eks/005_introduction/architecture/_index.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/introduction/architecture/_index.md rename to content/using_ec2_spot_instances_with_eks/005_introduction/architecture/_index.md diff --git a/content/using_ec2_spot_instances_with_eks/introduction/architecture/architecture_control.md b/content/using_ec2_spot_instances_with_eks/005_introduction/architecture/architecture_control.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/introduction/architecture/architecture_control.md rename to content/using_ec2_spot_instances_with_eks/005_introduction/architecture/architecture_control.md diff --git a/content/using_ec2_spot_instances_with_eks/introduction/architecture/architecture_control_and_data_overview.md b/content/using_ec2_spot_instances_with_eks/005_introduction/architecture/architecture_control_and_data_overview.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/introduction/architecture/architecture_control_and_data_overview.md rename to content/using_ec2_spot_instances_with_eks/005_introduction/architecture/architecture_control_and_data_overview.md diff --git a/content/using_ec2_spot_instances_with_eks/introduction/architecture/architecture_worker.md b/content/using_ec2_spot_instances_with_eks/005_introduction/architecture/architecture_worker.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/introduction/architecture/architecture_worker.md rename to content/using_ec2_spot_instances_with_eks/005_introduction/architecture/architecture_worker.md diff --git a/content/using_ec2_spot_instances_with_eks/introduction/architecture/cluster_setup_options.md b/content/using_ec2_spot_instances_with_eks/005_introduction/architecture/cluster_setup_options.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/introduction/architecture/cluster_setup_options.md rename to content/using_ec2_spot_instances_with_eks/005_introduction/architecture/cluster_setup_options.md diff --git a/content/using_ec2_spot_instances_with_eks/introduction/basics/_index.md b/content/using_ec2_spot_instances_with_eks/005_introduction/basics/_index.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/introduction/basics/_index.md rename to content/using_ec2_spot_instances_with_eks/005_introduction/basics/_index.md diff --git a/content/using_ec2_spot_instances_with_eks/introduction/basics/concepts_nodes.md b/content/using_ec2_spot_instances_with_eks/005_introduction/basics/concepts_nodes.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/introduction/basics/concepts_nodes.md rename to content/using_ec2_spot_instances_with_eks/005_introduction/basics/concepts_nodes.md diff --git a/content/using_ec2_spot_instances_with_eks/introduction/basics/concepts_objects.md b/content/using_ec2_spot_instances_with_eks/005_introduction/basics/concepts_objects.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/introduction/basics/concepts_objects.md rename to content/using_ec2_spot_instances_with_eks/005_introduction/basics/concepts_objects.md diff --git a/content/using_ec2_spot_instances_with_eks/introduction/basics/concepts_objects_details_1.md b/content/using_ec2_spot_instances_with_eks/005_introduction/basics/concepts_objects_details_1.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/introduction/basics/concepts_objects_details_1.md rename to content/using_ec2_spot_instances_with_eks/005_introduction/basics/concepts_objects_details_1.md diff --git a/content/using_ec2_spot_instances_with_eks/introduction/basics/concepts_objects_details_2.md b/content/using_ec2_spot_instances_with_eks/005_introduction/basics/concepts_objects_details_2.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/introduction/basics/concepts_objects_details_2.md rename to content/using_ec2_spot_instances_with_eks/005_introduction/basics/concepts_objects_details_2.md diff --git a/content/using_ec2_spot_instances_with_eks/introduction/basics/what_is_k8s.md b/content/using_ec2_spot_instances_with_eks/005_introduction/basics/what_is_k8s.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/introduction/basics/what_is_k8s.md rename to content/using_ec2_spot_instances_with_eks/005_introduction/basics/what_is_k8s.md diff --git a/content/using_ec2_spot_instances_with_eks/introduction/eks/_index.md b/content/using_ec2_spot_instances_with_eks/005_introduction/eks/_index.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/introduction/eks/_index.md rename to content/using_ec2_spot_instances_with_eks/005_introduction/eks/_index.md diff --git a/content/using_ec2_spot_instances_with_eks/introduction/eks/eks_control_plane.md b/content/using_ec2_spot_instances_with_eks/005_introduction/eks/eks_control_plane.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/introduction/eks/eks_control_plane.md rename to content/using_ec2_spot_instances_with_eks/005_introduction/eks/eks_control_plane.md diff --git a/content/using_ec2_spot_instances_with_eks/introduction/eks/eks_customers.md b/content/using_ec2_spot_instances_with_eks/005_introduction/eks/eks_customers.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/introduction/eks/eks_customers.md rename to content/using_ec2_spot_instances_with_eks/005_introduction/eks/eks_customers.md diff --git a/content/using_ec2_spot_instances_with_eks/introduction/eks/eks_high_architecture.md b/content/using_ec2_spot_instances_with_eks/005_introduction/eks/eks_high_architecture.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/introduction/eks/eks_high_architecture.md rename to content/using_ec2_spot_instances_with_eks/005_introduction/eks/eks_high_architecture.md diff --git a/content/using_ec2_spot_instances_with_eks/introduction/eks/eks_high_level.md b/content/using_ec2_spot_instances_with_eks/005_introduction/eks/eks_high_level.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/introduction/eks/eks_high_level.md rename to content/using_ec2_spot_instances_with_eks/005_introduction/eks/eks_high_level.md diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/_index.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/_index.md similarity index 57% rename from content/using_ec2_spot_instances_with_eks/prerequisites/_index.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/_index.md index c6bae9b7..ed72eaac 100644 --- a/content/using_ec2_spot_instances_with_eks/prerequisites/_index.md +++ b/content/using_ec2_spot_instances_with_eks/010_prerequisites/_index.md @@ -7,7 +7,7 @@ weight: 10 # Getting Started To start the workshop, follow one of the following depending on whether you are... -* ...[running the workshop on your own (in your own account)]({{< ref "/using_ec2_spot_instances_with_eks/prerequisites/self_paced.md" >}}), or -* ...[attending an AWS hosted event (using AWS provided hashes)]({{< ref "/using_ec2_spot_instances_with_eks/prerequisites/aws_event.md" >}}) +* ...[running the workshop on your own (in your own account)]({{< ref "/using_ec2_spot_instances_with_eks/010_prerequisites/self_paced.md" >}}), or +* ...[attending an AWS hosted event (using AWS provided hashes)]({{< ref "/using_ec2_spot_instances_with_eks/010_prerequisites/aws_event.md" >}}) -Once you have completed with either setup, continue with **[Create a Workspace]({{< ref "/using_ec2_spot_instances_with_eks/prerequisites/workspace.md" >}})** +Once you have completed with either setup, continue with **[Create a Workspace]({{< ref "/using_ec2_spot_instances_with_eks/010_prerequisites/workspace.md" >}})** diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/ap-southeast-1.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/ap-southeast-1.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/prerequisites/ap-southeast-1.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/ap-southeast-1.md diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/at_an_aws_updateiam.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/at_an_aws_updateiam.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/prerequisites/at_an_aws_updateiam.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/at_an_aws_updateiam.md diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/at_an_aws_validaterole.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/at_an_aws_validaterole.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/prerequisites/at_an_aws_validaterole.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/at_an_aws_validaterole.md diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/attach_workspaceiam.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/attach_workspaceiam.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/prerequisites/attach_workspaceiam.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/attach_workspaceiam.md diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/aws_event.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/aws_event.md similarity index 89% rename from content/using_ec2_spot_instances_with_eks/prerequisites/aws_event.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/aws_event.md index 9d592d81..917dc36f 100644 --- a/content/using_ec2_spot_instances_with_eks/prerequisites/aws_event.md +++ b/content/using_ec2_spot_instances_with_eks/010_prerequisites/aws_event.md @@ -9,7 +9,7 @@ weight: 20 {{% notice warning %}} Only complete this section if you are at an AWS hosted event (such as re:Invent, Kubecon, Immersion Day, or any other event hosted by an AWS employee). If you -are running the workshop on your own, go to: [Start the workshop on your own]({{< ref "/using_ec2_spot_instances_with_eks/prerequisites/self_paced.md" >}}). +are running the workshop on your own, go to: [Start the workshop on your own]({{< ref "/using_ec2_spot_instances_with_eks/010_prerequisites/self_paced.md" >}}). {{% /notice %}} ### Login to the AWS Workshop Portal @@ -42,7 +42,7 @@ We have deployed the below resources required to get started with the workshop u + An EKS cluster with the name `eksworkshop-eksctl` and a [EKS managed node group](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html) with 2 on-demand instances. -{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/eksctl/create_eks_cluster_eksctl_command.md" %}} +{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/020_eksctl/create_eks_cluster_eksctl_command.md" %}} #### Use your resources @@ -53,7 +53,7 @@ In this workshop, you'll need to reference the resources created by the CloudFor 1. In the stack details pane, click the **Outputs** tab. -![cnf_output](/images/using_ec2_spot_instances_with_eks/prerequisites/cnf_output.png) +![cnf_output](/images/using_ec2_spot_instances_with_eks/010_prerequisites/cnf_output.png) It is recommended that you keep this window open so you can easily refer to the outputs and resources throughout the workshop. @@ -65,9 +65,9 @@ you will notice additional Cloudformation stacks were also deployed which is the - Click on the url against `Cloud9IDE` from the outputs -{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/prerequisites/workspace_at_launch.md" %}} +{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/010_prerequisites/workspace_at_launch.md" %}} -{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/prerequisites/update_workspace_settings.md" %}} +{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/010_prerequisites/update_workspace_settings.md" %}} ### Validate the IAM role {#validate_iam} @@ -78,10 +78,10 @@ aws sts get-caller-identity ``` -{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/prerequisites/at_an_aws_validaterole.md" %}} +{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/010_prerequisites/at_an_aws_validaterole.md" %}} {{% notice note %}} -Since we have already setup the prerequisites, **you can head straight to [Test the Cluster]({{< relref "/using_ec2_spot_instances_with_eks/eksctl/test.md" >}})** +Since we have already setup the prerequisites, **you can head straight to [Test the Cluster]({{< relref "/using_ec2_spot_instances_with_eks/020_eksctl/test.md" >}})** {{% /notice %}} {{% /expand%}} diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/awscli.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/awscli.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/prerequisites/awscli.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/awscli.md diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/clone.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/clone.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/prerequisites/clone.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/clone.md diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/eu-west-1.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/eu-west-1.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/prerequisites/eu-west-1.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/eu-west-1.md diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/k8stools.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/k8stools.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/prerequisites/k8stools.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/k8stools.md diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/on_your_own_updateiam.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/on_your_own_updateiam.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/prerequisites/on_your_own_updateiam.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/on_your_own_updateiam.md diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/on_your_own_validaterole.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/on_your_own_validaterole.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/prerequisites/on_your_own_validaterole.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/on_your_own_validaterole.md diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/prerequisites.files/eks-spot-workshop-quickstart-cnf.yml b/content/using_ec2_spot_instances_with_eks/010_prerequisites/prerequisites.files/eks-spot-workshop-quickstart-cnf.yml similarity index 100% rename from content/using_ec2_spot_instances_with_eks/prerequisites/prerequisites.files/eks-spot-workshop-quickstart-cnf.yml rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/prerequisites.files/eks-spot-workshop-quickstart-cnf.yml diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/self_paced.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/self_paced.md similarity index 91% rename from content/using_ec2_spot_instances_with_eks/prerequisites/self_paced.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/self_paced.md index d1616205..1862d908 100644 --- a/content/using_ec2_spot_instances_with_eks/prerequisites/self_paced.md +++ b/content/using_ec2_spot_instances_with_eks/010_prerequisites/self_paced.md @@ -5,7 +5,7 @@ weight: 10 --- {{% notice warning %}} -Only complete this section if you are running the workshop on your own. If you are at an AWS hosted event (such as re:Invent, Kubecon, Immersion Day, etc), go to [Start the workshop at an AWS event]({{< ref "/using_ec2_spot_instances_with_eks/prerequisites/aws_event.md" >}}). +Only complete this section if you are running the workshop on your own. If you are at an AWS hosted event (such as re:Invent, Kubecon, Immersion Day, etc), go to [Start the workshop at an AWS event]({{< ref "/using_ec2_spot_instances_with_eks/010_prerequisites/aws_event.md" >}}). {{% /notice %}} ### Running the workshop on your own @@ -34,4 +34,4 @@ as an IAM user with administrator access to the AWS account: ![Login URL](/images/using_ec2_spot_instances_with_eks/prerequisites/iam-4-save-url.png) -Once you have completed the step above, **you can head straight to [Create a Workspace]({{< ref "/using_ec2_spot_instances_with_eks/prerequisites/workspace.md" >}})** \ No newline at end of file +Once you have completed the step above, **you can head straight to [Create a Workspace]({{< ref "/using_ec2_spot_instances_with_eks/010_prerequisites/workspace.md" >}})** \ No newline at end of file diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/sshkey.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/sshkey.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/prerequisites/sshkey.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/sshkey.md diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/update_workspace_settings.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/update_workspace_settings.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/prerequisites/update_workspace_settings.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/update_workspace_settings.md diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/update_workspaceiam.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/update_workspaceiam.md similarity index 86% rename from content/using_ec2_spot_instances_with_eks/prerequisites/update_workspaceiam.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/update_workspaceiam.md index ac0c0834..11fbdceb 100644 --- a/content/using_ec2_spot_instances_with_eks/prerequisites/update_workspaceiam.md +++ b/content/using_ec2_spot_instances_with_eks/010_prerequisites/update_workspaceiam.md @@ -4,7 +4,7 @@ chapter: false weight: 60 --- -{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/prerequisites/update_workspace_settings.md" %}} +{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/010_prerequisites/update_workspace_settings.md" %}} We should configure our aws cli with our current region as default: ``` @@ -17,5 +17,5 @@ aws configure set default.region ${AWS_REGION} aws configure get default.region ``` -{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/prerequisites/validate_workspace_role.md" %}} +{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/010_prerequisites/validate_workspace_role.md" %}} diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/us-east-1.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/us-east-1.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/prerequisites/us-east-1.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/us-east-1.md diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/us-east-2.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/us-east-2.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/prerequisites/us-east-2.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/us-east-2.md diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/us-west-2.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/us-west-2.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/prerequisites/us-west-2.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/us-west-2.md diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/validate_workspace_role.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/validate_workspace_role.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/prerequisites/validate_workspace_role.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/validate_workspace_role.md diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/workspace.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/workspace.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/prerequisites/workspace.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/workspace.md diff --git a/content/using_ec2_spot_instances_with_eks/prerequisites/workspace_at_launch.md b/content/using_ec2_spot_instances_with_eks/010_prerequisites/workspace_at_launch.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/prerequisites/workspace_at_launch.md rename to content/using_ec2_spot_instances_with_eks/010_prerequisites/workspace_at_launch.md diff --git a/content/using_ec2_spot_instances_with_eks/eksctl/_index.md b/content/using_ec2_spot_instances_with_eks/020_eksctl/_index.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/eksctl/_index.md rename to content/using_ec2_spot_instances_with_eks/020_eksctl/_index.md diff --git a/content/using_ec2_spot_instances_with_eks/eksctl/create_eks_cluster_eksctl_command.md b/content/using_ec2_spot_instances_with_eks/020_eksctl/create_eks_cluster_eksctl_command.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/eksctl/create_eks_cluster_eksctl_command.md rename to content/using_ec2_spot_instances_with_eks/020_eksctl/create_eks_cluster_eksctl_command.md diff --git a/content/using_ec2_spot_instances_with_eks/eksctl/launcheks.files/eksworkshop-kubeflow.yml.template b/content/using_ec2_spot_instances_with_eks/020_eksctl/launcheks.files/eksworkshop-kubeflow.yml.template similarity index 100% rename from content/using_ec2_spot_instances_with_eks/eksctl/launcheks.files/eksworkshop-kubeflow.yml.template rename to content/using_ec2_spot_instances_with_eks/020_eksctl/launcheks.files/eksworkshop-kubeflow.yml.template diff --git a/content/using_ec2_spot_instances_with_eks/eksctl/launcheks.files/eksworkshop.yml.template b/content/using_ec2_spot_instances_with_eks/020_eksctl/launcheks.files/eksworkshop.yml.template similarity index 100% rename from content/using_ec2_spot_instances_with_eks/eksctl/launcheks.files/eksworkshop.yml.template rename to content/using_ec2_spot_instances_with_eks/020_eksctl/launcheks.files/eksworkshop.yml.template diff --git a/content/using_ec2_spot_instances_with_eks/eksctl/launcheks.md b/content/using_ec2_spot_instances_with_eks/020_eksctl/launcheks.md similarity index 82% rename from content/using_ec2_spot_instances_with_eks/eksctl/launcheks.md rename to content/using_ec2_spot_instances_with_eks/020_eksctl/launcheks.md index 6abc101f..d3446e69 100644 --- a/content/using_ec2_spot_instances_with_eks/eksctl/launcheks.md +++ b/content/using_ec2_spot_instances_with_eks/020_eksctl/launcheks.md @@ -6,7 +6,7 @@ weight: 20 {{% notice warning %}} -**DO NOT PROCEED** with this step unless you have [validated the IAM role]({{< relref "../prerequisites/update_workspaceiam.md#validate_iam" >}}) in use by the Cloud9 IDE. You will not be able to run the necessary kubectl commands in the later modules unless the EKS cluster is built using the IAM role. +**DO NOT PROCEED** with this step unless you have [validated the IAM role]({{< relref "../010_prerequisites/update_workspaceiam.md#validate_iam" >}}) in use by the Cloud9 IDE. You will not be able to run the necessary kubectl commands in the later modules unless the EKS cluster is built using the IAM role. {{% /notice %}} #### Challenge: @@ -14,9 +14,9 @@ weight: 20 {{%expand "Expand here to see the solution" %}} -{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/prerequisites/validate_workspace_role.md" %}} +{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/010_prerequisites/validate_workspace_role.md" %}} -If you do not see the correct role, please go back and **[validate the IAM role]({{< relref "../prerequisites/update_workspaceiam.md" >}})** for troubleshooting. +If you do not see the correct role, please go back and **[validate the IAM role]({{< relref "../010_prerequisites/update_workspaceiam.md" >}})** for troubleshooting. If you do see the correct role, proceed to next step to create an EKS cluster. {{% /expand %}} @@ -26,7 +26,7 @@ If you do see the correct role, proceed to next step to create an EKS cluster. The following command will create an eks cluster with the name `eksworkshop-eksctl`. It will also create a nodegroup with 2 on-demand instances. -{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/eksctl/create_eks_cluster_eksctl_command.md" %}} +{{% insert-md-from-file file="using_ec2_spot_instances_with_eks/020_eksctl/create_eks_cluster_eksctl_command.md" %}} eksctl allows us to pass parameters to initialize the cluster. While initializing the cluster, eksctl does also allow us to create nodegroups. diff --git a/content/using_ec2_spot_instances_with_eks/eksctl/prerequisites.md b/content/using_ec2_spot_instances_with_eks/020_eksctl/prerequisites.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/eksctl/prerequisites.md rename to content/using_ec2_spot_instances_with_eks/020_eksctl/prerequisites.md diff --git a/content/using_ec2_spot_instances_with_eks/eksctl/test.md b/content/using_ec2_spot_instances_with_eks/020_eksctl/test.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/eksctl/test.md rename to content/using_ec2_spot_instances_with_eks/020_eksctl/test.md diff --git a/content/using_ec2_spot_instances_with_eks/helm_root/_index.md b/content/using_ec2_spot_instances_with_eks/030_helm_root/_index.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/helm_root/_index.md rename to content/using_ec2_spot_instances_with_eks/030_helm_root/_index.md diff --git a/content/using_ec2_spot_instances_with_eks/helm_root/deploy_metric_server.md b/content/using_ec2_spot_instances_with_eks/030_helm_root/deploy_metric_server.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/helm_root/deploy_metric_server.md rename to content/using_ec2_spot_instances_with_eks/030_helm_root/deploy_metric_server.md diff --git a/content/using_ec2_spot_instances_with_eks/helm_root/helm_deploy.md b/content/using_ec2_spot_instances_with_eks/030_helm_root/helm_deploy.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/helm_root/helm_deploy.md rename to content/using_ec2_spot_instances_with_eks/030_helm_root/helm_deploy.md diff --git a/content/using_ec2_spot_instances_with_eks/helm_root/install_kube_ops_view.md b/content/using_ec2_spot_instances_with_eks/030_helm_root/install_kube_ops_view.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/helm_root/install_kube_ops_view.md rename to content/using_ec2_spot_instances_with_eks/030_helm_root/install_kube_ops_view.md diff --git a/content/using_ec2_spot_instances_with_eks/spotmanagednodegroups/_index.md b/content/using_ec2_spot_instances_with_eks/040_spotmanagednodegroups/_index.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/spotmanagednodegroups/_index.md rename to content/using_ec2_spot_instances_with_eks/040_spotmanagednodegroups/_index.md diff --git a/content/using_ec2_spot_instances_with_eks/spotmanagednodegroups/selecting_instance_types.md b/content/using_ec2_spot_instances_with_eks/040_spotmanagednodegroups/selecting_instance_types.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/spotmanagednodegroups/selecting_instance_types.md rename to content/using_ec2_spot_instances_with_eks/040_spotmanagednodegroups/selecting_instance_types.md diff --git a/content/using_ec2_spot_instances_with_eks/spotmanagednodegroups/spotlifecycle.md b/content/using_ec2_spot_instances_with_eks/040_spotmanagednodegroups/spotlifecycle.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/spotmanagednodegroups/spotlifecycle.md rename to content/using_ec2_spot_instances_with_eks/040_spotmanagednodegroups/spotlifecycle.md diff --git a/content/using_ec2_spot_instances_with_eks/spotmanagednodegroups/why_select_managed_node_group.md b/content/using_ec2_spot_instances_with_eks/040_spotmanagednodegroups/why_select_managed_node_group.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/spotmanagednodegroups/why_select_managed_node_group.md rename to content/using_ec2_spot_instances_with_eks/040_spotmanagednodegroups/why_select_managed_node_group.md diff --git a/content/using_ec2_spot_instances_with_eks/spotmanagednodegroups/workers_eksctl.files/spot_nodegroup_4vcpu_16gb.yml b/content/using_ec2_spot_instances_with_eks/040_spotmanagednodegroups/workers_eksctl.files/spot_nodegroup_4vcpu_16gb.yml similarity index 100% rename from content/using_ec2_spot_instances_with_eks/spotmanagednodegroups/workers_eksctl.files/spot_nodegroup_4vcpu_16gb.yml rename to content/using_ec2_spot_instances_with_eks/040_spotmanagednodegroups/workers_eksctl.files/spot_nodegroup_4vcpu_16gb.yml diff --git a/content/using_ec2_spot_instances_with_eks/spotmanagednodegroups/workers_eksctl.files/spot_nodegroup_8vcpu_32gb.yml b/content/using_ec2_spot_instances_with_eks/040_spotmanagednodegroups/workers_eksctl.files/spot_nodegroup_8vcpu_32gb.yml similarity index 100% rename from content/using_ec2_spot_instances_with_eks/spotmanagednodegroups/workers_eksctl.files/spot_nodegroup_8vcpu_32gb.yml rename to content/using_ec2_spot_instances_with_eks/040_spotmanagednodegroups/workers_eksctl.files/spot_nodegroup_8vcpu_32gb.yml diff --git a/content/using_ec2_spot_instances_with_eks/spotmanagednodegroups/workers_eksctl.md b/content/using_ec2_spot_instances_with_eks/040_spotmanagednodegroups/workers_eksctl.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/spotmanagednodegroups/workers_eksctl.md rename to content/using_ec2_spot_instances_with_eks/040_spotmanagednodegroups/workers_eksctl.md diff --git a/content/using_ec2_spot_instances_with_eks/spotworkers/_index.md b/content/using_ec2_spot_instances_with_eks/050_selfmanagednodegroups/_index.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/spotworkers/_index.md rename to content/using_ec2_spot_instances_with_eks/050_selfmanagednodegroups/_index.md diff --git a/content/using_ec2_spot_instances_with_eks/spotworkers/deployhandler.md b/content/using_ec2_spot_instances_with_eks/050_selfmanagednodegroups/deployhandler.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/spotworkers/deployhandler.md rename to content/using_ec2_spot_instances_with_eks/050_selfmanagednodegroups/deployhandler.md diff --git a/content/using_ec2_spot_instances_with_eks/spotworkers/preferspot.files/deployment-solution.yml b/content/using_ec2_spot_instances_with_eks/050_selfmanagednodegroups/preferspot.files/deployment-solution.yml similarity index 100% rename from content/using_ec2_spot_instances_with_eks/spotworkers/preferspot.files/deployment-solution.yml rename to content/using_ec2_spot_instances_with_eks/050_selfmanagednodegroups/preferspot.files/deployment-solution.yml diff --git a/content/using_ec2_spot_instances_with_eks/spotworkers/selecting_instance_types.md b/content/using_ec2_spot_instances_with_eks/050_selfmanagednodegroups/selecting_instance_types.md similarity index 77% rename from content/using_ec2_spot_instances_with_eks/spotworkers/selecting_instance_types.md rename to content/using_ec2_spot_instances_with_eks/050_selfmanagednodegroups/selecting_instance_types.md index 430c2da5..590efa62 100644 --- a/content/using_ec2_spot_instances_with_eks/spotworkers/selecting_instance_types.md +++ b/content/using_ec2_spot_instances_with_eks/050_selfmanagednodegroups/selecting_instance_types.md @@ -6,6 +6,6 @@ draft: false --- ### Selecting instances for Spot Instance Diversification -[See "Selecting Instance Types" under Spot managed node groups]({{< relref "/using_ec2_spot_instances_with_eks/spotmanagednodegroups/selecting_instance_types.md" >}}) +[See "Selecting Instance Types" under Spot managed node groups]({{< relref "/using_ec2_spot_instances_with_eks/040_spotmanagednodegroups/selecting_instance_types.md" >}}) Once you finish selecting instance types, proceed to "Adding Spot Workers with eksctl" section to continue creating Self Managed Spot node groups. diff --git a/content/using_ec2_spot_instances_with_eks/spotworkers/spotworkers.files/cloudformation_mixed_workers.yml b/content/using_ec2_spot_instances_with_eks/050_selfmanagednodegroups/spotworkers.files/cloudformation_mixed_workers.yml similarity index 100% rename from content/using_ec2_spot_instances_with_eks/spotworkers/spotworkers.files/cloudformation_mixed_workers.yml rename to content/using_ec2_spot_instances_with_eks/050_selfmanagednodegroups/spotworkers.files/cloudformation_mixed_workers.yml diff --git a/content/using_ec2_spot_instances_with_eks/spotworkers/spotworkers.files/eksctl_mixed_workers_bootstrap.yml b/content/using_ec2_spot_instances_with_eks/050_selfmanagednodegroups/spotworkers.files/eksctl_mixed_workers_bootstrap.yml similarity index 100% rename from content/using_ec2_spot_instances_with_eks/spotworkers/spotworkers.files/eksctl_mixed_workers_bootstrap.yml rename to content/using_ec2_spot_instances_with_eks/050_selfmanagednodegroups/spotworkers.files/eksctl_mixed_workers_bootstrap.yml diff --git a/content/using_ec2_spot_instances_with_eks/spotworkers/workers_eksctl.md b/content/using_ec2_spot_instances_with_eks/050_selfmanagednodegroups/workers_eksctl.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/spotworkers/workers_eksctl.md rename to content/using_ec2_spot_instances_with_eks/050_selfmanagednodegroups/workers_eksctl.md diff --git a/content/using_ec2_spot_instances_with_eks/deploy/_index.md b/content/using_ec2_spot_instances_with_eks/060_deploy/_index.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/deploy/_index.md rename to content/using_ec2_spot_instances_with_eks/060_deploy/_index.md diff --git a/content/using_ec2_spot_instances_with_eks/deploy/deploy.md b/content/using_ec2_spot_instances_with_eks/060_deploy/deploy.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/deploy/deploy.md rename to content/using_ec2_spot_instances_with_eks/060_deploy/deploy.md diff --git a/content/using_ec2_spot_instances_with_eks/deploy/monte_carlo_pi.md b/content/using_ec2_spot_instances_with_eks/060_deploy/monte_carlo_pi.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/deploy/monte_carlo_pi.md rename to content/using_ec2_spot_instances_with_eks/060_deploy/monte_carlo_pi.md diff --git a/content/using_ec2_spot_instances_with_eks/deploy/tolerations_and_affinity.files/monte-carlo-pi-service-final-self-managed.yml b/content/using_ec2_spot_instances_with_eks/060_deploy/tolerations_and_affinity.files/monte-carlo-pi-service-final-self-managed.yml similarity index 100% rename from content/using_ec2_spot_instances_with_eks/deploy/tolerations_and_affinity.files/monte-carlo-pi-service-final-self-managed.yml rename to content/using_ec2_spot_instances_with_eks/060_deploy/tolerations_and_affinity.files/monte-carlo-pi-service-final-self-managed.yml diff --git a/content/using_ec2_spot_instances_with_eks/deploy/tolerations_and_affinity.files/monte-carlo-pi-service-final.yml b/content/using_ec2_spot_instances_with_eks/060_deploy/tolerations_and_affinity.files/monte-carlo-pi-service-final.yml similarity index 100% rename from content/using_ec2_spot_instances_with_eks/deploy/tolerations_and_affinity.files/monte-carlo-pi-service-final.yml rename to content/using_ec2_spot_instances_with_eks/060_deploy/tolerations_and_affinity.files/monte-carlo-pi-service-final.yml diff --git a/content/using_ec2_spot_instances_with_eks/deploy/tolerations_and_affinity.md b/content/using_ec2_spot_instances_with_eks/060_deploy/tolerations_and_affinity.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/deploy/tolerations_and_affinity.md rename to content/using_ec2_spot_instances_with_eks/060_deploy/tolerations_and_affinity.md diff --git a/content/using_ec2_spot_instances_with_eks/scaling/_index.md b/content/using_ec2_spot_instances_with_eks/070_scaling/_index.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/scaling/_index.md rename to content/using_ec2_spot_instances_with_eks/070_scaling/_index.md diff --git a/content/using_ec2_spot_instances_with_eks/scaling/cleanup.md b/content/using_ec2_spot_instances_with_eks/070_scaling/cleanup.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/scaling/cleanup.md rename to content/using_ec2_spot_instances_with_eks/070_scaling/cleanup.md diff --git a/content/using_ec2_spot_instances_with_eks/scaling/deploy_ca.files/cluster_autoscaler.yml b/content/using_ec2_spot_instances_with_eks/070_scaling/deploy_ca.files/cluster_autoscaler.yml similarity index 100% rename from content/using_ec2_spot_instances_with_eks/scaling/deploy_ca.files/cluster_autoscaler.yml rename to content/using_ec2_spot_instances_with_eks/070_scaling/deploy_ca.files/cluster_autoscaler.yml diff --git a/content/using_ec2_spot_instances_with_eks/scaling/deploy_ca.md b/content/using_ec2_spot_instances_with_eks/070_scaling/deploy_ca.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/scaling/deploy_ca.md rename to content/using_ec2_spot_instances_with_eks/070_scaling/deploy_ca.md diff --git a/content/using_ec2_spot_instances_with_eks/scaling/deploy_hpa.md b/content/using_ec2_spot_instances_with_eks/070_scaling/deploy_hpa.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/scaling/deploy_hpa.md rename to content/using_ec2_spot_instances_with_eks/070_scaling/deploy_hpa.md diff --git a/content/using_ec2_spot_instances_with_eks/scaling/test_ca.md b/content/using_ec2_spot_instances_with_eks/070_scaling/test_ca.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/scaling/test_ca.md rename to content/using_ec2_spot_instances_with_eks/070_scaling/test_ca.md diff --git a/content/using_ec2_spot_instances_with_eks/scaling/test_hpa.md b/content/using_ec2_spot_instances_with_eks/070_scaling/test_hpa.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/scaling/test_hpa.md rename to content/using_ec2_spot_instances_with_eks/070_scaling/test_hpa.md diff --git a/content/using_ec2_spot_instances_with_eks/jenkins/_index.md b/content/using_ec2_spot_instances_with_eks/080_jenkins/_index.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/jenkins/_index.md rename to content/using_ec2_spot_instances_with_eks/080_jenkins/_index.md diff --git a/content/using_ec2_spot_instances_with_eks/jenkins/autoscaling_nodes.md b/content/using_ec2_spot_instances_with_eks/080_jenkins/autoscaling_nodes.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/jenkins/autoscaling_nodes.md rename to content/using_ec2_spot_instances_with_eks/080_jenkins/autoscaling_nodes.md diff --git a/content/using_ec2_spot_instances_with_eks/jenkins/increasing_resilience.md b/content/using_ec2_spot_instances_with_eks/080_jenkins/increasing_resilience.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/jenkins/increasing_resilience.md rename to content/using_ec2_spot_instances_with_eks/080_jenkins/increasing_resilience.md diff --git a/content/using_ec2_spot_instances_with_eks/jenkins/jenkins_cleanup.md b/content/using_ec2_spot_instances_with_eks/080_jenkins/jenkins_cleanup.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/jenkins/jenkins_cleanup.md rename to content/using_ec2_spot_instances_with_eks/080_jenkins/jenkins_cleanup.md diff --git a/content/using_ec2_spot_instances_with_eks/jenkins/running_jobs.md b/content/using_ec2_spot_instances_with_eks/080_jenkins/running_jobs.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/jenkins/running_jobs.md rename to content/using_ec2_spot_instances_with_eks/080_jenkins/running_jobs.md diff --git a/content/using_ec2_spot_instances_with_eks/jenkins/setup_agents.md b/content/using_ec2_spot_instances_with_eks/080_jenkins/setup_agents.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/jenkins/setup_agents.md rename to content/using_ec2_spot_instances_with_eks/080_jenkins/setup_agents.md diff --git a/content/using_ec2_spot_instances_with_eks/jenkins/setup_jenkins.md b/content/using_ec2_spot_instances_with_eks/080_jenkins/setup_jenkins.md similarity index 100% rename from content/using_ec2_spot_instances_with_eks/jenkins/setup_jenkins.md rename to content/using_ec2_spot_instances_with_eks/080_jenkins/setup_jenkins.md diff --git a/content/using_ec2_spot_instances_with_eks/090_emroneks/010_prereqs.md b/content/using_ec2_spot_instances_with_eks/090_emroneks/010_prereqs.md new file mode 100644 index 00000000..9266d537 --- /dev/null +++ b/content/using_ec2_spot_instances_with_eks/090_emroneks/010_prereqs.md @@ -0,0 +1,130 @@ +--- +title: "Prerequisites" +date: 2021-03-15T16:24:50-04:00 +weight: 10 +draft: false +--- + + +In this chapter, we will prepare your EKS cluster so that it is integrated with EMR on EKS. + +{{% notice note %}} +If you don't have EKS cluster with Spot Managed Node Groups then please review instructions from **[start the workshop]({{< relref "../010_prerequisites" >}})**, **[launch using eksctl]({{< relref "../020_eksctl" >}})**, **[create Spot managed node groups]({{< relref "../040_spotmanagednodegroups" >}})** and **[deploy cluster autoscaler]({{< relref "../070_scaling/deploy_ca.md" >}})** modules. +{{% /notice %}} + +### Create namespace and RBAC permissions + +Let's create a namespace '**spark**' in our EKS cluster. After this, we will use the automation powered by eksctl for creating RBAC permissions and for adding EMR on EKS service-linked role into aws-auth configmap + +``` +kubectl create namespace spark + +eksctl create iamidentitymapping --cluster eksworkshop-eksctl --namespace spark --service-name "emr-containers" +``` + +### Enable IAM Roles for Service Account (IRSA) + +Your cluster should already have OpenID Connect provider URL. Only configuration that is needed is to associate IAM with OIDC. You can do that by running this command + +``` +eksctl utils associate-iam-oidc-provider --cluster eksworkshop-eksctl --approve +``` + +### Create IAM Role for job execution + +Let's create the role that EMR will use for job execution. This is the role, EMR jobs will assume when they run on EKS. + +``` +cat < ~/environment/emr-trust-policy.json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "Service": "elasticmapreduce.amazonaws.com" + }, + "Action": "sts:AssumeRole" + } + ] +} +EoF + +aws iam create-role --role-name EMRContainers-JobExecutionRole --assume-role-policy-document file://~/environment/emr-trust-policy.json + +``` + +Next, we need to attach the required IAM policies to the role so it can write logs to s3 and cloudwatch. +``` +cat < ~/environment/EMRContainers-JobExecutionRole.json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "s3:PutObject", + "s3:GetObject", + "s3:ListBucket" + ], + "Resource": "*" + }, + { + "Effect": "Allow", + "Action": [ + "logs:PutLogEvents", + "logs:CreateLogStream", + "logs:DescribeLogGroups", + "logs:DescribeLogStreams" + ], + "Resource": [ + "arn:aws:logs:*:*:*" + ] + } + ] +} +EoF +aws iam put-role-policy --role-name EMRContainers-JobExecutionRole --policy-name EMR-Containers-Job-Execution --policy-document file://~/environment/EMRContainers-JobExecutionRole.json +``` + +### Update trust relationship for job execution role + +Now we need to update the trust relationship between IAM role we just created with EMR service identity. + +``` +aws emr-containers update-role-trust-policy --cluster-name eksworkshop-eksctl --namespace spark --role-name EMRContainers-JobExecutionRole +``` + +### Register EKS cluster with EMR + +The final step is to register EKS cluster with EMR. + +``` +aws emr-containers create-virtual-cluster \ +--name eksworkshop-eksctl \ +--container-provider '{ + "id": "eksworkshop-eksctl", + "type": "EKS", + "info": { + "eksInfo": { + "namespace": "spark" + } + } +}' +``` +After you register, you should get confirmation that your EMR virtual cluster is created. A virtual cluster is an EMR concept which means that EMR service is registered to Kubernetes namespace and it can run jobs in that namespace. + +```output + "id": "av6h2hk8fsyu12m5ru8zjg8ht", + "name": "eksworkshop-eksctl", + "arn": "arn:aws:emr-containers:xx-xxxx-x:xxxxxxxxxxxx:/virtualclusters/av6h2hk8fsyu12m5ru8zjg8ht" +``` + + +### Create S3 code bucket +Let's create a s3 bucket to upload sample scripts and logs. + +```sh +export s3DemoBucket=s3://emr-eks-demo-${ACCOUNT_ID}-${AWS_REGION} +aws s3 mb $s3DemoBucket +``` diff --git a/content/using_ec2_spot_instances_with_eks/090_emroneks/020_deploy.md b/content/using_ec2_spot_instances_with_eks/090_emroneks/020_deploy.md new file mode 100644 index 00000000..2cff078d --- /dev/null +++ b/content/using_ec2_spot_instances_with_eks/090_emroneks/020_deploy.md @@ -0,0 +1,116 @@ +--- +title: "Run Sample Workload" +date: 2021-05-11T13:38:18+08:00 +weight: 20 +draft: false +--- + +### Spark Pod Template +Beginning with Amazon EMR versions `5.33.0 or 6.3.0`, Amazon EMR on EKS supports Spark’s pod template feature. Pod templates are specifications that determine how to run each pod. You can use pod template files to define the driver or executor pod’s configurations that Spark configurations do not support. + +{{% notice info %}} +For more information about the pod templates support in EMR on EKS, see [Pod Templates](https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/pod-templates.html). +{{% /notice %}} + +To reduce costs, you can schedule Spark driver tasks to run on On-Demand instances while scheduling Spark executor tasks to run on Spot instances. + +With pod templates you can define label `eks.amazonaws.com/capacityType` as a [node selector](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/), so that you can schedule Spark driver pods on On-demand Instances and Spark executor pods on the Spot Instances. + +Now, you will create a sample pod template for Spark Driver. Using nodeSelector `eks.amazonaws.com/capacityType: ON_DEMAND` this will run on On-demand Instances. +```sh +cat > spark_driver_pod_template.yml < spark_executor_pod_template.yml <