diff --git a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/_index.md b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/_index.md index 5b5fbf5c..9fd912c6 100644 --- a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/_index.md +++ b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/_index.md @@ -13,13 +13,22 @@ You can perform all workshop steps one-by-one to get to the expected results, bu Many workshop steps imply manual actions in the AWS console to better demonstrate the underlying concepts, but in a Production environment it is better to automate them using Infrastructure as Code (IaC), such as [AWS CloudFormation](https://aws.amazon.com/cloudformation/) and [AWS CDK](https://docs.aws.amazon.com/cdk/v2/guide/home.html). ## Workshop Details -This workshop will be broken down into a series of labs that flow on from each other (that is, you must complete each lab in order before proceeding with the next). The lab exercises that will be covered are: +This workshop will be broken down into a series of sections that flow on from each other (that is, you must complete each section before proceeding with the next). The whole flow looks as following: + +![GitLab on Spot workshop flow](/images/gitlab-spot/lab-flow.png) + + +The exercises that will be covered are: * [Starting the workshop](gitlab-spot/before.html) where you will log in to AWS accounts and deploy GitLab, if it is not yet deployed * [Workshop Preparation](gitlab-spot/prep.html) where you will save GitLab access details and create an AWS Cloud9 environment to execute the workshop steps -* [Lab 1: Create a GitLab repository](gitlab-spot/lab1.html) where you will create a repository in GitLab CI/CD and create a demo application. -* [Lab 2: Configure GitLab runners on Spot instances](gitlab-spot/lab2.html) where you will deploy the GitLab Runners in an auto-scaling group on spot instances -* [Lab 3: Building the demo app](gitlab-spot/lab3.html) where you will push the changes and make sure that your pipeline executes successfully. -* [Lab 4: Deploying Amazon EKS on Spot instances](gitlab-spot/lab4.html) where you will create a new Kubernetes cluster in Amazon EKS that will only have worker nodes on spot instances -* [Lab 5: Installing the demo app into Amazon EKS](gitlab-spot/lab5.html) where you will modify your GitLab CI/CD scripts to add a stage of deploying on Amazon EKS and test the result -* [Workshop Cleanup](gitlab-spot/cleanup.html) where you will remove all the resources created during the workshop \ No newline at end of file +* [Create a GitLab repository](gitlab-spot/lab1.html) where you will create a repository in GitLab CI/CD and create a demo application. +* [Configure GitLab runners on Spot instances](gitlab-spot/lab2.html) where you will deploy the GitLab Runners in an auto-scaling group on spot instances +* [Building the demo app](gitlab-spot/lab3.html) where you will push the changes and make sure that your pipeline executes successfully. +* [Deploying Amazon EKS on Spot instances](gitlab-spot/lab4.html) where you will create a new Kubernetes cluster in Amazon EKS that will only have worker nodes on spot instances +* [Installing the demo app into Amazon EKS](gitlab-spot/lab5.html) where you will modify your GitLab CI/CD scripts to add a stage of deploying on Amazon EKS and test the result +* [Workshop Cleanup](gitlab-spot/cleanup.html) where you will remove all the resources created during the workshop + +The final architecture that we will be building looks the following way: + +![GitLab on Spot workshop architecture diagram](/images/gitlab-spot/gitlab-spot-architecture.png) \ No newline at end of file diff --git a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/before/aws_event.md b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/before/aws_event.md index 3d8205f2..d870bfc6 100644 --- a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/before/aws_event.md +++ b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/before/aws_event.md @@ -20,7 +20,7 @@ If you are at an AWS event, an AWS account created for you to use throughout the ### Get the SSH key and log in to AWS Console 1. In the Team Dashboard screen, choose **SSH Key**. -2. In the popup page, choose **Download Key**. You will only need it if you decide to do an optional task of configuring runners using Docker Machine (this had been the only way before Auto-Scaling Groups support was added). +2. In the popup page, choose **Download Key**. You don't need the key to complete the labs, but might want to still have it if you decide to explore the environment. ![Event Engine Screenshot: SSH Key](/images/gitlab-spot/EE-SSHKey.png) @@ -32,7 +32,7 @@ You are now logged in to the AWS console in an account that was created for you, ### Open the pre-provisioned CloudFormation stack -In the next section we will get the login details of the GitLab environment that was pre-provisioned for you via AWS CloudFormation. First, you need to find and open the CloudFormation stack: it is the only one in the account. +In the next section we will get the login details of the GitLab environment that was pre-provisioned for you via AWS CloudFormation. First, you need to find and open the CloudFormation stack: it is the oldest one in the account. {{%expand "Click to reveal detailed instructions" %}} 1. In the AWS Console enter **CloudFormation** in the search box at the top of the screen and open the service: @@ -40,7 +40,7 @@ In the next section we will get the login details of the GitLab environment that ![AWS Console Screenshot: Search for CloudFormation](/images/gitlab-spot/AWSConsole-CloudFormationSearch.png) 2. In the navigation pane on the left choose **Stacks**. -3. You should see exactly one stack in the list (with the name like `mod-...`), click on it. +3. You should see two stacks in the list: the one for AWS Cloud9 environment starting with `aws-cloud9-...` and the main one starting with `mod-...`. You will need the main stack (latter one), click on it. {{% /expand%}} If there is no CloudFormation stack present, provision it as specified in the section [**...On your own**](self_paced.html). diff --git a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/before/self_paced.md b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/before/self_paced.md index 97238c7a..0cc6e41d 100644 --- a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/before/self_paced.md +++ b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/before/self_paced.md @@ -19,7 +19,7 @@ Your account must have the ability to create new IAM roles and scope other IAM p ### Create an SSH key -You will need an SSH key if you decide to do an optional task of configuring runners using Docker Machine (this had been the only way before Auto-Scaling Groups support was added). The following steps show how to create it. +You don't need the key to complete the labs, but it is still configured when creating the instances, because you might want to explore the environment and log in to the provisioned instances. The following steps show how to create it. 1. Open **EC2** service in the AWS Console. 2. In the navigation pane choose **Key Pairs** in the **Network & Security** section. @@ -37,7 +37,7 @@ Now you will deploy a GitLab instance without any runners. As it is not the purp 2. In the navigation pane choose **Stacks**. 3. Choose **Create stack** and in the dropdown choose **With new resources (standard)**. 4. In the **Template source** field select **Upload a template file**, choose the file you saved in the [**Preparation**](#preparation) section above, and choose **Next**. -5. In the **Stack name** field enter `mod-gitlab-spot-workshop`, in the **SSHKeyName** field select `ee-default-key-pair` or the name of the key you used in the steps above, and choose **Next**. +5. In the **Stack name** field enter `mod-gitlab-spot-workshop`, in the **EEKeyPair** field select `ee-default-key-pair` or the name of the key you used in the steps above. Leave the default values in other fields and choose **Next**. 6. Choose **Next**. 7. Mark the checkbox **I acknowledge that AWS CloudFormation might create IAM resources.** and choose **Create stack**. 8. Wait until the stack is in `CREATE_COMPLETE` status (it should take approximately 15 minutes) and continue with [**Workshop Preparation**](/amazon-ec2-spot-cicd-workshop/gitlab-spot/prep.html). \ No newline at end of file diff --git a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/cleanup.md b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/cleanup.md index 622871a8..a844b8c0 100644 --- a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/cleanup.md +++ b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/cleanup.md @@ -23,13 +23,11 @@ export TF_VAR_alb_policy=$(aws iam get-policy --policy-arn arn:aws:iam::$(aws st terraform destroy ``` -3. When asked if you want to destroy all resources, type `yes` and press Enter. After that, you can close your Cloud9 environment. -4. Depending on how you deployed the runners, do one of the following: - - If you deployed runners using Auto-Scaling Group, in CloudFormation console delete the stack `linux-docker-scaling-spotonly` (the NESTED stack will be removed automatically). - - If you performed the optional steps to deploy runners using Docker Machine, terminate all runner instances in the EC2 console (name starts with `runner-`) if they have not been yet terminated automatically. Also, terminate the instace `GitLabRunnerManager`. Finally, in the EC2 console choose **Security Groups** in the navigation pane and delete the security groups `GitLabRunner` first and then `GitLabRunnerManager`. -5. In Cloud9 console remove the environment you created. -6. In the IAM console remove all roles you created (`gitlab-spot-workshop-admin` and `GitLabRunner`) and all policies (`EKS-ReadAll`, `AWSLoadBalancerControllerIAMPolicy`, unless you had it before the workshop, and, if you did the Docker Run lab, `IAM-PassRole`). -7. In the ECR console open the repository and remove all images inside it (you do not need to remove the repository itself: it will be done automatically when removing the GitLab stack). +3. When asked if you want to destroy all resources, type `yes` and press Enter. Wait until the process is finished and close your Cloud9 environment. +4. In CloudFormation console delete the stack `linux-docker-scaling-spotonly`. +5. In the IAM console remove the role `GitLabRunner` and the policies `GitLabRunnerPolicy` and `AWSLoadBalancerControllerIAMPolicy`, unless you had it before the workshop. +6. In the ECR console open the repository and remove all images inside it (you do not need to remove the repository itself: it will be done automatically when removing the GitLab stack). +7. In the S3 console find a bucket with `gitlabworkshopc9outputbucket` in its name and remove all objects inside it. 8. If you created the GitLab stack in CloudFormation yourself, remove it too (if you used the one created automatically, you will not be able to delete it, so you can leave it as is). ### Thank you diff --git a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab1.md b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab1.md index 9a58d97f..d11c3fde 100644 --- a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab1.md +++ b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab1.md @@ -1,5 +1,5 @@ +++ -title = "Lab 1: Create a GitLab repository" +title = "Create a GitLab repository" weight = 30 +++ In this lab you will create a new repository in GitLab and configure it in AWS Cloud9. Then you will create the source code of the demo app and commit it into the repository, but before you configure the actual CI/CD in the next lab, will not push it. @@ -34,16 +34,18 @@ You will now initialize Git repository in the demo app and add your GitLab envir cd ~/environment/amazon-ec2-spot-cicd-workshop/gitlab-spot/demo-app/ ``` -2. Execute the following command to substitute the `ECR_ADDRESS` placeholder with the actual address of ECR repository created in the CloudFormation stack and save it into `.gitlab-ci.yml` file as expected by GitLab CI/CD: +2. Execute the following command to substitute the `ECR_ADDRESS` and other placeholders with their actual values (like the address of ECR repository created in the CloudFormation stack) and save it into `.gitlab-ci.yml` file as expected by GitLab CI/CD: ``` TOKEN=$(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 600") REGION=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/placement/region) -export ECR_ADDRESS=$(aws ecr describe-repositories --repository-names gitlab-spot-demo --region $REGION --query repositories[0].repositoryUri --output text | awk -F'/' '{print $1}') -sed "s/\${ECR_ADDRESS}/${ECR_ADDRESS}/g" template-gitlab-ci.yml | sed "s/\${AWS_REGION}/${REGION}/g" > .gitlab-ci.yml +export ECR_INFO=$(aws ecr describe-repositories --region $REGION --query "repositories[?contains(repositoryName, 'gitlab-spot-demo')].repositoryUri" --output text) +export ECR_ADDRESS=$(echo $ECR_INFO | awk -F'/' '{print $1}') +export ECR_NAME=$(echo $ECR_INFO | awk -F'/' '{print $2}') +sed "s/\${ECR_ADDRESS}/${ECR_ADDRESS}/g" template-gitlab-ci.yml | sed "s/\${ECR_NAME}/${ECR_NAME}/g" | sed "s/\${AWS_REGION}/${REGION}/g" > .gitlab-ci.yml ``` -3. In the file tree on the left open file `gitlab-spot-workshop/amazon-ec2-spot-cicd-workshop/gitlab-spot/demo-app/.gitlab-ci.yml` (if you don't see it, make sure you have enabled the hidden files in [**Workshop Preparation**](prep.html)). Look through it to understand what it does. +3. In the file tree on the left open file `amazon-ec2-spot-cicd-workshop/gitlab-spot/demo-app/.gitlab-ci.yml` (if you don't see it, make sure you have enabled the hidden files in [**Workshop Preparation**](prep.html)). Look through it to understand what it does. 4. Define your name and email that will be used in Git (replace `Your Name` and `youremail@test.tld` with the values you prefer): ``` @@ -66,4 +68,4 @@ git add . git commit -m "Initial commit" ``` -You are now ready to do the key step in configuring GitLab CI/CD on Spot instances: add the runners. Please proceed to [**Lab 2: Configure GitLab runners on Spot instances**](lab2.html). \ No newline at end of file +You are now ready to do the key step in configuring GitLab CI/CD on Spot instances: add the runners. Please proceed to [**Configure GitLab runners on Spot instances**](lab2.html). \ No newline at end of file diff --git a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2.md b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2.md new file mode 100644 index 00000000..d43884ca --- /dev/null +++ b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2.md @@ -0,0 +1,187 @@ ++++ +title = "Configure GitLab runners on Spot instances" +chapter = false +weight = 40 ++++ + +In this lab you will configure GitLab CI/CD runners using **GitLab HA Scaling Runner Vending Machine for AWS** solution. You can find more about its features at https://gitlab.com/guided-explorations/aws/gitlab-runner-autoscaling-aws-asg/-/blob/main/FEATURES.md. It is built using Infrastructure as Code (IaC) with AWS CloudFormation, but you can implement similar logic using any IaC solution of your choice. + +There are also other ways to create GitLab runners on spot instances that we are not reviewing in this workshop: using Docker Machine, runners inside containers in a Kubernetes cluster with the worker nodes on spot instances (we will deploy such cluster for application testing, but not for executing the runners) or in Amazon ECS with Fargate Spot. + +### Create an IAM role for GitLab Runners + +You will now create a new IAM role that has access to Amazon EKS, Amazon S3, AWS Systems Manager, Amazon EC2 Auto Scaling, and Amazon ECR, which the GitLab Runners will need to build and deploy the demo app. + +1. In the AWS Console enter **IAM** in the search box at the top of the screen and open the service. +2. Choose **Policies** in the navigation pane. +3. Choose **Create Policy**. +4. Switch to **JSON** tab and paste the below policy into the editor window: + +``` +{ + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "EKSGetAllClusters", + "Effect": "Allow", + "Action": [ + "eks:DescribeNodegroup", + "eks:ListNodegroups", + "eks:DescribeCluster", + "eks:ListClusters", + "eks:AccessKubernetesApi", + "ssm:GetParameter", + "eks:ListUpdates", + "eks:ListFargateProfiles" + ], + "Resource": "*" + }, + { + "Sid": "ASGSelfAccess", + "Effect": "Allow", + "Action": [ + "iam:ListAccountAliases", + "autoscaling:DescribeAutoScalingInstances", + "autoscaling:DescribeAutoScalingGroups", + "autoscaling:DescribeLifecycle*" + ], + "Resource": "*" + }, + { + "Sid": "ASGLifeCycleAccess", + "Effect": "Allow", + "Action": [ + "autoscaling:CompleteLifecycleAction", + "autoscaling:RecordLifecycleActionHeartbeat" + ], + "Resource": "arn:aws:autoscaling:*:*:autoScalingGroup:*:autoScalingGroupName/linux-docker-scaling-spotonly*" + }, + { + "Sid": "AllowRunnerJobsToDoPredictiveScaling", + "Effect": "Allow", + "Action": [ + "autoscaling:UpdateAutoScalingGroup" + ], + "Resource": "arn:aws:autoscaling:*:*:autoScalingGroup:*:autoScalingGroupName/linux-docker-scaling-spotonly*" + }, + { + "Sid": "EC2SelfAccess", + "Effect": "Allow", + "Action": [ + "ec2:DescribeInstances", + "ec2:DescribeTags" + ], + "Resource": "*" + }, + { + "Sid": "S3CacheBucketAccess", + "Effect": "Allow", + "Action": [ + "s3:GetObject", + "s3:GetObjectVersion", + "s3:GetBucketVersioning", + "s3:PutObject", + "s3:DeleteObject" + ], + "Resource": "arn:aws:s3:::*linux-docker-scaling-spotonly*" + }, + { + "Sid": "ECRAccess", + "Effect": "Allow", + "Action": "ecr:GetAuthorizationToken", + "Resource": "*" + }, + { + "Sid": "ECRAccessRepo", + "Effect": "Allow", + "Action": [ + "ecr:BatchGetImage", + "ecr:BatchCheckLayerAvailability", + "ecr:CompleteLayerUpload", + "ecr:GetDownloadUrlForLayer", + "ecr:InitiateLayerUpload", + "ecr:PutImage", + "ecr:UploadLayerPart" + ], + "Resource": "arn:aws:ecr:*:*:*gitlab-spot-demo*" + } + ] +} +``` + +5. Choose **Next: Tags**. +6. Choose **Next: Review**. +7. In the **Name** field type `GitLabRunnerPolicy` and choose **Create policy**: + +![IAM Console Screenshot: Create GitLabRunnerPolicy policy](/images/gitlab-spot/AWSConsole-IAMCreatePolicy.png) + +8. In the navigation pane choose **Roles** +9. Choose **Create role**. +10. In the **Use case** section select **EC2** and choose **Next**. +11. In the **Permissions policies** table find the following policies one-by-one and select each of them: + - `GitLabRunnerPolicy` + - `AmazonSSMManagedInstanceCore` + - `CloudWatchAgentServerPolicy` + - `AmazonSSMAutomationRole` + - `AmazonSSMMaintenanceWindowRole` + +{{% notice warning %}} +Note that here you are assigning permission policies as required for the demo app. In a real Production environment you should always follow the least privilege principle. You can find the best practices in [AWS Blogs](https://aws.amazon.com/blogs/security/techniques-for-writing-least-privilege-iam-policies/). +{{% /notice %}} + +12. Make sure you have 5 policies selected and choose **Next**. +13. In the **Role name** field type `GitLabRunner` and choose **Create role**. + +### Deploy the CloudFormation stack + +You will now get the runner configuration information from GitLab and then start the AWS CloudFormation stack deployment. + +1. Return to the browser tab with GitLab. +2. In the **GitLab Spot Workshop** project choose **Settings** > **CI/CD** in the navigation pane. +3. Expand the **Runners** section and save into a text file both GitLab URL and the registration token that are displayed on the screen: + +![GitLab Screenshot: Runners configuration](/images/gitlab-spot/GitLab-RunnersRegistration.png) + +4. Download the CloudFormation template latest version located at https://gitlab.com/guided-explorations/aws/gitlab-runner-autoscaling-aws-asg/-/blob/main/GitLabElasticScalingRunner.cf.yml. + +5. Return to the browser tab with CloudFormation or open it again using the search box at the top. +6. Choose **Create stack** and in the dropdown choose **With new resources (standard)**. +7. In the **Template source** field select **Upload a template file**, choose the `GitLabElasticScalingRunner.cf.yml` file you have just downloaded above, and choose **Next**. +8. In the **Stack name** field enter `linux-docker-scaling-spotonly` (**extremely important**, as this name is used in the IAM policy above). + +Investigate the parameters presented and try deploying the stack yourself. Expand the section below to see step-by-step instructions. + +{{%expand "Click to reveal detailed instructions" %}} +9. In the **GitLab Instance URL** field enter GitLab URL you saved previously (in the format `https://xxx.cloudfront.net`). +10. In the **One or more runner Registration tokens from the target instance.** field enter the token you saved previously. +11. In the **The S3 bucket that will be used for a shared runner cache.** leave the default value (do not enter the GitLabCacheBucket value you saved from CloudFormation Output values, as at the moment the template will still try to create policies using the custom bucket and will fail if a different one is provided). +12. In the **The number of instances that should be configured. Generally 1 for warm HA and 2 for hot HA.** enter `2`. +13. In the **Override automatic IAM Instance Profile for a precreated one.** type `GitLabRunner`. +14. In the **The VPC in the account and region should be used.** enter the VPC ID you saved from CloudFormation Output values in [**Workshop Preparation**](/amazon-ec2-spot-cicd-workshop/gitlab-spot/prep.html) (in the format `vpc-...`). +15. In the field **Second priority instance type to be used - if previous priorities are exhausted. Used for Ec2 Fleet when not using spot.** enter `m5d.large`. For the third and fourth priority enter `m5n.large` and `c5.large` respectively. +16. In the field **Whether to AutoScale the Scaling Group.** select `true` in the dropdown. +17. In the field **Maximum instances in the Scaling Group.** type `10`. +18. In the field **Scale in threshold (percent).** type `40`. +19. In the field **How long the scale in threshold should be experienced before triggering scale out.** type `60`. +20. In the field **Scale out threshold (percent).** type `70`. +21. In the field **How long the scale out threshold should be experienced before triggering scale out.** type `60`. +22. In the field **Percentage of non-spot instances when scaling above OnDemandBaseCapacity.** type `0`. +23. Leave other parameters with their default values and choose **Next**, then once again **Next**. +24. In the bottom of the screen mark the checkbox **I acknowledge that AWS CloudFormation might create IAM resources with custom names.**, and choose **Create stack**: +![CloudFormation Console Screenshot: Create GitLab Runners stack](/images/gitlab-spot/AWSConsole-CloudFormationGitLabRunnersStack.png) +24. Wait until the stack is in `CREATE_COMPLETE` status, which should take approximately 5 minutes. +{{% /expand%}} + +{{% notice tip %}} +You had to use the full template to customize the IAM role used by the runners. If your production scenario does not require this and you are fine with the default permissions, you can use one of the easy buttons (out-of-the-box scenarios) provided at https://gitlab.com/guided-explorations/aws/gitlab-runner-autoscaling-aws-asg#easy-buttons-provided to deploy the stack: this will require less parameters to customize. +{{% /notice %}} + +Now you can open the EC2 console and verify that your runner(s) are using spot instances: open the corresponding EC2 instance (its name starts with `linux-docker-scaling-spotonly-`) and check the **Lifecycle** field: + +![EC2 Console Screenshot: Runner lifecycle](/images/gitlab-spot/AWSConsole-EC2RunnerLifecycle.png) + +25. Return to the browser tab with GitLab, refresh the CI/CD settings page and make sure that your runner(s) have appeared in the **Runners** section: + +![GitLab Screenshot: Runner available](/images/gitlab-spot/GitLab-RunnerAvailable.png) + +You can now proceed to build your application in [**Building the demo app**](/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab3.html). \ No newline at end of file diff --git a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/_index.md b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/_index.md deleted file mode 100644 index 296dbcf6..00000000 --- a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/_index.md +++ /dev/null @@ -1,128 +0,0 @@ -+++ -title = "Lab 2: Configure GitLab runners on Spot instances" -chapter = false -weight = 40 -+++ - -In this lab you will configure GitLab CI/CD runners using **GitLab HA Scaling Runner Vending Machine for AWS** solution. You can find more about its features at https://gitlab.com/guided-explorations/aws/gitlab-runner-autoscaling-aws-asg/-/blob/main/FEATURES.md. It is built using Infrastructure as Code (IaC) with AWS CloudFormation, but you can implement similar logic using any IaC solution of your choice. - -If you want to use the previous way of manually configuring runners using Docker Machine (which does not use Auto-Scaling Groups and thus has some drawbacks compared to the current approach), see the steps in the [**Configuring GitLab runners using Docker Machine (optional)**](lab2/docker-machine.html) section instead. - -There are also other ways to create GitLab runners on spot instances that we are not reviewing in this workshop: runners inside containers in a Kubernetes cluster with the worker nodes on spot instances (we will deploy such cluster for application testing, but not for executing the runners) or in Amazon ECS with Fargate Spot. - -### Deploy the CloudFormation stack - -You will now get the runner configuration information from GitLab and then start the AWS CloudFormation stack deployment. - -1. Return to the browser tab with GitLab. -2. In the **GitLab Spot Workshop** repository choose **Settings** > **CI/CD** in the navigation pane. -3. Expand the **Runners** section and save into a text file both GitLab URL and the registration token that are displayed on the screen: - -![GitLab Screenshot: Runners configuration](/images/gitlab-spot/GitLab-RunnersRegistration.png) - -4. Use one of the links at https://gitlab.com/guided-explorations/aws/gitlab-runner-autoscaling-aws-asg#easy-buttons-provided to deploy the stack (make sure to change the link to the correct region). We will be using the **Amazon Linux 2 Docker Simple Scaling Spot Instances**, you can use one of the below links to deploy it depending on the region where you are running the workshop: - -{{< tabs name="Region" >}} - {{< tab name="N. Virginia" include="gitlab-easybutton/us-east-1.md" />}} - {{< tab name="Oregon" include="gitlab-easybutton/us-west-2.md" />}} - {{< tab name="Frankfurt" include="gitlab-easybutton/eu-central-1.md" />}} - {{< tab name="Ireland" include="gitlab-easybutton/eu-west-1.md" />}} - {{< tab name="Ohio" include="gitlab-easybutton/us-east-2.md" />}} - {{< tab name="Singapore" include="gitlab-easybutton/ap-southeast-1.md" />}} -{{< /tabs >}} - -Investigate the parameters presented and try deploying the stack yourself. Expand the section below to see step-by-step instructions. - -{{%expand "Click to reveal detailed instructions" %}} -5. In the **GitLab Instance URL** field enter GitLab URL you saved previously (in the format `https://xxx.cloudfront.net`). -6. In the **One or more runner Registration tokens from the target instance.** field enter the token you saved previously. -7. In the **The VPC in the account and region should be used.** enter the VPC ID you saved from CloudFormation Output values in [**Workshop Preparation**](/amazon-ec2-spot-cicd-workshop/gitlab-spot/prep.html) (in the format `vpc-...`). -8. Leave the rest parameters with their default values, in the bottom of the screen mark the checkboxes **I acknowledge that AWS CloudFormation might create IAM resources with custom names.** and **I acknowledge that AWS CloudFormation might require the following capability: CAPABILITY_AUTO_EXPAND**, and choose **Create stack**: - -![CloudFormation Console Screenshot: Create GitLab Runners stack](/images/gitlab-spot/AWSConsole-CloudFormationGitLabRunnersStack.png) -9. Wait until the stack is in `CREATE_COMPLETE` status, which should take approximately 5-10 minutes. -{{% /expand%}} - -### Enable access to AWS services for GitLab Runners - -You will now create a new IAM role that has access to Amazon EKS, Amazon S3, AWS Systems Manager, Amazon EC2 Auto Scaling, and Amazon ECR, which the GitLab Runners will need to build and deploy the demo app, and then update the CloudFormation stack to use it. - -1. Return to the browser tab with IAM console or open it again using the search box at the top. -2. Choose **Policies** in the navigation pane. -3. Choose **Create Policy**. -4. Switch to **JSON** tab and paste the below policy into the editor window: - -``` -{ - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Action": [ - "eks:DescribeNodegroup", - "eks:ListNodegroups", - "eks:DescribeCluster", - "eks:ListClusters", - "eks:AccessKubernetesApi", - "ssm:GetParameter", - "eks:ListUpdates", - "eks:ListFargateProfiles" - ], - "Resource": "*" - } - ] -} -``` - -5. Choose **Next: Tags**. -6. Choose **Next: Review**. -7. In the **Name** field type `EKS-ReadAll` and choose **Create policy**: - -![IAM Console Screenshot: Create EKS-ReadAll policy](/images/gitlab-spot/AWSConsole-IAMCreatePolicy.png) - -8. In the navigation pane choose **Roles** -9. Choose **Create role**. -10. In the **Use case** section select **EC2** and choose **Next**. -11. In the **Permissions policies** table find the following policies one-by-one and select each of them: - - `EKS-ReadAll` - - `AmazonS3FullAccess` - - `AmazonEC2RoleforSSM` - - `CloudWatchAgentServerPolicy` - - `AmazonSSMAutomationRole` - - `AmazonSSMMaintenanceWindowRole` - - `AmazonEC2ContainerRegistryFullAccess` - - A policy created by CloudFormation stack containing `EC2SelfAccessPolicy` in its name - - A policy created by CloudFormation stack containing `ASGSelfAccessPolicy` in its name - -{{% notice warning %}} -Note that here you are assigning quite broad permission policies to the IAM role to avoid creating specific custom policies, but in a real Production environment you should follow the least privilege principle. You can find the best practices in [AWS Blogs](https://aws.amazon.com/blogs/security/techniques-for-writing-least-privilege-iam-policies/). -{{% /notice %}} - -12. Make sure you have 9 policies selected and choose **Next**. -13. In the **Role name** field type **GitLabRunner** and choose **Create role**. -14. Return to CloudFormation stacks and open the stack starting with `linux-docker-scaling-spotonly-` and marked as **NESTED**: - -![CloudFormation Console Screenshot: GitLab nested stack](/images/gitlab-spot/AWSConsole-CloudFormationNestedStack.png) - -15. Select **Update**. -16. In the popup page select **Update nested stack** and then choose **Update stack**. -17. Choose **Next** (leave the default option of **Use current template** unchanged). -18. In the **Override automatic IAM Instance Profile for a precreated one** field type `GitLabRunner`. -19. In the **4ASGUpdateMinInstancesInService** field type `0`, this would allow to update the group faster by leaving no active runners in place. It should not be used in a real environment if you have CI/CD jobs constantly coming in, as there might be no runners to take them. -20. Choose **Next** and then again **Next**. -21. On the final screen select the **I acknowledge that AWS CloudFormation might create IAM resources with custom names.** checkbox and choose **Update stack**. -22. Wait until the stack is in the `UPDATE_COMPLETE` status which should take approximately 5 minutes. - -After that you can open the EC2 console and verify that your runner(s) are using spot instances: open the corresponding EC2 instance (its name starts with `linux-docker-scaling-spotonly-`) and check the **Lifecycle** field: - -![EC2 Console Screenshot: Runner lifecycle](/images/gitlab-spot/AWSConsole-EC2RunnerLifecycle.png) - -23. Return to the browser tab with GitLab, refresh the CI/CD settings page and make sure that your runner(s) have appeared in the **Runners** section: - -![GitLab Screenshot: Runner available](/images/gitlab-spot/GitLab-RunnerAvailable.png) - -{{% notice tip %}} -You had to do an extra step to update the IAM role of the runners, because this parameter is not available in the easy buttons templates. However, you could instead use the full template right away and provide all the required parameters directly. You can find more information at https://gitlab.com/guided-explorations/aws/gitlab-runner-autoscaling-aws-asg. -{{% /notice %}} - -You can now proceed to build your application in [**Lab 3: Building the demo app**](/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab3.html). \ No newline at end of file diff --git a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/docker-machine.md b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/docker-machine.md deleted file mode 100644 index 5c290eb6..00000000 --- a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/docker-machine.md +++ /dev/null @@ -1,235 +0,0 @@ -+++ -title = "Configuring GitLab runners using Docker Machine (optional)" -weight = 10 -+++ - -{{% notice info %}} -Only complete this lab if you want to test the legacy way of deploying spot instances in GitLab using Docker Machine and if you have not performed the lab with runners in an auto-scaling group. Otherwise, skip it and proceed to [**Lab 3: Building the demo app**](/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab3.html). -{{% /notice %}} - -### Create an IAM role for GitLab Runners - -You will now create a new IAM role that has access to Amazon EKS, Amazon S3, Amazon EC2, and Amazon ECR, which the GitLab Runners will need to build and deploy the demo app. - -1. Return to the browser tab with IAM console or open it again using the search box at the top. -2. Choose **Policies** in the navigation pane. -3. Choose **Create Policy**. -4. Switch to **JSON** tab and paste the below policy into the editor window: - -``` -{ - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Action": [ - "eks:DescribeNodegroup", - "eks:ListNodegroups", - "eks:DescribeCluster", - "eks:ListClusters", - "eks:AccessKubernetesApi", - "ssm:GetParameter", - "eks:ListUpdates", - "eks:ListFargateProfiles" - ], - "Resource": "*" - } - ] -} -``` - -5. Choose **Next: Tags**. -6. Choose **Next: Review**. -7. In the **Name** field type `EKS-ReadAll` and choose **Create policy**: - -![IAM Console Screenshot: Create EKS-ReadAll policy](/images/gitlab-spot/AWSConsole-IAMCreatePolicy.png) - -8. Repeat the same steps to create another policy with the following JSON and the name `IAM-PassRole`: - -``` -{ - "Version": "2012-10-17", - "Statement": [ - { - "Action": [ - "iam:PassRole" - ], - "Effect": "Allow", - "Resource": "arn:aws:iam::*:role/GitLabRunner*" - } - ] -} -``` - -9. In the navigation pane choose **Roles** -10. Choose **Create role**. -11. In the **Use case** section select **EC2** and choose **Next**. -12. In the **Permissions policies** table find the following policies one-by-one and select each of them: - - `EKS-ReadAll` - - `IAM-PassRole` - - `AmazonS3FullAccess` - - `AmazonEC2FullAccess` - - `AmazonEC2ContainerRegistryFullAccess` - -{{% notice warning %}} -Note that here you are assigning quite broad permission policies to the IAM role to avoid creating specific custom policies, but in a real Production environment you should follow the least privilege principle. You can find the best practices in [AWS Blogs](https://aws.amazon.com/blogs/security/techniques-for-writing-least-privilege-iam-policies/). -{{% /notice %}} - -13. Make sure you have 5 policies selected and choose **Next**. -14. In the **Role name** field type **GitLabRunner** and choose **Create role**. - -### Deploy GitLab Runner Manager - -You will now configure a new GitLab Runner that will serve as a Runner Manager and using Docker Machine will create new EC2 Spot instances: the actual CI/CD jobs will be executed on them. You can use x86_64 architecture, but to additionally optimize the costs, we suggest that you run the instance on [AWS Graviton](https://aws.amazon.com/ec2/graviton/) processor (as there are no CI/CD jobs running on it, the architecture is not important). - -1. In the browser tab with AWS Console, type `EC2` in the search box at the top and open the service. -2. Choose **Instances** in the navigation pane. -3. Choose **Launch instances**. -4. In the field **Name** type `GitLabRunnerManager`. -5. In the **Architecture** dropdown of the **Application and OS Images (Amazon Machine Image)** pane select `64-bit (Arm)`. Leave the Operating System as **Amazon Linux**. -6. In the **Instance type** dropdown select `t4g.nano`. -7. In the **Key pair name** dropdown select `ee-default-keypair` or any other key you created previously if you run this lab in your own account. -8. In the **Network settings** pane choose **Edit** and then: - - In the **VPC** field select the VPC ID you saved from CloudFormation Output values in [**Workshop Preparation**](/amazon-ec2-spot-cicd-workshop/gitlab-spot/prep.html), it contains `GitLabSpotWorkshop VPC` in its name. - - In the **Security group name** field type `GitLabRunnerManager` -9. Expand the **Advanced details** and in the **IAM instance profile** dropdown select `GitLabRunner`. - -{{% notice warning %}} -In Production configuration it is better to separate the IAM role for the Runner Manager and the actual Runner to follow the least privilege principle. -{{% /notice %}} - -10. Leave the rest settings as default and choose **Launch instance**. - -![EC2 Console Screenshot: Launch an instnace](/images/gitlab-spot/AWSConsole-EC2LaunchInstance.png) - -### Configure GitLab Runner and Docker Machine -You will now configure the instance to serve as a Runner Manager by installing all the necessary tools into it. - -1. Connect to the instance by SSH: you can select it in the EC2 console and then choose **Connect**, select the **EC2 Instance Connect** tab and choose **Connect**. Alternatively, you can return to the browser tab with Cloud9 environment and connect to it using the command below, substituting the `` to the private IP address of the instance: - -``` -ssh -i ~/environment/ee-default-keypair.pem ec2-user@ -``` - -2. After connecting, install the GitLab Runner using the following commands: - -``` -curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.rpm.sh" | sudo bash -export GITLAB_RUNNER_DISABLE_SKEL=true; sudo -E yum install gitlab-runner -y -``` - -3. Install Docker: - -``` -sudo yum update -y -sudo amazon-linux-extras install docker -y -sudo service docker start -sudo usermod -a -G docker ec2-user -``` - -4. Terminate the SSH session and then connect again. Verify that Docker is working by running: - -``` -docker info -``` - -5. Install Docker Machine: - -``` -base=https://github.com/docker/machine/releases/download/v0.16.2 \ -&& curl -L $base/docker-machine-$(uname -s)-$(uname -m) >/tmp/docker-machine \ -&& sudo mv /tmp/docker-machine /usr/local/bin/docker-machine \ -&& chmod +x /usr/local/bin/docker-machine -docker-machine version -``` - -### Configure security group for runners -You will now create a new security group that would allow the communication from the Runner Manager on ports 22 and 2376 (used by Docker Machine). - -{{%expand "Click to reveal detailed instructions" %}} -1. Return to the browser tab with EC2 console or open it again. -2. Choose **Security Groups** in the **Network & Security** section of the navigation pane. -3. Choose **Create security group**. -4. In the **Security group name** field type `GitLabRunner`. -5. In the **Description** field type `GitLab Runner Security Group`. -6. In the **VPC** field select the VPC ID you saved from CloudFormation Output values in [**Workshop Preparation**](/amazon-ec2-spot-cicd-workshop/gitlab-spot/prep.html), it contains `GitLabSpotWorkshop VPC` in its name. -6. Create two inbound rules by choosing **Add rule** in the **Inbound rules** pane: - - **Type** = `SSH`, **Source** = `Custom`, select the `GitLabRunnerManager` security group - - **Type** = `Custom TCP`, **Port range** = `2376`, **Source** = `Custom`, select the `GitLabRunnerManager` security group -7. Choose **Create security group**. -{{% /expand%}} - -### Configure runner -Finally, you will configure the runner and register it in GitLab. - -1. Return to the browser tab with GitLab. -2. In the **GitLab Spot Workshop** repository choose **Settings** > **CI/CD** in the navigation pane. -3. Expand the **Runners** section and save into a text file both GitLab URL and the registration token that are displayed on the screen: - -![GitLab Screenshot: Runners configuration](/images/gitlab-spot/GitLab-RunnersRegistration.png) - -3. Return to the browser tab with Cloud9. If you are connected via SSH to the Runner Manager in the terminal, create another terminal and run the following commands in it to get the AMI (Amazon Machine Image) ID to use for the runners, copy and save it: - -``` -TOKEN=$(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 600") -REGION=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/placement/region) -aws ssm get-parameters --names /aws/service/canonical/ubuntu/server/20.04/stable/current/amd64/hvm/ebs-gp2/ami-id --region $REGION --query "Parameters[0].Value" --output text -``` - -4. Return to the browser tab with SSH session with the Runner Manager (reconnect if it has been disconnected by timeout) and run the below command, substituting the values in the angle brackets with the required parameters (without angle brackets): - -``` -sudo gitlab-runner register \ - --non-interactive \ - --url "" \ - --registration-token "" \ - --executor "docker+machine" \ - --description "GitLab Spot Workshop Runner" \ - --docker-privileged \ - --docker-volumes "/var/run/docker.sock:/var/run/docker.sock" \ - --docker-image "docker:latest" \ - --docker-disable-cache="false" \ - --cache-type "s3" \ - --cache-path "/" \ - --cache-shared="true" \ - --cache-s3-server-address "s3.amazonaws.com" \ - --cache-s3-bucket-name "" \ - --cache-s3-bucket-location "" \ - --machine-idle-time 600 \ - --machine-machine-driver "amazonec2" \ - --machine-machine-name "gitlab-spot-workshop-%s" \ - --machine-machine-options "amazonec2-iam-instance-profile=GitLabRunner" \ - --machine-machine-options "amazonec2-region=" \ - --machine-machine-options "amazonec2-vpc-id=" \ - --machine-machine-options "amazonec2-subnet-id=" \ - --machine-machine-options "amazonec2-zone=" \ - --machine-machine-options "amazonec2-use-private-address=true" \ - --machine-machine-options "amazonec2-tags=runner-manager-name,aws-runner,gitlab,true,gitlab-runner-autoscale,true" \ - --machine-machine-options "amazonec2-security-group=GitLabRunner" \ - --machine-machine-options "amazonec2-request-spot-instance=true" \ - --machine-machine-options "amazonec2-instance-type=m5.xlarge" \ - --machine-machine-options "amazonec2-ami=" -``` - -Example: - -![GitLab Runner Screenshot: Runner manager registration](/images/gitlab-spot/GitLab-RunnerManagerRegistration.png) - -5. After successful registration, change the global `concurrent` parameter and restart the runner: - -``` -sudo sed -i 's/concurrent\s*=\s*1/concurrent = 10/' /etc/gitlab-runner/config.toml -sudo gitlab-runner restart -sudo gitlab-runner status -``` - -6. You can now exit the SSH session. -7. Return to the browser tab with GitLab and refresh the **CI/CD** page to make sure that your runner is now present in the **Runners** section: - -![GitLab Screenshot: Docker Machine runner available](/images/gitlab-spot/GitLab-DockerMachineRunnerAvailable.png) - -{{% notice tip %}} -One of the drawbacks of Docker Machine approach is that you can only specify one instance type and one subnet per Runner Manager. As such, it might happen that there is no spot capacity for it available and your CI/CD pipeline hangs running with the latest output in the job execution log staying at `Preparing the "docker+machine" executor`. If this happens, return to the EC2 console and choose **Spot Requests** in the navigation pane. Check if your requests are indeed not getting fulfilled, and if so, on the Runner Manager instance try changing the instance type or the subnet and availability zone in the `/etc/gitlab-runner/config.toml` file and restart the `gitlab-runner`. For a more permanent solution, please use auto-scaling groups as discussed in [**Lab 2: Configure GitLab runners on Spot instances**](/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2.html). -{{% /notice %}} - -You can now proceed to build your application in [**Lab 3: Building the demo app**](/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab3.html). \ No newline at end of file diff --git a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/gitlab-easybutton/ap-southeast-1.md b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/gitlab-easybutton/ap-southeast-1.md deleted file mode 100644 index 8cfd4d81..00000000 --- a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/gitlab-easybutton/ap-southeast-1.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -title: "Singapore" -chapter: false -disableToc: true -hidden: true ---- - -Deploy GitLab stack: https://ap-southeast-1.console.aws.amazon.com/cloudformation/home?region=ap-southeast-1#/stacks/create/review?templateURL=https://gl-public-templates.s3.amazonaws.com/cfn/v1.4.9-alpha14/easybutton-amazon-linux-2-docker-simple-scaling-spotonly.cf.yml&stackName=linux-docker-scaling-spotonly diff --git a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/gitlab-easybutton/eu-central-1.md b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/gitlab-easybutton/eu-central-1.md deleted file mode 100644 index 31288421..00000000 --- a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/gitlab-easybutton/eu-central-1.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -title: "Frankfurt" -chapter: false -disableToc: true -hidden: true ---- - -Deploy GitLab stack: https://eu-central-1.console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stacks/create/review?templateURL=https://gl-public-templates.s3.amazonaws.com/cfn/v1.4.9-alpha14/easybutton-amazon-linux-2-docker-simple-scaling-spotonly.cf.yml&stackName=linux-docker-scaling-spotonly \ No newline at end of file diff --git a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/gitlab-easybutton/eu-west-1.md b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/gitlab-easybutton/eu-west-1.md deleted file mode 100644 index 1bd930a4..00000000 --- a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/gitlab-easybutton/eu-west-1.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -title: "Ireland" -chapter: false -disableToc: true -hidden: true ---- - -Deploy GitLab stack: https://eu-west-1.console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/create/review?templateURL=https://gl-public-templates.s3.amazonaws.com/cfn/v1.4.9-alpha14/easybutton-amazon-linux-2-docker-simple-scaling-spotonly.cf.yml&stackName=linux-docker-scaling-spotonly \ No newline at end of file diff --git a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/gitlab-easybutton/us-east-1.md b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/gitlab-easybutton/us-east-1.md deleted file mode 100644 index f5d3be57..00000000 --- a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/gitlab-easybutton/us-east-1.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -title: "N.Virginia" -chapter: false -disableToc: true -hidden: true ---- - -Deploy GitLab stack: https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/review?templateURL=https://gl-public-templates.s3.amazonaws.com/cfn/v1.4.9-alpha14/easybutton-amazon-linux-2-docker-simple-scaling-spotonly.cf.yml&stackName=linux-docker-scaling-spotonly \ No newline at end of file diff --git a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/gitlab-easybutton/us-east-2.md b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/gitlab-easybutton/us-east-2.md deleted file mode 100644 index 3c734335..00000000 --- a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/gitlab-easybutton/us-east-2.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -title: "Ohio" -chapter: false -disableToc: true -hidden: true ---- - -Deploy GitLab stack: https://us-east-2.console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/create/review?templateURL=https://gl-public-templates.s3.amazonaws.com/cfn/v1.4.9-alpha14/easybutton-amazon-linux-2-docker-simple-scaling-spotonly.cf.yml&stackName=linux-docker-scaling-spotonly \ No newline at end of file diff --git a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/gitlab-easybutton/us-west-2.md b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/gitlab-easybutton/us-west-2.md deleted file mode 100644 index 1ed6d0fa..00000000 --- a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab2/gitlab-easybutton/us-west-2.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -title: "Oregon" -chapter: false -disableToc: true -hidden: true ---- - -Deploy GitLab stack: https://us-west-2.console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/create/review?templateURL=https://gl-public-templates.s3.amazonaws.com/cfn/v1.4.9-alpha14/easybutton-amazon-linux-2-docker-simple-scaling-spotonly.cf.yml&stackName=linux-docker-scaling-spotonly \ No newline at end of file diff --git a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab3.md b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab3.md index e06b7c0e..347ea914 100644 --- a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab3.md +++ b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab3.md @@ -1,12 +1,12 @@ +++ -title = "Lab 3: Building the demo app" +title = "Building the demo app" weight = 50 +++ In this lab you will push the changes to your origin repository and verify that the pipeline has successfully finished both in GitLab and by checking the image in Amazon ECR. {{%expand "Click to reveal detailed instructions" %}} -1. Return to the browser tab with Cloud9 and execute the following command in the terminal. Specify `root` as the username and the same password you used in [**Lab 1: Create a GitLab repository**](lab1.html) to log in to GitLab: +1. Return to the browser tab with Cloud9 and execute the following command in the terminal. Specify `root` as the username and the same password you used in [**Create a GitLab repository**](lab1.html) to log in to GitLab: ``` git push -u origin main @@ -19,13 +19,13 @@ git push -u origin main 4. Return to the browser tab with AWS Console. 5. Type `ECR` in the search box at the top and open the **Elastic Container Registry** service. -6. Open the **gitlab-spot-demo** repository and verify that it contains an image that has just been built in GitLab: +6. Open the repository with **gitlab-spot-demo** in its name and verify that it contains an image that has just been built in GitLab: ![ECR Console Screenshot: Images](/images/gitlab-spot/AWSConsole-ECRImages.png) {{% /expand%}} -You have successfully built the image and can now proceed to [**Lab 4: Deploying Amazon EKS on Spot instances**](lab4.html). +You have successfully built the image and can now proceed to [**Deploying Amazon EKS on Spot instances**](lab4.html). ### Challenges @@ -33,6 +33,18 @@ If this and previous labs seemed too easy, try completing the following challeng **Challenge 1:** Configure shared runners for the whole GitLab CI/CD and not just the current repository. Create an additional repository and verify that your runners serve it too. -**Challenge 2:** If you used the auto-scaling group approach, trigger auto-scaling of the instances to get more runners created (tip: by default, it is done by CPU load, so you can run a CPU load tool inside your build scripts to simulate it). +{{%expand "Click to reveal a hint" %}} +Open the Admin Area of GitLab interface and in the Runners section get the registration code. Use it instead of the one from the project. +{{% /expand%}} + +**Challenge 2:** Trigger auto-scaling of the instances to get more runners created. + +{{%expand "Click to reveal a hint" %}} +By default, auto-scaling is done by CPU load, so you can run a CPU load generator inside your build scripts to simulate it, for example, you can use `stress` or `stress-ng` tools for this. +{{% /expand%}} + +**Challenge 3:** Modify the auto-scaling group to perform scaling by the number of the jobs instead of the CPU load. -**Challenge 3:** If you used the auto-scaling group approach, try modifying it to perform scaling by the number of the jobs instead of the CPU load (you will need to create a custom CloudWatch metric for this). \ No newline at end of file +{{%expand "Click to reveal a hint" %}} +Create a Lambda function that gets the number of jobs from [GitLab API](https://docs.gitlab.com/ee/api/jobs.html#list-project-jobs) and publishes it as a custom CloudWatch metric. Use this metric to scale the group. +{{% /expand%}} \ No newline at end of file diff --git a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab4.md b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab4.md index 0e264ce1..8eaf013f 100644 --- a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab4.md +++ b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab4.md @@ -1,5 +1,5 @@ +++ -title = "Lab 4: Deploying Amazon EKS on Spot instances" +title = "Deploying Amazon EKS on Spot instances" weight = 50 +++ @@ -29,7 +29,7 @@ terraform apply ``` 3. When asked for the parameters, enter the following values: - * **kubernetes_version**: the latest version of Kubernetes that you can find in [Amazon EKS documentation](https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html): only major and minor version separated with a dot without patch release, for example, `1.22` + * **kubernetes_version**: the latest version of Kubernetes that you can find in [Amazon EKS documentation](https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html): only major and minor version separated with a dot without patch release, for example, `1.23` * **vpc_id**: the VPC ID you saved from CloudFormation Output values in [**Workshop Preparation**](/amazon-ec2-spot-cicd-workshop/gitlab-spot/prep.html) in the format `vpc-...` ![Cloud9 Console Screenshot: Terraform variables](/images/gitlab-spot/Cloud9-TerraformVars.png) @@ -88,7 +88,7 @@ cat << EoF EoF ``` -3. Copy the output of the last command, then in the file tree on the left open the file `gitlab-spot-workshop/aws-auth.yaml` by double-clicking it, and finally add the previous command's output that you copied before into `mapRoles` section. You should have something similar to the below screenshot: +3. Copy the output of the last command, then in the file tree on the left open the file `aws-auth.yaml` by double-clicking it, and finally add the previous command's output that you copied before into `mapRoles` section. You should have something similar to the below screenshot: ![Cloud9 Console Screenshot: aws-auth ConfigMap](/images/gitlab-spot/Cloud9-AWSAuth.png) @@ -101,7 +101,7 @@ kubectl apply -f aws-auth.yaml {{% /expand%}} -You are now ready for the final steps to deploy your demo application into the cluster in [**Lab 5: Installing the demo app into Amazon EKS**](lab5.html). +You are now ready for the final steps to deploy your demo application into the cluster in [**Installing the demo app into Amazon EKS**](lab5.html). ### Challenge diff --git a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab5.md b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab5.md index d05d0ef8..2b970498 100644 --- a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab5.md +++ b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab5.md @@ -1,12 +1,12 @@ +++ -title = "Lab 5: Installing the demo app into Amazon EKS" +title = "Installing the demo app into Amazon EKS" weight = 50 +++ -In this lab you will deploy the demo application you built earlier in the new Amazon EKS cluster deployed fully on spot instances. +In this section you will deploy the demo application you built earlier in the new Amazon EKS cluster deployed fully on spot instances. {{%expand "Click to reveal detailed instructions" %}} -1. In the Cloud9 file tree on the left open file `gitlab-spot-workshop/amazon-ec2-spot-cicd-workshop/gitlab-spot/demo-app/.gitlab-ci.yml` (if you don't see it, make sure you have enabled the hidden files in [**Workshop Preparation**](prep.html)). +1. In the Cloud9 file tree on the left open file `amazon-ec2-spot-cicd-workshop/gitlab-spot/demo-app/.gitlab-ci.yml` (if you don't see it, make sure you have enabled the hidden files in [**Workshop Preparation**](prep.html)). 2. Change the jobs `deploy_to_eks` and `test_on_eks` to the following ones: ``` diff --git a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/prep.md b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/prep.md index bfbabb36..0d059910 100644 --- a/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/prep.md +++ b/content/amazon-ec2-spot-cicd-workshop/gitlab-spot/prep.md @@ -7,115 +7,48 @@ You should now have the CloudFormation stack with GitLab opened. If not, refer t ### Save GitLab access details -Switch to the **Outputs** tab and save all the information in the table to a text file. You will need it in the later labs. +Switch to the **Outputs** tab and save all the information in the table to a text file. You will need it in the later steps of this workshop. ![CloudFormation Console Screenshot: Stack outputs](/images/gitlab-spot/AWSConsole-CloudFormationStackOutput.png) -### Create an AWS Cloud9 environment +### Log in to AWS Cloud9 environment You can execute the steps of this workshop directly on your workstation, but then you will need to make sure that you have the command-line tools for Git, Terraform, kubectl and AWS CLI installed. Instead of that, to not change any local settings, we recommend to use [AWS Cloud9](https://aws.amazon.com/cloud9/): a cloud IDE where you can get access to the terminal and install all the required tools. -{{% notice warning %}} -If you are running the workshop on your own, the Cloud9 workspace should be built by an IAM user with Administrator privileges, not the root account user. Please ensure you are logged in as an IAM user, not the root -account user. -{{% /notice %}} +A Cloud9 environment has already been provisioned for you in the CloudFormation template (created in the [**Starting the workshop**](before.html) section). You will now log in to it and do the final configuration steps: disable the AWS managed temporary credentials and download the required workshop files. -{{% notice info %}} -If you are at an AWS hosted event (such as re:Invent, Kubecon, Immersion Day, or any other event hosted by -an AWS employee) follow the instructions on the region that should be used to launch resources -{{% /notice %}} +1. In the AWS Console enter **Cloud9** in the search box at the top of the screen and open the service. +2. On the **Your environments** page find the environment called like `GitLabWorkshopC9Instance-...` and click the **Open IDE** button for it: -{{% notice tip %}} -Ad blockers, javascript disablers, and tracking blockers should be disabled for -the cloud9 domain, or connecting to the workspace might be impacted. -Cloud9 requires third-party-cookies. You can whitelist the [specific domains]( https://docs.aws.amazon.com/cloud9/latest/user-guide/troubleshooting.html#troubleshooting-env-loading). -{{% /notice %}} +![AWS Console Screenshot: Cloud9 Environment](/images/gitlab-spot/AWSConsole-Cloud9Environment.png) -Launch Cloud9 in your closest region: - -{{< tabs name="Region" >}} - {{< tab name="N. Virginia" include="cloud9/us-east-1.md" />}} - {{< tab name="Oregon" include="cloud9/us-west-2.md" />}} - {{< tab name="Frankfurt" include="cloud9/eu-central-1.md" />}} - {{< tab name="Ireland" include="cloud9/eu-west-1.md" />}} - {{< tab name="Ohio" include="cloud9/us-east-2.md" />}} - {{< tab name="Singapore" include="cloud9/ap-southeast-1.md" />}} -{{< /tabs >}} - -1. Choose **Create environment**. -2. Name it `gitlab-spot-workshop` and choose **Next step**. -3. Expand **Network settings (advanced)** section and select the VPC from the Output parameters you saved above (should contain `GitLabSpotWorkshop VPC` in its name), the subnet can be any of the two: - -![Cloud9 Console Screenshot: Select VPC](/images/gitlab-spot/AWSConsole-CreateCloud9SelectVPC.png) - -4. Choose **Next step**. -5. Choose **Create environment**. -6. When the environment comes up, close all tabs inside it and open a new terminal by clicking **+** > **New Terminal**, you should see a screen like this: +3. When the environment comes up, close all tabs inside it and open a new terminal by clicking **+** > **New Terminal**, you should see a screen like this: ![Cloud9 Environment Screenshot: Initial view](/images/gitlab-spot/Cloud9-Initial.png) -### Create an IAM role - -By default, Cloud9 uses AWS managed temporary credentials to make calls to AWS API from the terminal. Instead, you will create an IAM role and EC2 instance profile to ensure the correct Role Based Access Control (RBAC) permissions in Kubernetes cluster later on. - -1. Return to the browser tab with AWS Console (if you closed it, click the icon with the cloud and digit 9 in the top-left corner and then choose **Go To Your Dashboard**) -2. Type `IAM` in the search box at the top and open the service. -3. In the navigation pane choose **Roles** and then choose **Create role**. -4. In the **Use case** section select **EC2** and choose **Next**. -5. In the **Permissions policies** table find `AdministratorAccess` and mark the checkbox next to it, then choose **Next** - -![IAM Console Screenshot: AdministratorAccess permissions](/images/gitlab-spot/AWSConsole-IAMRolePermissions.png) - -6. In the **Role name** field type `gitlab-spot-workshop-admin` and choose **Create role** in the bottom of the screen. - -### Attach IAM role to EC2 instance - -Now, you will attach the new role to the EC2 instance used by your Cloud9 environment. - -1. Type `EC2` in the search box at the top and open the service. -2. In the navigation pane choose **Instances**. -3. You should see two instances in the list. Select the one with name starting with `aws-cloud9-`, then choose **Actions** > **Security** > **Modify IAM role**: - -![EC2 Console Screenshot: Modify the instance role](/images/gitlab-spot/AWSConsole-EC2Instances.png) - -4. In the **IAM role** dropdown select `gitlab-spot-workshop-admin` and choose **Save**. - -### Configure Cloud9 environment - -You will now finalize AWS Cloud9 environment configuration by disabling the AWS managed temporary credentials and download the required workshop files. - -1. Return to the browser tab with Cloud9 environment. -2. Open preferences tab by choosing the cogwheel icon at the top-right corner of the screen. Then choose **AWS Settings** in the navigation pane. -3. Disable **AWS managed temporary credentials** toggle: +4. Open preferences tab by choosing the cogwheel icon at the top-right corner of the screen. Then choose **AWS Settings** in the navigation pane. +5. Disable **AWS managed temporary credentials** toggle: ![Cloud9 Screenshot: Preferences](/images/gitlab-spot/Cloud9-Preferences.png) -4. Close the **Preferences** tab and in the terminal tab, execute the following command to verify that you are using the correct role (in the output you should see `gitlab-spot-workshop-admin`): +6. Close the **Preferences** tab and in the terminal tab, execute the following command to verify that you are using the correct role (in the output you should see `GitLabWorkshopC9Role` and green `OK`): ``` -aws sts get-caller-identity --query Arn +aws sts get-caller-identity --query Arn | grep GitLabWorkshopC9Role && echo -e "\033[0;32mOK\033[0m" || echo -e "\033[0;31mNOT OK\033[0m" ``` ![Cloud9 Screenshot: Caller identity](/images/gitlab-spot/Cloud9-CallerIdentity.png) -5. Execute the following commands in the terminal to copy all workshop files: - -``` -git clone https://github.com/awslabs/ec2-spot-workshops.git -cp -ar ec2-spot-workshops/workshops/amazon-ec2-spot-cicd-workshop/ . -rm -rf ec2-spot-workshops/ -``` - -6. Now you will upload the SSH key you saved in the previous section. Choose **File** > **Upload Local Files...**. -7. In the popup page choose **Select files** and select the file `ee-default-keypair.pem` that you downloaded previously. -8. Make sure the file appeared in the file tree on the left, close the popup page, and modify the file permissions by executing the following command in the terminal: +7. Now you will upload the SSH key you saved in the previous section. Choose **File** > **Upload Local Files...**. +8. In the popup page choose **Select files** and select the file `ee-default-keypair.pem` that you downloaded previously. +9. Make sure the file appeared in the file tree on the left, close the popup page, and modify the file permissions by executing the following command in the terminal: ``` chmod 400 ~/environment/ee-default-keypair.pem ``` -9. Finally, enable Cloud9 to show hidden files (you will need it to modify the scripts of GitLab CI/CD later). To do this choose the small cogwheel icon right above the file tree and choose **Show Hidden Files**: +10. Finally, enable Cloud9 to show hidden files (you will need it to modify the scripts of GitLab CI/CD later). To do this choose the small cogwheel icon right above the file tree and choose **Show Hidden Files** if it has not been enabled yet (if there is a tick to the left of it, do not click again, as it will disable the display of hidden files): ![Cloud9 Screenshot: Show hidden files](/images/gitlab-spot/Cloud9-ShowHiddenFiles.png) -You are now ready to start the main labs of the workshop! Please proceed to [**Lab 1: Create a GitLab repository**](lab1.html). \ No newline at end of file +You are now ready to start the main sections of the workshop! Please proceed to [**Create a GitLab repository**](lab1.html). \ No newline at end of file diff --git a/static/images/gitlab-spot/AWSConsole-Cloud9Environment.png b/static/images/gitlab-spot/AWSConsole-Cloud9Environment.png new file mode 100644 index 00000000..4ed370a9 Binary files /dev/null and b/static/images/gitlab-spot/AWSConsole-Cloud9Environment.png differ diff --git a/static/images/gitlab-spot/AWSConsole-CloudFormationGitLabRunnersStack.png b/static/images/gitlab-spot/AWSConsole-CloudFormationGitLabRunnersStack.png index fbf6d1b9..9e4146b0 100644 Binary files a/static/images/gitlab-spot/AWSConsole-CloudFormationGitLabRunnersStack.png and b/static/images/gitlab-spot/AWSConsole-CloudFormationGitLabRunnersStack.png differ diff --git a/static/images/gitlab-spot/AWSConsole-CloudFormationNestedStack.png b/static/images/gitlab-spot/AWSConsole-CloudFormationNestedStack.png deleted file mode 100644 index 9a6e9830..00000000 Binary files a/static/images/gitlab-spot/AWSConsole-CloudFormationNestedStack.png and /dev/null differ diff --git a/static/images/gitlab-spot/AWSConsole-CloudFormationStackOutput.png b/static/images/gitlab-spot/AWSConsole-CloudFormationStackOutput.png index d1a79054..39c3c8d0 100644 Binary files a/static/images/gitlab-spot/AWSConsole-CloudFormationStackOutput.png and b/static/images/gitlab-spot/AWSConsole-CloudFormationStackOutput.png differ diff --git a/static/images/gitlab-spot/AWSConsole-CreateCloud9SelectVPC.png b/static/images/gitlab-spot/AWSConsole-CreateCloud9SelectVPC.png deleted file mode 100644 index 6f8f1b37..00000000 Binary files a/static/images/gitlab-spot/AWSConsole-CreateCloud9SelectVPC.png and /dev/null differ diff --git a/static/images/gitlab-spot/AWSConsole-EC2Instances.png b/static/images/gitlab-spot/AWSConsole-EC2Instances.png deleted file mode 100644 index fd124f5a..00000000 Binary files a/static/images/gitlab-spot/AWSConsole-EC2Instances.png and /dev/null differ diff --git a/static/images/gitlab-spot/AWSConsole-EC2LaunchInstance.png b/static/images/gitlab-spot/AWSConsole-EC2LaunchInstance.png deleted file mode 100644 index 5c33d346..00000000 Binary files a/static/images/gitlab-spot/AWSConsole-EC2LaunchInstance.png and /dev/null differ diff --git a/static/images/gitlab-spot/AWSConsole-ECRImages.png b/static/images/gitlab-spot/AWSConsole-ECRImages.png index 0466cdde..03d3780f 100644 Binary files a/static/images/gitlab-spot/AWSConsole-ECRImages.png and b/static/images/gitlab-spot/AWSConsole-ECRImages.png differ diff --git a/static/images/gitlab-spot/AWSConsole-IAMCreatePolicy.png b/static/images/gitlab-spot/AWSConsole-IAMCreatePolicy.png index 6051bdeb..2ea81b4b 100644 Binary files a/static/images/gitlab-spot/AWSConsole-IAMCreatePolicy.png and b/static/images/gitlab-spot/AWSConsole-IAMCreatePolicy.png differ diff --git a/static/images/gitlab-spot/AWSConsole-IAMRolePermissions.png b/static/images/gitlab-spot/AWSConsole-IAMRolePermissions.png deleted file mode 100644 index aa6b0013..00000000 Binary files a/static/images/gitlab-spot/AWSConsole-IAMRolePermissions.png and /dev/null differ diff --git a/static/images/gitlab-spot/Cloud9-CallerIdentity.png b/static/images/gitlab-spot/Cloud9-CallerIdentity.png index 176ed0c7..b0fc5672 100644 Binary files a/static/images/gitlab-spot/Cloud9-CallerIdentity.png and b/static/images/gitlab-spot/Cloud9-CallerIdentity.png differ diff --git a/static/images/gitlab-spot/Cloud9-Initial.png b/static/images/gitlab-spot/Cloud9-Initial.png index a28ed1ec..71f5b7f2 100644 Binary files a/static/images/gitlab-spot/Cloud9-Initial.png and b/static/images/gitlab-spot/Cloud9-Initial.png differ diff --git a/static/images/gitlab-spot/GitLab-DockerMachineRunnerAvailable.png b/static/images/gitlab-spot/GitLab-DockerMachineRunnerAvailable.png deleted file mode 100644 index 40abeb9a..00000000 Binary files a/static/images/gitlab-spot/GitLab-DockerMachineRunnerAvailable.png and /dev/null differ diff --git a/static/images/gitlab-spot/GitLab-RunnerAvailable.png b/static/images/gitlab-spot/GitLab-RunnerAvailable.png index f2f8bc6e..11f554a7 100644 Binary files a/static/images/gitlab-spot/GitLab-RunnerAvailable.png and b/static/images/gitlab-spot/GitLab-RunnerAvailable.png differ diff --git a/static/images/gitlab-spot/GitLab-RunnerManagerRegistration.png b/static/images/gitlab-spot/GitLab-RunnerManagerRegistration.png deleted file mode 100644 index 4f03b4d6..00000000 Binary files a/static/images/gitlab-spot/GitLab-RunnerManagerRegistration.png and /dev/null differ diff --git a/static/images/gitlab-spot/gitlab-spot-architecture.png b/static/images/gitlab-spot/gitlab-spot-architecture.png new file mode 100644 index 00000000..9c76e0aa Binary files /dev/null and b/static/images/gitlab-spot/gitlab-spot-architecture.png differ diff --git a/static/images/gitlab-spot/lab-flow.png b/static/images/gitlab-spot/lab-flow.png new file mode 100644 index 00000000..f3426cc9 Binary files /dev/null and b/static/images/gitlab-spot/lab-flow.png differ diff --git a/workshops/amazon-ec2-spot-cicd-workshop/gitlab-spot/demo-app/template-gitlab-ci.yml b/workshops/amazon-ec2-spot-cicd-workshop/gitlab-spot/demo-app/template-gitlab-ci.yml index 99b4721a..ccc0e538 100644 --- a/workshops/amazon-ec2-spot-cicd-workshop/gitlab-spot/demo-app/template-gitlab-ci.yml +++ b/workshops/amazon-ec2-spot-cicd-workshop/gitlab-spot/demo-app/template-gitlab-ci.yml @@ -1,6 +1,6 @@ variables: ECR_REGISTRY: ${ECR_ADDRESS} - IMAGE_TAG: gitlab-spot-demo:$CI_COMMIT_SHA + IMAGE_TAG: ${ECR_NAME}:$CI_COMMIT_SHA REGION: ${AWS_REGION} K8S_CLUSTER_NAME: gitlab-spot-workshop DOCKER_DRIVER: overlay2 diff --git a/workshops/amazon-ec2-spot-cicd-workshop/gitlab-spot/gitlab-deploy.yml b/workshops/amazon-ec2-spot-cicd-workshop/gitlab-spot/gitlab-deploy.yml index 9e90357d..20e38bd4 100644 --- a/workshops/amazon-ec2-spot-cicd-workshop/gitlab-spot/gitlab-deploy.yml +++ b/workshops/amazon-ec2-spot-cicd-workshop/gitlab-spot/gitlab-deploy.yml @@ -2,9 +2,12 @@ AWSTemplateFormatVersion: 2010-09-09 Description: GitLab Spot Workshop Parameters: - SSHKeyName: + EEKeyPair: Description: Name of the EC2 key for GitLab Type: 'AWS::EC2::KeyPair::KeyName' + EETeamRoleArn: + Description: ARN of the user / role that will be owner of Cloud9 environment. Leave empty to use the current entity + Type: String AmiId: Description: Do not change Type: 'AWS::SSM::Parameter::Value' @@ -25,6 +28,11 @@ Parameters: Description: Project name for the tags Type: String Default: GitLabSpotWorkshop + AllowedPattern : '^[a-zA-Z0-9\-]+$' + +Conditions: + EmptyC9Owner: !Equals [!Ref EETeamRoleArn, ''] + IsEERole: !Equals [!Ref EETeamRoleArn, !Sub 'arn:aws:iam::${AWS::AccountId}:role/TeamRole'] Resources: VPC: @@ -205,6 +213,9 @@ Resources: Role: !GetAtt LambdaExecutionRole.Arn MemorySize: 128 Timeout: 3 + Tags: + - Key: Project + Value: !Ref EnvironmentName GitLabPassword: Type: AWS::CloudFormation::CustomResource @@ -212,58 +223,108 @@ Resources: Length: 16 ServiceToken: !GetAtt RandomStringLambdaFunction.Arn - GitLabInstance: - Type: AWS::EC2::Instance + LowerStringLambdaFunction: + Type: AWS::Lambda::Function + Properties: + Architectures: + - arm64 + Code: + ZipFile: | + import json + import cfnresponse + import random + import string + + def handler(event, context): + responseData = {} + responseData["OutputString"] = event["ResourceProperties"]["InputString"].lower() + + cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData) + Handler: index.handler + Runtime: python3.8 + Role: !GetAtt LambdaExecutionRole.Arn + MemorySize: 128 + Timeout: 3 + Tags: + - Key: Project + Value: !Ref EnvironmentName + + EnvironmentNameLower: + Type: AWS::CloudFormation::CustomResource + Properties: + InputString: !Ref EnvironmentName + ServiceToken: !GetAtt LowerStringLambdaFunction.Arn + + GitLabLaunchTemplate: + Type: AWS::EC2::LaunchTemplate DependsOn: CloudFrontDistribution Properties: - ImageId: !Ref AmiId - InstanceType: m5.xlarge - KeyName: !Ref SSHKeyName - NetworkInterfaces: - - AssociatePublicIpAddress: true - DeviceIndex: 0 - GroupSet: - - !Ref EC2SecurityGroup - SubnetId: !Ref Subnet1 - UserData: - Fn::Base64: !Sub | - #!/bin/bash -xe - apt-get update - apt-get install -y curl openssh-server ca-certificates tzdata perl python3-setuptools - curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.deb.sh | bash - GITLAB_ROOT_EMAIL="test@workshop.tld" GITLAB_ROOT_PASSWORD="${GitLabPassword.RandomString}" EXTERNAL_URL="http://${CloudFrontDistribution.DomainName}" apt-get install gitlab-ee - sed -i 's|http://${CloudFrontDistribution.DomainName}|https://${CloudFrontDistribution.DomainName}|' /etc/gitlab/gitlab.rb - echo "letsencrypt['enable'] = false" >> /etc/gitlab/gitlab.rb - echo "nginx['listen_port'] = 80" >> /etc/gitlab/gitlab.rb - echo "nginx['listen_https'] = false" >> /etc/gitlab/gitlab.rb - gitlab-ctl reconfigure - gitlab-rails runner "ApplicationSetting.last.update_attribute(:signup_enabled, false)" - wget https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-py3-latest.tar.gz - python3 /usr/lib/python3/dist-packages/easy_install.py --script-dir /opt/aws/bin aws-cfn-bootstrap-py3-latest.tar.gz - /opt/aws/bin/cfn-signal \ - -e $? \ - --stack ${AWS::StackName} \ - --resource GitLabInstance \ - --region ${AWS::Region} + LaunchTemplateData: + ImageId: !Ref AmiId + InstanceType: m5.xlarge + KeyName: !Ref EEKeyPair + NetworkInterfaces: + - AssociatePublicIpAddress: true + DeviceIndex: 0 + Groups: + - !Ref EC2SecurityGroup + UserData: + Fn::Base64: !Sub | + #!/bin/bash -xe + apt-get update + apt-get install -y curl openssh-server ca-certificates tzdata perl python3-setuptools + curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.deb.sh | bash + GITLAB_ROOT_EMAIL="test@workshop.tld" GITLAB_ROOT_PASSWORD="${GitLabPassword.RandomString}" EXTERNAL_URL="http://${CloudFrontDistribution.DomainName}" apt-get install gitlab-ee + sed -i 's|http://${CloudFrontDistribution.DomainName}|https://${CloudFrontDistribution.DomainName}|' /etc/gitlab/gitlab.rb + echo "letsencrypt['enable'] = false" >> /etc/gitlab/gitlab.rb + echo "nginx['listen_port'] = 80" >> /etc/gitlab/gitlab.rb + echo "nginx['listen_https'] = false" >> /etc/gitlab/gitlab.rb + gitlab-ctl reconfigure + gitlab-rails runner "ApplicationSetting.last.update_attribute(:signup_enabled, false)" + wget https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-py3-latest.tar.gz + python3 /usr/lib/python3/dist-packages/easy_install.py --script-dir /opt/aws/bin aws-cfn-bootstrap-py3-latest.tar.gz + /opt/aws/bin/cfn-signal \ + -e $? \ + --stack ${AWS::StackName} \ + --resource GitLabAutoScalingGroup \ + --region ${AWS::Region} + TagSpecifications: + - ResourceType: instance + Tags: + - Key: Name + Value: !Sub ${EnvironmentName} GitLab + - Key: Project + Value: !Ref EnvironmentName + + GitLabAutoScalingGroup: + Type: AWS::AutoScaling::AutoScalingGroup + DependsOn: InternetGatewayAttachment + Properties: + LaunchTemplate: + LaunchTemplateId: !Ref GitLabLaunchTemplate + Version: !GetAtt GitLabLaunchTemplate.LatestVersionNumber + MinSize: 1 + MaxSize: 1 + TargetGroupARNs: [!Ref EC2TargetGroup] + VPCZoneIdentifier: + - Ref: Subnet1 + - Ref: Subnet2 Tags: - - Key: Name - Value: !Sub ${EnvironmentName} GitLab - Key: Project + PropagateAtLaunch: false Value: !Ref EnvironmentName CreationPolicy: ResourceSignal: + Timeout: PT45M Count: 1 - Timeout: "PT45M" EC2TargetGroup: Type: AWS::ElasticLoadBalancingV2::TargetGroup Properties: - Name: GitLab-Target-Group + Matcher: + HttpCode: 200,302 Port: 80 Protocol: HTTP - Targets: - - Id: !Ref GitLabInstance - Port: 80 VpcId: !Ref VPC Tags: - Key: Name @@ -333,7 +394,7 @@ Resources: DemoRepo: Type: AWS::ECR::Repository Properties: - RepositoryName: gitlab-spot-demo + RepositoryName: !Sub ${EnvironmentNameLower.OutputString}-gitlab-spot-demo Tags: - Key: Project Value: !Ref EnvironmentName @@ -341,6 +402,233 @@ Resources: GitLabCacheBucket: Type: AWS::S3::Bucket + GitLabWorkshopC9Role: + Type: AWS::IAM::Role + Properties: + Tags: + - Key: Environment + Value: AWS Example + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + Service: + - ec2.amazonaws.com + - ssm.amazonaws.com + Action: + - sts:AssumeRole + ManagedPolicyArns: + - arn:aws:iam::aws:policy/AdministratorAccess + Path: "/" + + GitLabWorkshopC9LambdaExecutionRole: + Type: AWS::IAM::Role + Properties: + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + Service: + - lambda.amazonaws.com + Action: + - sts:AssumeRole + Path: "/" + Policies: + - PolicyName: + Fn::Join: + - '' + - - GitLabWorkshopC9LambdaPolicy- + - Ref: EnvironmentName + - '-' + - Ref: AWS::Region + PolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Action: + - logs:CreateLogGroup + - logs:CreateLogStream + - logs:PutLogEvents + Resource: arn:aws:logs:*:*:* + - Effect: Allow + Action: + - cloudformation:DescribeStacks + - cloudformation:DescribeStackEvents + - cloudformation:DescribeStackResource + - cloudformation:DescribeStackResources + - ec2:DescribeInstances + - ec2:AssociateIamInstanceProfile + - ec2:ModifyInstanceAttribute + - ec2:ReplaceIamInstanceProfileAssociation + - iam:ListInstanceProfiles + - iam:PassRole + Resource: "*" + + GitLabWorkshopC9BootstrapInstanceLambda: + Description: Bootstrap Cloud9 instance + Type: Custom::GitLabWorkshopC9BootstrapInstanceLambda + DependsOn: + - GitLabWorkshopC9BootstrapInstanceLambdaFunction + - GitLabWorkshopC9Instance + - GitLabWorkshopC9InstanceProfile + - GitLabWorkshopC9LambdaExecutionRole + Properties: + ServiceToken: + Fn::GetAtt: + - GitLabWorkshopC9BootstrapInstanceLambdaFunction + - Arn + REGION: + Ref: AWS::Region + StackName: + Ref: AWS::StackName + EnvironmentId: + Ref: GitLabWorkshopC9Instance + LabIdeInstanceProfileName: + Ref: GitLabWorkshopC9InstanceProfile + LabIdeInstanceProfileArn: + Fn::GetAtt: + - GitLabWorkshopC9InstanceProfile + - Arn + + GitLabWorkshopC9BootstrapInstanceLambdaFunction: + Type: AWS::Lambda::Function + Properties: + Handler: index.lambda_handler + Role: + Fn::GetAtt: + - GitLabWorkshopC9LambdaExecutionRole + - Arn + Runtime: python3.8 + MemorySize: 256 + Timeout: 600 + Code: + ZipFile: | + import boto3 + import json + import time + import traceback + import cfnresponse + + def lambda_handler(event, context): + # logger.info('event: {}'.format(event)) + # logger.info('context: {}'.format(context)) + responseData = {} + + if event['RequestType'] == 'Create': + try: + # Open AWS clients + ec2 = boto3.client('ec2') + + # Get the InstanceId of the Cloud9 IDE + instance = ec2.describe_instances(Filters=[{'Name': 'tag:aws:cloud9:environment','Values': [event['ResourceProperties']['EnvironmentId']]}])['Reservations'][0]['Instances'][0] + # logger.info('instance: {}'.format(instance)) + + # Create the IamInstanceProfile request object + iam_instance_profile = { + 'Arn': event['ResourceProperties']['LabIdeInstanceProfileArn'], + 'Name': event['ResourceProperties']['LabIdeInstanceProfileName'] + } + # logger.info('iam_instance_profile: {}'.format(iam_instance_profile)) + + # Wait for Instance to become ready before adding Role + instance_state = instance['State']['Name'] + # logger.info('instance_state: {}'.format(instance_state)) + while instance_state != 'running': + time.sleep(5) + instance_state = ec2.describe_instances(InstanceIds=[instance['InstanceId']]) + # logger.info('instance_state: {}'.format(instance_state)) + + # attach instance profile + response = ec2.associate_iam_instance_profile(IamInstanceProfile=iam_instance_profile, InstanceId=instance['InstanceId']) + # logger.info('response - associate_iam_instance_profile: {}'.format(response)) + r_ec2 = boto3.resource('ec2') + + responseData = {'Success': 'Started bootstrapping for instance: '+instance['InstanceId']} + cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData, 'CustomResourcePhysicalID') + + except Exception as e: + # logger.error(e, exc_info=True) + responseData = {'Error': traceback.format_exc()} + cfnresponse.send(event, context, cfnresponse.FAILED, responseData, 'CustomResourcePhysicalID') + else: + responseData = {'Success': 'Nothing to do'} + cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData) + Tags: + - Key: Project + Value: !Ref EnvironmentName + + GitLabWorkshopC9InstanceProfile: + Type: AWS::IAM::InstanceProfile + Properties: + Path: "/" + Roles: + - Ref: GitLabWorkshopC9Role + + GitLabWorkshopC9Instance: + Type: AWS::Cloud9::EnvironmentEC2 + Properties: + Description: AWS Cloud9 instance for GitLab Spot Workshop + AutomaticStopTimeMinutes: 3600 + InstanceType: t2.micro + OwnerArn: !If + - EmptyC9Owner + - !Ref AWS::NoValue + - + !If [IsEERole, !Sub 'arn:aws:sts::${AWS::AccountId}:assumed-role/TeamRole/MasterKey', !Ref EETeamRoleArn] + SubnetId: !Ref Subnet1 + Tags: + - Key: Project + Value: !Ref EnvironmentName + - Key: SSMBootstrap + Value: Active + + GitLabWorkshopC9SSMDocument: + Type: AWS::SSM::Document + Properties: + Content: Yaml + DocumentType: Command + Content: + schemaVersion: '2.2' + description: Bootstrap Cloud9 Instance + mainSteps: + - action: aws:runShellScript + name: GitLabWorkshopC9bootstrap + inputs: + runCommand: + - "#!/bin/bash" + - date + - echo LANG=en_US.utf-8 >> /etc/environment + - echo LC_ALL=en_US.UTF-8 >> /etc/environment + - . /home/ec2-user/.bashrc + - git clone https://github.com/awslabs/ec2-spot-workshops.git /home/ec2-user/environment/spot-workshop + - cp -ar /home/ec2-user/environment/spot-workshop/workshops/amazon-ec2-spot-cicd-workshop/ /home/ec2-user/environment/ + - rm -rf /home/ec2-user/environment/spot-workshop/ + - chown -R ec2-user:ec2-user /home/ec2-user/environment/amazon-ec2-spot-cicd-workshop/ + - echo "Bootstrap completed with return code $?" + Tags: + - Key: Project + Value: !Ref EnvironmentName + + GitLabWorkshopC9OutputBucket: + Type: AWS::S3::Bucket + DeletionPolicy: Delete + + GitLabWorkshopC9BootstrapAssociation: + Type: AWS::SSM::Association + DependsOn: GitLabWorkshopC9OutputBucket + Properties: + Name: !Ref GitLabWorkshopC9SSMDocument + OutputLocation: + S3Location: + OutputS3BucketName: !Ref GitLabWorkshopC9OutputBucket + OutputS3KeyPrefix: bootstrapoutput + Targets: + - Key: tag:SSMBootstrap + Values: + - Active + Outputs: VPC: Description: The VPC with GitLab instance