Skip to content

Commit

Permalink
Changes to the workshop structure based on the initial review
Browse files Browse the repository at this point in the history
  • Loading branch information
mikegolubev committed Aug 30, 2022
1 parent dbc111a commit 3f14050
Show file tree
Hide file tree
Showing 37 changed files with 591 additions and 573 deletions.
23 changes: 16 additions & 7 deletions content/amazon-ec2-spot-cicd-workshop/gitlab-spot/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,22 @@ You can perform all workshop steps one-by-one to get to the expected results, bu
Many workshop steps imply manual actions in the AWS console to better demonstrate the underlying concepts, but in a Production environment it is better to automate them using Infrastructure as Code (IaC), such as [AWS CloudFormation](https://aws.amazon.com/cloudformation/) and [AWS CDK](https://docs.aws.amazon.com/cdk/v2/guide/home.html).

## Workshop Details
This workshop will be broken down into a series of labs that flow on from each other (that is, you must complete each lab in order before proceeding with the next). The lab exercises that will be covered are:
This workshop will be broken down into a series of sections that flow on from each other (that is, you must complete each section before proceeding with the next). The whole flow looks as following:

![GitLab on Spot workshop flow](/images/gitlab-spot/lab-flow.png)


The exercises that will be covered are:

* [Starting the workshop](gitlab-spot/before.html) where you will log in to AWS accounts and deploy GitLab, if it is not yet deployed
* [Workshop Preparation](gitlab-spot/prep.html) where you will save GitLab access details and create an AWS Cloud9 environment to execute the workshop steps
* [Lab 1: Create a GitLab repository](gitlab-spot/lab1.html) where you will create a repository in GitLab CI/CD and create a demo application.
* [Lab 2: Configure GitLab runners on Spot instances](gitlab-spot/lab2.html) where you will deploy the GitLab Runners in an auto-scaling group on spot instances
* [Lab 3: Building the demo app](gitlab-spot/lab3.html) where you will push the changes and make sure that your pipeline executes successfully.
* [Lab 4: Deploying Amazon EKS on Spot instances](gitlab-spot/lab4.html) where you will create a new Kubernetes cluster in Amazon EKS that will only have worker nodes on spot instances
* [Lab 5: Installing the demo app into Amazon EKS](gitlab-spot/lab5.html) where you will modify your GitLab CI/CD scripts to add a stage of deploying on Amazon EKS and test the result
* [Workshop Cleanup](gitlab-spot/cleanup.html) where you will remove all the resources created during the workshop
* [Create a GitLab repository](gitlab-spot/lab1.html) where you will create a repository in GitLab CI/CD and create a demo application.
* [Configure GitLab runners on Spot instances](gitlab-spot/lab2.html) where you will deploy the GitLab Runners in an auto-scaling group on spot instances
* [Building the demo app](gitlab-spot/lab3.html) where you will push the changes and make sure that your pipeline executes successfully.
* [Deploying Amazon EKS on Spot instances](gitlab-spot/lab4.html) where you will create a new Kubernetes cluster in Amazon EKS that will only have worker nodes on spot instances
* [Installing the demo app into Amazon EKS](gitlab-spot/lab5.html) where you will modify your GitLab CI/CD scripts to add a stage of deploying on Amazon EKS and test the result
* [Workshop Cleanup](gitlab-spot/cleanup.html) where you will remove all the resources created during the workshop

The final architecture that we will be building looks the following way:

![GitLab on Spot workshop architecture diagram](/images/gitlab-spot/gitlab-spot-architecture.png)
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ If you are at an AWS event, an AWS account created for you to use throughout the
### Get the SSH key and log in to AWS Console

1. In the Team Dashboard screen, choose **SSH Key**.
2. In the popup page, choose **Download Key**. You will only need it if you decide to do an optional task of configuring runners using Docker Machine (this had been the only way before Auto-Scaling Groups support was added).
2. In the popup page, choose **Download Key**. You don't need the key to complete the labs, but might want to still have it if you decide to explore the environment.

![Event Engine Screenshot: SSH Key](/images/gitlab-spot/EE-SSHKey.png)

Expand All @@ -32,15 +32,15 @@ You are now logged in to the AWS console in an account that was created for you,

### Open the pre-provisioned CloudFormation stack

In the next section we will get the login details of the GitLab environment that was pre-provisioned for you via AWS CloudFormation. First, you need to find and open the CloudFormation stack: it is the only one in the account.
In the next section we will get the login details of the GitLab environment that was pre-provisioned for you via AWS CloudFormation. First, you need to find and open the CloudFormation stack: it is the oldest one in the account.

{{%expand "Click to reveal detailed instructions" %}}
1. In the AWS Console enter **CloudFormation** in the search box at the top of the screen and open the service:

![AWS Console Screenshot: Search for CloudFormation](/images/gitlab-spot/AWSConsole-CloudFormationSearch.png)

2. In the navigation pane on the left choose **Stacks**.
3. You should see exactly one stack in the list (with the name like `mod-...`), click on it.
3. You should see two stacks in the list: the one for AWS Cloud9 environment starting with `aws-cloud9-...` and the main one starting with `mod-...`. You will need the main stack (latter one), click on it.
{{% /expand%}}

If there is no CloudFormation stack present, provision it as specified in the section [**...On your own**](self_paced.html).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Your account must have the ability to create new IAM roles and scope other IAM p

### Create an SSH key

You will need an SSH key if you decide to do an optional task of configuring runners using Docker Machine (this had been the only way before Auto-Scaling Groups support was added). The following steps show how to create it.
You don't need the key to complete the labs, but it is still configured when creating the instances, because you might want to explore the environment and log in to the provisioned instances. The following steps show how to create it.

1. Open **EC2** service in the AWS Console.
2. In the navigation pane choose **Key Pairs** in the **Network & Security** section.
Expand All @@ -37,7 +37,7 @@ Now you will deploy a GitLab instance without any runners. As it is not the purp
2. In the navigation pane choose **Stacks**.
3. Choose **Create stack** and in the dropdown choose **With new resources (standard)**.
4. In the **Template source** field select **Upload a template file**, choose the file you saved in the [**Preparation**](#preparation) section above, and choose **Next**.
5. In the **Stack name** field enter `mod-gitlab-spot-workshop`, in the **SSHKeyName** field select `ee-default-key-pair` or the name of the key you used in the steps above, and choose **Next**.
5. In the **Stack name** field enter `mod-gitlab-spot-workshop`, in the **EEKeyPair** field select `ee-default-key-pair` or the name of the key you used in the steps above. Leave the default values in other fields and choose **Next**.
6. Choose **Next**.
7. Mark the checkbox **I acknowledge that AWS CloudFormation might create IAM resources.** and choose **Create stack**.
8. Wait until the stack is in `CREATE_COMPLETE` status (it should take approximately 15 minutes) and continue with [**Workshop Preparation**](/amazon-ec2-spot-cicd-workshop/gitlab-spot/prep.html).
12 changes: 5 additions & 7 deletions content/amazon-ec2-spot-cicd-workshop/gitlab-spot/cleanup.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,11 @@ export TF_VAR_alb_policy=$(aws iam get-policy --policy-arn arn:aws:iam::$(aws st
terraform destroy
```

3. When asked if you want to destroy all resources, type `yes` and press Enter. After that, you can close your Cloud9 environment.
4. Depending on how you deployed the runners, do one of the following:
- If you deployed runners using Auto-Scaling Group, in CloudFormation console delete the stack `linux-docker-scaling-spotonly` (the NESTED stack will be removed automatically).
- If you performed the optional steps to deploy runners using Docker Machine, terminate all runner instances in the EC2 console (name starts with `runner-`) if they have not been yet terminated automatically. Also, terminate the instace `GitLabRunnerManager`. Finally, in the EC2 console choose **Security Groups** in the navigation pane and delete the security groups `GitLabRunner` first and then `GitLabRunnerManager`.
5. In Cloud9 console remove the environment you created.
6. In the IAM console remove all roles you created (`gitlab-spot-workshop-admin` and `GitLabRunner`) and all policies (`EKS-ReadAll`, `AWSLoadBalancerControllerIAMPolicy`, unless you had it before the workshop, and, if you did the Docker Run lab, `IAM-PassRole`).
7. In the ECR console open the repository and remove all images inside it (you do not need to remove the repository itself: it will be done automatically when removing the GitLab stack).
3. When asked if you want to destroy all resources, type `yes` and press Enter. Wait until the process is finished and close your Cloud9 environment.
4. In CloudFormation console delete the stack `linux-docker-scaling-spotonly`.
5. In the IAM console remove the role `GitLabRunner` and the policies `GitLabRunnerPolicy` and `AWSLoadBalancerControllerIAMPolicy`, unless you had it before the workshop.
6. In the ECR console open the repository and remove all images inside it (you do not need to remove the repository itself: it will be done automatically when removing the GitLab stack).
7. In the S3 console find a bucket with `gitlabworkshopc9outputbucket` in its name and remove all objects inside it.
8. If you created the GitLab stack in CloudFormation yourself, remove it too (if you used the one created automatically, you will not be able to delete it, so you can leave it as is).

### Thank you
Expand Down
14 changes: 8 additions & 6 deletions content/amazon-ec2-spot-cicd-workshop/gitlab-spot/lab1.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
+++
title = "Lab 1: Create a GitLab repository"
title = "Create a GitLab repository"
weight = 30
+++
In this lab you will create a new repository in GitLab and configure it in AWS Cloud9. Then you will create the source code of the demo app and commit it into the repository, but before you configure the actual CI/CD in the next lab, will not push it.
Expand Down Expand Up @@ -34,16 +34,18 @@ You will now initialize Git repository in the demo app and add your GitLab envir
cd ~/environment/amazon-ec2-spot-cicd-workshop/gitlab-spot/demo-app/
```

2. Execute the following command to substitute the `ECR_ADDRESS` placeholder with the actual address of ECR repository created in the CloudFormation stack and save it into `.gitlab-ci.yml` file as expected by GitLab CI/CD:
2. Execute the following command to substitute the `ECR_ADDRESS` and other placeholders with their actual values (like the address of ECR repository created in the CloudFormation stack) and save it into `.gitlab-ci.yml` file as expected by GitLab CI/CD:

```
TOKEN=$(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 600")
REGION=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/placement/region)
export ECR_ADDRESS=$(aws ecr describe-repositories --repository-names gitlab-spot-demo --region $REGION --query repositories[0].repositoryUri --output text | awk -F'/' '{print $1}')
sed "s/\${ECR_ADDRESS}/${ECR_ADDRESS}/g" template-gitlab-ci.yml | sed "s/\${AWS_REGION}/${REGION}/g" > .gitlab-ci.yml
export ECR_INFO=$(aws ecr describe-repositories --region $REGION --query "repositories[?contains(repositoryName, 'gitlab-spot-demo')].repositoryUri" --output text)
export ECR_ADDRESS=$(echo $ECR_INFO | awk -F'/' '{print $1}')
export ECR_NAME=$(echo $ECR_INFO | awk -F'/' '{print $2}')
sed "s/\${ECR_ADDRESS}/${ECR_ADDRESS}/g" template-gitlab-ci.yml | sed "s/\${ECR_NAME}/${ECR_NAME}/g" | sed "s/\${AWS_REGION}/${REGION}/g" > .gitlab-ci.yml
```

3. In the file tree on the left open file `gitlab-spot-workshop/amazon-ec2-spot-cicd-workshop/gitlab-spot/demo-app/.gitlab-ci.yml` (if you don't see it, make sure you have enabled the hidden files in [**Workshop Preparation**](prep.html)). Look through it to understand what it does.
3. In the file tree on the left open file `amazon-ec2-spot-cicd-workshop/gitlab-spot/demo-app/.gitlab-ci.yml` (if you don't see it, make sure you have enabled the hidden files in [**Workshop Preparation**](prep.html)). Look through it to understand what it does.
4. Define your name and email that will be used in Git (replace `Your Name` and `[email protected]` with the values you prefer):

```
Expand All @@ -66,4 +68,4 @@ git add .
git commit -m "Initial commit"
```

You are now ready to do the key step in configuring GitLab CI/CD on Spot instances: add the runners. Please proceed to [**Lab 2: Configure GitLab runners on Spot instances**](lab2.html).
You are now ready to do the key step in configuring GitLab CI/CD on Spot instances: add the runners. Please proceed to [**Configure GitLab runners on Spot instances**](lab2.html).
Loading

0 comments on commit 3f14050

Please sign in to comment.