Skip to content

Commit

Permalink
Merge pull request #1 from matthewmc1/storage
Browse files Browse the repository at this point in the history
Storage
  • Loading branch information
matthewmc1 authored Jan 27, 2022
2 parents 58bde51 + 5c6743b commit 9de068f
Show file tree
Hide file tree
Showing 21 changed files with 104 additions and 76 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,4 @@ override.tf.json
# !example_override.tf

# Include tfplan files to ignore the plan output of command: terraform plan -out=tfplan
# example: *tfplan*
*tfplan*
13 changes: 0 additions & 13 deletions AWS/1-Configure-Credentials-To-Access-AWS.md

This file was deleted.

19 changes: 19 additions & 0 deletions GCP/1-Configure-Credentials-To-Access-GCP.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Configure Credentials To Access GCP At The Programmatic Level


The purpse of this lab is to configure IAM credentials on your local computer so that you can access GCP at a programmatic level (SDKs, CLI, Terraform, etc.)

## Install gcloud CLI
1. [Cloud SDK](https://cloud.google.com/sdk/docs/install)

## Billing Account

You should have a billing account assosicated to your account prior to starting this, if you have never used GCP before this will also entitle you to credits on sign-up to use this project but make sure to destroy after so that you are not charged for this.

## Login & Create Project
1. Running locally to create, first run `gcloud auth application-default login` - this will login using the Google sign-in option.
2. Set your default project for running `gcloud projects create devops-the-hardway`
3. Confirm project is created `gcloud projects list`
4. Link billing account to the project `gcloud beta billing projects link devops-the-hardway --billing-account {BILLING-ID}`
4. Set default project `gcloud config set project devops-the-hardway`

52 changes: 26 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# DevOps-The-Hard-Way-AWS
# DevOps-The-Hard-Way-GCP

This tutorial contains a full, real-world solution for setting up an environment that is using DevOps technologies and practices for deploying apps and cloud services/cloud infrastructure to AWS.
This tutorial contains a full, real-world solution for setting up an environment that is using DevOps technologies and practices for deploying apps and cloud services/cloud infrastructure to GCP.


The repository contains free labs, documentation, diagrams, and docs for setting up an entire workflow and DevOps environment from a real-world perspective in AWS.
The repository contains free labs, documentation, diagrams, and docs for setting up an entire workflow and DevOps environment from a real-world perspective in GCP.

## DevOps Scenario
The scenario that you're currently facing is you work in an organization that is very monolithic. There is a ton of bare metal, virtualization, manual deployments of applications, and **old school** practices based on the current teams knowledge of IT.
Expand All @@ -24,44 +24,44 @@ https://github.com/AdminTurnedDevOps/Python-Sample-Application
## Technology Details
You will be using the following technologies and platforms to set up a DevOps environment.

1. AWS
- AWS will be used to host the application, cloud infrastructure, and any other services we may need to ensure the Uber app is deployed properly.
1. GCP
- GCP will be used to host the application, cloud infrastructure, and any other services we may need to ensure the Uber app is deployed properly.
2. GitHub
- To store the application and infrastructure/automation code
3. Python
- Python will be used for the Uber app (it is written in Python) and some automation efforts that aren't in Terraform.
4. Terraform
- Create an S3 bucket to store Terraform State files
- Create an AWS ECR repository with Terraform
- Create an EKS cluster
- Create an GCS bucket to store Terraform State files
- Create an GCP ECR repository with Terraform
- Create an GKE cluster
5. Docker
- Create a Docker image
- Store the Docker image in AWS ECR
- Store the Docker image in GCP Artifact Registry
6. Kubernetes
- To run the Docker image that's created for the containerized Uber app. Kubernetes, in this case, EKS, will be used to orchestrate the container.
7. CI/CD
- Use GitHub Actions to create an EKS cluster
- Use GitHub Actions to create an GKE Cluster
8. Automated testing
- Testing Terraform code with Checkov

## Labs
1. [Prerequisites](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-AWS/blob/main/prerequisites.md)
1. [Prerequisites](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-GCP/blob/main/prerequisites.md)
2. VPC - When running EKS, it requires specific networking. Because all environments will most likely be different, there's a CloudFormation template for this exact purpose.
- [Create EKS VPC](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-AWS/tree/main/EKS-VPC-CloudFormation)
3. AWS:
- [Configure credentials to access AWS at a programmatic level](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-AWS/blob/main/AWS/1-Configure-Credentials-To-Access-AWS.md)
4. Terraform - The purpose of the Terraform section is to create all of the AWS cloud services you'll need from an environment/infrastructure perspective to run the Uber application.
- [Create S3 Bucket To Store TFSTATE Files](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-AWS/blob/main/Terraform-AWS-Services-Creation/1-Create-S3-Bucket-To-Store-TFSTATE-Files.md)
- [Create an Elastic Container Registry](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-AWS/blob/main/Terraform-AWS-Services-Creation/2-Create-ECR.md)
- [Create An EKS Cluster IAM Role, And Policy For EKS](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-AWS/blob/main/Terraform-AWS-Services-Creation/3-Create-EKS-Cluster-IAM-Role-And-Policy.md)
- [Create An EKS Cluster](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-AWS/blob/main/Terraform-AWS-Services-Creation/3-Create-EKS-Cluster-IAM-Role-And-Policy.md)
5. Docker - The purpose of the Docker section is to create a Docker image from the app that the organization is running on-prem (the uber app), containerize it, and store the container inside of a container repository. For the container repo, you'll use AWS ECR.
- [Create The Docker Image](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-AWS/blob/main/Docker/1-Create-Docker-Image.md)
- [Log Into AWS ECR Repository](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-AWS/blob/main/Docker/Push%20Image%20To%20ECR.md)
- [Create VPC](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-GCP/tree/main/VPC)
3. GCP:
- [Configure credentials to access GCP at a programmatic level](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-GCP/blob/main/GCP/1-Configure-Credentials-To-Access-GCP.md)
4. Terraform - The purpose of the Terraform section is to create all of the GCP cloud services you'll need from an environment/infrastructure perspective to run the Uber application.
- [Create S3 Bucket To Store TFSTATE Files](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-GCP/blob/main/Terraform-GCP-Services-Creation/1-Create-S3-Bucket-To-Store-TFSTATE-Files.md)
- [Create an Elastic Container Registry](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-GCP/blob/main/Terraform-GCP-Services-Creation/2-Create-ECR.md)
- [Create An EKS Cluster IAM Role, And Policy For EKS](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-GCP/blob/main/Terraform-GCP-Services-Creation/3-Create-EKS-Cluster-IAM-Role-And-Policy.md)
- [Create An EKS Cluster](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-GCP/blob/main/Terraform-GCP-Services-Creation/3-Create-EKS-Cluster-IAM-Role-And-Policy.md)
5. Docker - The purpose of the Docker section is to create a Docker image from the app that the organization is running on-prem (the uber app), containerize it, and store the container inside of a container repository. For the container repo, you'll use GCP ECR.
- [Create The Docker Image](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-GCP/blob/main/Docker/1-Create-Docker-Image.md)
- [Log Into GCP ECR Repository](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-GCP/blob/main/Docker/Push%20Image%20To%20ECR.md)
6. Kubernetes - The purpose of the Kubernetes section is to connect to EKS locally and to write the Kubernetes manifest to deploy the Python Uber app.
- [Connect To EKS From The Terminal](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-AWS/blob/main/kubernetes_manifest/1-Connect-To-EKS.md)
- [Create A Kubernetes Manifest](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-AWS/blob/main/kubernetes_manifest/2-Create-Kubernetes-Manifest.md)
- [Connect To EKS From The Terminal](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-GCP/blob/main/kubernetes_manifest/1-Connect-To-EKS.md)
- [Create A Kubernetes Manifest](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-GCP/blob/main/kubernetes_manifest/2-Create-Kubernetes-Manifest.md)
7. Automated Testing - The purpose of the Automation Testing section is to ensure that all of the Terraform code is performing as it should be from a policy, security, and static code analysis perspective.
- [Install And Run Checkov](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-AWS/blob/main/Terraform-Static-Code-Analysis/1-Checkov-For-Terraform.md)
- [Install And Run Checkov](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-GCP/blob/main/Terraform-Static-Code-Analysis/1-Checkov-For-Terraform.md)
8. CICD - The purpose of this section is to automatically create an EKS cluster with CICD using GitHub Actions
- [Create a GitHub Actions CICD pipeline](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-AWS/blob/main/Terraform-AWS-Services-Creation/4-Run-CICD-For-EKS-Cluster.md)
- [Create a GitHub Actions CICD pipeline](https://github.com/AdminTurnedDevOps/DevOps-The-Hard-Way-GCP/blob/main/Terraform-GCP-Services-Creation/4-Run-CICD-For-EKS-Cluster.md)

This file was deleted.

1 change: 0 additions & 1 deletion Terraform-AWS-Services-Creation/ECR/terraform.tfvars

This file was deleted.

18 changes: 0 additions & 18 deletions Terraform-AWS-Services-Creation/terraform-state-s3-bucket/main.tf

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Create an Google Cloud Storage bucket to store Terraform state files

In this lab you will create an Google Cloud Storage bucket that will be used to store Terraform state files

## Create The Terraform Configurations

1. You can find the Terraform configuration for the Google Cloud Storage bucket [here](https://github.com/mmcgibbon1/DevOps-The-Hard-Way-GCP/tree/trunk/Terraform-GCP-Services-Creation/terraform-state-gcs-bucket). The Terraform configuration files are used to create an Google Cloud Storage bucket that will store your TFSTATE.

The Terraform `main.tf` will do a few things:
- Create the Google Cloud Storage bucket in the `EU` region for regional availability
- Ensure that version enabling is set to `True`


2. Create the bucket by running the following:
- `terraform init` - To initialize the working directory and pull down the provider
- `terraform plan -out gcs.tfplan` - To go through a "check" and confirm the configurations are valid and create a plan file based on the name provided.
- `terraform apply gcs.tfplan` - To create the resource

3. Sample output from `terraform plan -out gcs.tfplan`

```
# google_storage_bucket.terraform_state will be created
+ resource "google_storage_bucket" "terraform_state" {
+ force_destroy = false
+ id = (known after apply)
+ location = "EU"
+ name = "terraform-state-devopsthehardway-gcp"
+ project = (known after apply)
+ self_link = (known after apply)
+ storage_class = "STANDARD"
+ uniform_bucket_level_access = true
+ url = (known after apply)
+ versioning {
+ enabled = true
}
}
```
File renamed without changes.
1 change: 1 addition & 0 deletions Terraform-GCP-Services-Creation/ECR/terraform.tfvars
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
repo_name = "devopsthehardway-gcp"
13 changes: 13 additions & 0 deletions Terraform-GCP-Services-Creation/terraform-state-gcs-bucket/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
resource "google_storage_bucket" "terraform_state" {
name = "terraform-state-devopsthehardway-gcp"
location = "EU"

versioning {
enabled = true
}

project = var.project_id

uniform_bucket_level_access = true
}

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
project_id = "devops-the-hardway"
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
variable project_id {
type = string
default = "devops-the-hardway"
description = "Default Project"
}

0 comments on commit 9de068f

Please sign in to comment.