Merge pull request #107 from connorbrown-db/feature/github-workflows

Feature/GitHub workflows

.github/workflows/README.md

@@ -0,0 +1,167 @@
+# Testing Workflows
+<!-- TOC -->
+* [Testing Workflows](#testing-workflows)
+  * [Workflow features](#workflow-features)
+    * [Caching](#caching)
+    * [Test toggles](#test-toggles)
+  * [Tests](#tests)
+    * [TFLint](#tflint)
+      * [Note on configuring TFLint](#note-on-configuring-tflint)
+    * [Terraform Fmt](#terraform-fmt)
+    * [Terraform test](#terraform-test)
+      * [Basic test case](#basic-test-case)
+  * [Workflow Inputs](#workflow-inputs)
+    * [1. **Actions Settings**](#1-actions-settings)
+    * [2. **TFLint Settings**](#2-tflint-settings)
+    * [3. **Terraform Settings**](#3-terraform-settings)
+    * [4. **Terraform Format (fmt) Settings**](#4-terraform-format-fmt-settings)
+    * [5. **Terraform Test Settings**](#5-terraform-test-settings)
+  * [Usage Example](#usage-example)
+<!-- TOC -->
+
+The terraform-ruw workflow runs three tests:
+
+- `TFLint`
+- `terraform fmt`
+- `terraform test`
+
+It is a reusable workflow, and is called by other workflows via the `workflow_call` feature in GitHub Actions. Each
+Terraform module in this repo has a separate workflow with unique settings for testing. The workflow may be used
+multiple time for one module, with different settings per workflow.
+
+## Workflow features
+
+### Caching
+
+The workflow
+uses [GitHub Actions caching](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/caching-dependencies-to-speed-up-workflows)
+for:
+
+- [Terraform provider caching](https://developer.hashicorp.com/terraform/cli/config/config-file#plugin_cache_dir) to
+  reduce how frequently providers are downloaded on init.
+- tflint plugins
+
+### Test toggles
+
+The workflow supports enabling/disabling test jobs using workflow inputs. For example, the tflint job can be disabled by
+setting the `tflint_enabled` input to `false`.
+
+## Tests
+
+### TFLint
+
+[TFLint](https://github.com/terraform-linters/tflint/) is a linter for Terraform that can search for:
+
+- Resource misconfigurations (e.g. unavailable instance types)
+- Deprecated syntax
+- [Module style](https://developer.hashicorp.com/terraform/language/style) violations
+- Misc. issues per provider [plugin](https://github.com/terraform-linters/tflint/blob/master/docs/user-guide/plugins.md)
+
+#### Note on configuring TFLint
+
+TFLint can be configured using a .tflint.hcl file added to the `working_directory` of the workflow. This file is not
+required, though it is recommended so that provider plugins can be used for deeper inspection of the module. For more
+information on the configuration file,
+see [here](https://github.com/terraform-linters/tflint/blob/master/docs/user-guide/config.md).
+
+> Note that this file only applies to the terraform code that is colocated in the directory the file is in. TFLint is
+> configured to use the `--recursive` option by default, so this means that only the root configuration of a module will
+> use this configuration file by default. To avoid creating multiple .tflint.hcl files throughout a nested module
+> configuration, it is recommended to use the `tflint_args` input of the reusable workflow to add the
+`--config=$(pwd)/.tflint.hcl` argument. This will cause tflint to use the same configuration file as it recurses the
+> working directory tree. See azure-test.yml for an example.
+
+### Terraform Fmt
+
+Terraform fmt is run to check if files are properly formatted in the module. This test is configured with `-write=false`
+and `-check=false` by default, meaning that terraform fmt will not write changes to disk, and it will not fail the
+workflow if it finds that files need to be formatted. If you choose to require that files must be formatted correctly
+for this test to path, you may configure this requirement using the `terraform_fmt_check` input in the reusable
+workflow.
+
+### Terraform test
+
+Terraform test is run in the `working_directory` by default. To learn more about terraform test, see Hashicorp's
+documentation [here](https://developer.hashicorp.com/terraform/language/tests). It is up to each module developer to
+implement terraform tests as they see fit, but some general guidelines are as follows:
+
+- For consistency, tests should be placed in a directory called `tests` in each root module (e.g. `azure/tf/tests`).
+- Provider authentication is not currently supported "out of the box" for this repo. Until this is
+  implemented, [mock providers](https://developer.hashicorp.com/terraform/language/tests) can be used to replicate a
+  working provider.
+
+#### Basic test case
+
+A basic test that can be added to a root module is below. This will execute a terraform plan for a root module using a
+mocked provider (azurerm used as an example).
+
+```hcl
+mock_provider "azurerm" {}
+
+run "plan_test" {
+  command = plan
+}
+```
+
+This test **does not require authentication**, and will run as is. This test should closely replicate what will happen
+on a terraform plan if the provider were authenticated.
+
+## Workflow Inputs
+
+### 1. **Actions Settings**
+
+| Input               | Type   | Required | Default         | Description                                                            |
+|---------------------|--------|----------|-----------------|------------------------------------------------------------------------|
+| `working_directory` | string | Yes      | N/A             | The working directory where the Terraform code is located.             |
+| `environment`       | string | No       | `null`          | GitHub environment to use for the workflow. Can also be used for OIDC. |
+| `runs_on`           | string | No       | `ubuntu-latest` | Sets the runs-on option for all jobs in the workflow.                  |
+
+### 2. **TFLint Settings**
+
+| Input                             | Type    | Required | Default   | Description                                                                       |
+|-----------------------------------|---------|----------|-----------|-----------------------------------------------------------------------------------|
+| `tflint_enabled`                  | boolean | No       | `true`    | Whether to enable TFLint-related jobs.                                            |
+| `tflint_version`                  | string  | No       | `v0.52.0` | The version of TFLint to install.                                                 |
+| `tflint_minimum_failure_severity` | string  | No       | `error`   | The minimum severity required before TFLint considers a rule violation a failure. |
+| `tflint_args`                     | string  | No       | `null`    | Additional arguments to pass to the TFLint command. Example: `"-var 'foo=bar'"`.  |
+
+### 3. **Terraform Settings**
+
+| Input               | Type   | Required | Default | Description                                                                                                   |
+|---------------------|--------|----------|---------|---------------------------------------------------------------------------------------------------------------|
+| `terraform_version` | string | No       | `~>1.0` | The version of Terraform to install for both test and fmt. This supports version constraints (e.g., `~>1.0`). |
+
+### 4. **Terraform Format (fmt) Settings**
+
+| Input                   | Type    | Required | Default | Description                                                                                                      |
+|-------------------------|---------|----------|---------|------------------------------------------------------------------------------------------------------------------|
+| `terraform_fmt_enabled` | boolean | No       | `true`  | Whether to enable Terraform formatting jobs using `terraform fmt`.                                               |
+| `terraform_fmt_check`   | boolean | No       | `false` | Whether a formatting issue should cause the workflow to fail (passed to the `-check` option of `terraform fmt`). |
+
+### 5. **Terraform Test Settings**
+
+| Input                    | Type    | Required | Default | Description                                                                                           |
+|--------------------------|---------|----------|---------|-------------------------------------------------------------------------------------------------------|
+| `terraform_test_enabled` | boolean | No       | `true`  | Whether to enable Terraform testing jobs.                                                             |
+| `terraform_test_args`    | string  | No       | `null`  | Additional arguments to pass to the `terraform test` command (e.g., `-filter=tests/plan.tftest.hcl`). |
+
+## Usage Example
+
+```yaml
+name: Azure Tests
+on:
+  push:
+    paths:
+      - 'azure/tf/**'
+      - '.github/workflows/azure-test.yml'
+  pull_request:
+    paths:
+      - 'azure/tf/**'
+      - '.github/workflows/azure-test.yml'
+jobs:
+  test-azure:
+    uses: ./.github/workflows/terraform-ruw.yml
+    with:
+      working_directory: azure/tf
+      tflint_args: "--config=$(pwd)/.tflint.hcl" #This causes TFLint to reuse the same .tflint.hcl file for every subdirectory
+```

.github/workflows/azure-test.yml

@@ -0,0 +1,18 @@
+# This workflow is used for testing the Azure terraform module.
+
+name: Azure Tests
+on:
+  push:
+    paths:
+      - 'azure/tf/**'
+      - '.github/workflows/azure-test.yml'
+  pull_request:
+    paths:
+      - 'azure/tf/**'
+      - '.github/workflows/azure-test.yml'
+jobs:
+  test-azure:
+    uses: ./.github/workflows/terraform-ruw.yml
+    with:
+      working_directory: azure/tf
+      tflint_args: "--config=$(pwd)/.tflint.hcl"

.github/workflows/terraform-ruw.yml

@@ -0,0 +1,159 @@
+on:
+  # This makes the workflow reusable. It is not meant to be executed independently of a calling workflow.
+  workflow_call:
+    inputs:
+      # Actions settings
+      working_directory:
+        type: string
+        description: Working directory for this workflow
+        required: true
+      environment:
+        # Note: The environment setting will NOT allow secrets from that environment to be used in the RUW. Only
+        # environment variables and deploy rules are useful here. This can also be used for OIDC.
+        type: string
+        default: null
+        required: false
+        description: GitHub environment to use for this workflow
+      runs_on:
+        type: string
+        default: ubuntu-latest
+        required: false
+        description: Sets the runs-on option for all jobs in the workflow
+
+      # TFLint settings
+      tflint_enabled:
+        type: boolean
+        default: true
+        required: false
+        description: Should TFLint related jobs run
+      tflint_version:
+        type: string
+        description: Version of tflint to use
+        required: false
+        default: v0.52.0
+      tflint_minimum_failure_severity:
+        type: string
+        description: Minimum severity required before TFLint considers a rule finding an error
+        required: false
+        default: error
+      tflint_args:
+        type: string
+        description: Additional arguments to pass to the tflint command e.g. "-var 'foo=bar'"
+        required: false
+        default: null
+
+      # Terraform settings
+      terraform_version:
+        type: string
+        default: "~>1.0"
+        required: false
+        description: Version of Terraform to install for jobs that use it. This supports constraint strings also (e.g. ~>1.0)
+
+      # Terraform fmt settings
+      terraform_fmt_enabled:
+        type: boolean
+        default: true
+        required: false
+        description: Should terraform fmt related jobs run?
+      terraform_fmt_check:
+        type: boolean
+        default: false
+        required: false
+        description: Directly passed to the "-check" option for terraform fmt. Should a fmt diff cause the workflow to fail?
+
+      # Terraform test settings
+      terraform_test_enabled:
+        type: boolean
+        default: true
+        required: false
+        description: Should terraform test related jobs run
+      terraform_test_args:
+        type: string
+        default: null
+        required: false
+        description: Additional arguments to pass to the terraform test command e.g. "-var 'foo=bar' -filter=tests/mock_plan.tftest.hcl"
+
+jobs:
+  # TFLint Job
+  tflint:
+    environment: ${{ inputs.environment }}
+    if: inputs.tflint_enabled
+    runs-on: ${{ inputs.runs_on }}
+    steps:
+      - uses: actions/checkout@v4
+        name: Checkout source code
+      # A cache is configured to avoid downloading plugins on every run
+      - uses: actions/cache@v4
+        name: Cache TFLint Plugins
+        with:
+          path: ~/.tflint.d/plugins
+          key: tflint-${{ runner.os }}-${{ hashFiles('**/.tflint.hcl') }}
+      - uses: terraform-linters/setup-tflint@v4
+        name: Setup TFLint
+        with:
+          tflint_version: ${{ inputs.tflint_version }}
+      - name: Init TFLint
+        run: tflint --init
+        env:
+          # https://github.com/terraform-linters/tflint/blob/master/docs/user-guide/plugins.md#avoiding-rate-limiting
+          GITHUB_TOKEN: ${{ github.token }}
+        working-directory: ${{ inputs.working_directory }}
+      - name: Run TFLint
+        run: tflint -f compact --minimum-failure-severity=${{ inputs.tflint_minimum_failure_severity }} --recursive ${{ inputs.tflint_args }}
+        working-directory: ${{ inputs.working_directory }}
+
+  # Terraform fmt job
+  terraform-fmt:
+    environment: ${{ inputs.environment }}
+    runs-on: ${{ inputs.runs_on }}
+    if: inputs.terraform_fmt_enabled
+    steps:
+      - uses: actions/checkout@v4
+        name: Checkout source code
+      - uses: hashicorp/setup-terraform@v3
+        name: Setup Terraform
+        with:
+          terraform_version: inputs.terraform_version
+      - name: terraform fmt
+        run: terraform fmt -check=${{ inputs.terraform_fmt_check }} -write=false -recursive
+        working-directory: ${{ inputs.working_directory }}
+
+  # Terraform test job
+  terraform-test:
+    environment: ${{ inputs.environment }}
+    runs-on: ${{ inputs.runs_on }}
+    if: inputs.terraform_test_enabled
+    # This environment variable sets the plugin cache dir for Terraform, and is also used to configure a cache directory
+    env:
+      TF_PLUGIN_CACHE_DIR:  ${{ github.workspace }}/.terraform.d/plugin-cache
+      # Terraform will not use the cache if the dependency lock file is not committed to the repo. This is a workaround.
+      # https://developer.hashicorp.com/terraform/cli/config/config-file#allowing-the-provider-plugin-cache-to-break-the-dependency-lock-file
+      TF_PLUGIN_CACHE_MAY_BREAK_DEPENDENCY_LOCK_FILE: true
+    steps:
+      - uses: actions/checkout@v4
+        name: Checkout source code
+      # Create the cache directory
+      - run: mkdir -p ${{ env.TF_PLUGIN_CACHE_DIR }}
+      # Initialize the cache
+      - uses: actions/cache@v4
+        name: Cache Terraform Providers
+        with:
+          path: ${{ env.TF_PLUGIN_CACHE_DIR }}
+          # The cache key includes the OS, working directory, and a hash of all versions.tf in the repo.
+          # A change to any of these will cause a new cache to be created (or reused if it exists)
+          key: terraform-providers-${{ runner.os }}-${{ inputs.working_directory }}-${{ hashFiles('**/versions.tf') }}
+          restore-keys:
+            terraform-providers-${{ runner.os }}-${{ inputs.working_directory }}-
+            terraform-providers-${{ runner.os }}-
+      - name: List plugin cache contents
+        run: ls -R ${{ env.TF_PLUGIN_CACHE_DIR }}
+      - uses: hashicorp/setup-terraform@v3
+        name: Setup Terraform
+        with:
+          terraform_version: inputs.terraform_version
+      - name: terraform init
+        run: terraform init
+        working-directory: ${{ inputs.working_directory }}
+      - name: terraform test
+        run: terraform test ${{ inputs.terraform_test_args }}
+        working-directory: ${{ inputs.working_directory }}

.github/workflows/workflow-mock-test.yml

@@ -0,0 +1,19 @@
+# This workflow is used for testing the terraform-ruw reusable workflow.
+
+name: Workflow Mock Tests
+on:
+  push:
+    paths:
+      - '.workflow-mock/**'
+      - '.github/workflows/workflow-mock-test.yml'
+      - '.github/workflows/terraform-ruw.yml'
+  pull_request:
+    paths:
+      - '.workflow-mock/**'
+      - '.github/workflows/workflow-mock-test.yml'
+      - '.github/workflows/terraform-ruw.yml'
+jobs:
+  workflow-mock-test:
+    uses: ./.github/workflows/terraform-ruw.yml
+    with:
+      working_directory: .workflow-mock

.gitignore

@@ -51,3 +51,7 @@ terraform.rc
 .envrc*
 
 *.hcl
+
+# Include tflint configurations for CI workflows
+!**/.tflint.hcl
+!**/*.tftest.hcl

.workflow-mock/.tflint.hcl

@@ -0,0 +1,3 @@
+plugin "terraform" {
+  enabled = true
+}

.workflow-mock/README.md

@@ -0,0 +1,2 @@
+# Mock Directory
+This directory is only for use with testing the GitHub Actions workflow and should otherwise be ignored.

.workflow-mock/main.tf

@@ -0,0 +1,5 @@
+resource "null_resource" "some_resource" {
+  triggers = {
+    example = var.trigger_value
+  }
+}

.workflow-mock/outputs.tf

@@ -0,0 +1,3 @@
+output "test" {
+  value = "test"
+}