separate infrastructure environments

.gitignore

@@ -1,6 +1,7 @@
 *.db
 *.env
 *.mo
+*.tfbackend
 benefits/core/migrations/0002_*.py
 !benefits/core/migrations/0002_sample_data.py
 static/

docs/deployment/infrastructure.md

@@ -59,8 +59,6 @@ flowchart LR
 
 [Front Door](https://docs.microsoft.com/en-us/azure/frontdoor/front-door-overview) also includes the [Web Application Firewall (WAF)](https://docs.microsoft.com/en-us/azure/web-application-firewall/afds/afds-overview) and handles TLS termination. Front Door is managed by the DevSecOps team.
 
-On this page, "slot" will refer to the true [App Service slots](https://docs.microsoft.com/en-us/azure/app-service/deploy-staging-slots) for the different environments, or the overarching App Service resource for `production`. The latter is basically an implicit slot.
-
 ## Ownership
 
 The following things in Azure are managed by the California Department of Technology (CDT)'s DevSecOps (OET) team:
@@ -74,19 +72,22 @@ The following things in Azure are managed by the California Department of Techno
 
 ## Environments
 
-Within the `CDT Digital CA` directory ([how to switch](https://learn.microsoft.com/en-us/azure/devtest/offer/how-to-change-directory-tenants-visual-studio-azure)), there are two subscriptions, with a single [resource group](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-portal) under each:
+Within the `CDT Digital CA` directory ([how to switch](https://learn.microsoft.com/en-us/azure/devtest/offer/how-to-change-directory-tenants-visual-studio-azure)), there are two [Subscriptions](https://learn.microsoft.com/en-us/microsoft-365/enterprise/subscriptions-licenses-accounts-and-tenants-for-microsoft-cloud-offerings?view=o365-worldwide#subscriptions), with Resource Groups under each. Each environment corresponds to a single Resource Group, [Terraform Workspace](https://developer.hashicorp.com/terraform/language/state/workspaces), and branch.
 
-- `CDT/ODI Development` - Meant for experimentation with short-lived resources
-- `CDT/ODI Production` - Used for the `dev`, `test`, and `prod` environments, which are [slots in App Service](#benefits-application)
+| Environment | Subscription          | Resource Group                | Workspace | Branch |
+| ----------- | --------------------- | ----------------------------- | --------- | ------ |
+| Dev         | `CDT/ODI Development` | `RG-CDT-PUB-VIP-CALITP-D-001` | `dev`     | `dev`  |
+| Test        | `CDT/ODI Development` | `RG-CDT-PUB-VIP-CALITP-T-001` | `test`    | `test` |
+| Prod        | `CDT/ODI Production`  | `RG-CDT-PUB-VIP-CALITP-P-001` | `default` | `prod` |
 
-All resources in the `CDT/ODI Production` Resource Group should be reflected in Terraform in this repository. The exceptions are:
+All resources in these Resource Groups should be reflected in Terraform in this repository. The exceptions are:
 
 - Secrets, such as values under [Key Vault](https://azure.microsoft.com/en-us/services/key-vault/) and [App Service application settings](https://docs.microsoft.com/en-us/azure/app-service/configure-common#configure-app-settings). [`prevent_destroy`](https://developer.hashicorp.com/terraform/tutorials/state/resource-lifecycle#prevent-resource-deletion) is used on these Resources.
 - [Things managed by DevSecOps](#ownership)
 
 You'll see these referenced in Terraform as [data sources](https://developer.hashicorp.com/terraform/language/data-sources).
 
-For browsing the [Azure portal](https://portal.azure.com), [switching your `Default subscription filter`](https://docs.microsoft.com/en-us/azure/azure-portal/set-preferences) to only `CDT/ODI Production` is recommended.
+For browsing the [Azure portal](https://portal.azure.com), you can [switch your `Default subscription filter`](https://docs.microsoft.com/en-us/azure/azure-portal/set-preferences).
 
 ## Making changes
 
@@ -105,19 +106,17 @@ Terraform is [`plan`](https://www.terraform.io/cli/commands/plan)'d when code is
    - [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli)
    - [Terraform](https://www.terraform.io/downloads) - see exact version in [`azure-pipelines.yml`](https://github.com/cal-itp/benefits/blob/dev/terraform/azure-pipelines.yml)
 
-1. [Authenticate using the Azure CLI](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/azure_cli), specifying the `CDT/ODI Production` Subscription.
+1. [Authenticate using the Azure CLI](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/azure_cli).
 
    ```sh
    az login
-
-   az account set --subscription="CDT/ODI Production"
    ```
 
 1. Outside the [dev container](../../getting-started/), navigate to the [`terraform/`](https://github.com/cal-itp/benefits/tree/dev/terraform) directory.
-1. [Initialize Terraform.](https://www.terraform.io/cli/commands/init)
+1. [Initialize Terraform.](https://www.terraform.io/cli/commands/init) You can also use this script later to switch between [environments](#environments).
 
    ```sh
-   terraform init
+   ./init.sh <env>
    ```
 
 1. Make changes to Terraform files.

docs/deployment/troubleshooting.md

@@ -2,15 +2,15 @@
 
 ## Monitoring
 
-We have [ping tests](https://docs.microsoft.com/en-us/azure/azure-monitor/app/monitor-web-app-availability) set up to notify about availability of the dev, test, and prod deployments. Alerts go to [#benefits-notify](https://cal-itp.slack.com/archives/C022HHSEE3F).
+We have [ping tests](https://docs.microsoft.com/en-us/azure/azure-monitor/app/monitor-web-app-availability) set up to notify about availability of each [environment](../infrastructure/#environments). Alerts go to [#benefits-notify](https://cal-itp.slack.com/archives/C022HHSEE3F).
 
 ## Logs
 
 Logs can be found a couple of places:
 
 ### Azure App Service Logs
 
-[Open the `Logs` for the slot you are interested in.](https://docs.google.com/document/d/11EPDIROBvg7cRtU2V42c6VBxcW_o8HhcyORALNtL_XY/edit#heading=h.6pxjhslhxwvj) The following tables are likely of interest:
+[Open the `Logs` for the environment you are interested in.](https://docs.google.com/document/d/11EPDIROBvg7cRtU2V42c6VBxcW_o8HhcyORALNtL_XY/edit#heading=h.6pxjhslhxwvj) The following tables are likely of interest:
 
 - `AppServiceConsoleLogs`: `stdout` and `stderr` coming from the container
 - `AppServiceHTTPLogs`: requests coming through App Service
@@ -20,7 +20,7 @@ For some pre-defined queries, click `Queries`, then `Group by: Query type`, and
 
 ### [Azure Monitor Logs](https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-platform-logs)
 
-[Open the `Logs` for the slot you are interested in.](https://docs.google.com/document/d/11EPDIROBvg7cRtU2V42c6VBxcW_o8HhcyORALNtL_XY/edit#heading=h.n0oq4r1jo7zs)
+[Open the `Logs` for the environment you are interested in.](https://docs.google.com/document/d/11EPDIROBvg7cRtU2V42c6VBxcW_o8HhcyORALNtL_XY/edit#heading=h.n0oq4r1jo7zs)
 
 The following [tables](https://docs.microsoft.com/en-us/azure/azure-monitor/app/opencensus-python#telemetry-type-mappings) are likely of interest:
 
@@ -36,11 +36,9 @@ See [`Failures`](https://docs.microsoft.com/en-us/azure/azure-monitor/app/asp-ne
 After [setting up the Azure CLI](#making-changes), you can use the following command to [stream live logs](https://docs.microsoft.com/en-us/azure/app-service/troubleshoot-diagnostic-logs#in-local-terminal):
 
 ```sh
-az webapp log tail --resource-group RG-CDT-PUB-VIP-CALITP-P-001 --name AS-CDT-PUB-VIP-CALITP-P-001 --slot dev 2>&1 | grep -v /healthcheck
+az webapp log tail --resource-group RG-CDT-PUB-VIP-CALITP-P-001 --name AS-CDT-PUB-VIP-CALITP-P-001 2>&1 | grep -v /healthcheck
 ```
 
-`--slot dev` can be removed for production or changed for a different slot.
-
 ### SCM
 
 <https://as-cdt-pub-vip-calitp-p-001-dev.scm.azurewebsites.net/api/logs/docker>

docs/deployment/workflows.md

@@ -29,7 +29,7 @@ Push this image:tag into [GHCR][ghcr].
 
 ### 4. App Service deploy
 
-Each Azure App Service slot is configured to [listen to a webhook from GitHub, then deploy the image][webhook].
+Each Azure App Service instance is configured to [listen to a webhook from GitHub, then deploy the image][webhook].
 
 [deploy]: https://github.com/cal-itp/benefits/blob/dev/.github/workflows/deploy.yml
 [dockerfile]: https://github.com/cal-itp/benefits/blob/dev/Dockerfile

terraform/README.md

@@ -1 +1 @@
-[Documentation](https://docs.calitp.org/benefits/deployment/azure/)
+[Documentation](https://docs.calitp.org/benefits/deployment/infrastructure/)

terraform/app_service.tf

@@ -1,7 +1,7 @@
 resource "azurerm_service_plan" "main" {
-  name                = "ASP-CDT-PUB-VIP-CALITP-P-001"
-  location            = data.azurerm_resource_group.prod.location
-  resource_group_name = data.azurerm_resource_group.prod.name
+  name                = "ASP-CDT-PUB-VIP-CALITP-${local.env_letter}-001"
+  location            = data.azurerm_resource_group.main.location
+  resource_group_name = data.azurerm_resource_group.main.name
   os_type             = "Linux"
   sku_name            = "P2v2"
 
@@ -10,24 +10,20 @@ resource "azurerm_service_plan" "main" {
   }
 }
 
-# app_settings are managed manually through the portal since they contain secrets
-
-## PROD ##
-
 resource "azurerm_linux_web_app" "main" {
-  name                      = "AS-CDT-PUB-VIP-CALITP-P-001"
-  location                  = data.azurerm_resource_group.prod.location
-  resource_group_name       = data.azurerm_resource_group.prod.name
+  name                      = "AS-CDT-PUB-VIP-CALITP-${local.env_letter}-001"
+  location                  = data.azurerm_resource_group.main.location
+  resource_group_name       = data.azurerm_resource_group.main.name
   service_plan_id           = azurerm_service_plan.main.id
   https_only                = true
-  virtual_network_subnet_id = data.azurerm_subnet.main.id
+  virtual_network_subnet_id = local.subnet_id
 
   site_config {
     ftps_state             = "Disabled"
     vnet_route_all_enabled = true
     application_stack {
       docker_image     = "ghcr.io/cal-itp/benefits"
-      docker_image_tag = "prod"
+      docker_image_tag = local.env_name
     }
   }
 
@@ -48,186 +44,31 @@ resource "azurerm_linux_web_app" "main" {
     }
   }
 
-  # Confusingly named argument; these are settings / environment variables that should be unique to each slot. Also known as "deployment slot settings".
-  # https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/linux_web_app#app_setting_names
-  # https://docs.microsoft.com/en-us/azure/app-service/deploy-staging-slots#which-settings-are-swapped
-  sticky_settings {
-    # sort them so that they don't change when we rearrange them
-    app_setting_names = sort([
-      # custom config
-      "ANALYTICS_KEY",
-      "DJANGO_ALLOWED_HOSTS",
-      "DJANGO_DEBUG",
-      "DJANGO_LOAD_SAMPLE_DATA",
-      "DJANGO_LOG_LEVEL",
-      "DJANGO_MIGRATIONS_DIR",
-      "DJANGO_RATE_LIMIT",
-      "DJANGO_RATE_LIMIT_METHODS",
-      "DJANGO_RATE_LIMIT_PERIOD",
-      "DJANGO_RECAPTCHA_SECRET_KEY",
-      "DJANGO_RECAPTCHA_SITE_KEY",
-      "DJANGO_TRUSTED_ORIGINS",
-
-      # populated through auto-instrumentation
-      # https://docs.microsoft.com/en-us/azure/azure-monitor/app/azure-web-apps#enable-application-insights
-      "APPINSIGHTS_INSTRUMENTATIONKEY",
-      "APPINSIGHTS_PROFILERFEATURE_VERSION",
-      "APPINSIGHTS_SNAPSHOTFEATURE_VERSION",
-      "APPLICATIONINSIGHTS_CONFIGURATION_CONTENT",
-      "APPLICATIONINSIGHTS_CONNECTION_STRING",
-      "ApplicationInsightsAgent_EXTENSION_VERSION",
-      "DiagnosticServices_EXTENSION_VERSION",
-      "InstrumentationEngine_EXTENSION_VERSION",
-      "SnapshotDebugger_EXTENSION_VERSION",
-      "XDT_MicrosoftApplicationInsights_BaseExtensions",
-      "XDT_MicrosoftApplicationInsights_Mode",
-      "XDT_MicrosoftApplicationInsights_NodeJS",
-      "XDT_MicrosoftApplicationInsights_PreemptSdk",
-      "XDT_MicrosoftApplicationInsightsJava",
-    ])
-  }
-
   storage_account {
     access_key   = azurerm_storage_account.main.primary_access_key
     account_name = azurerm_storage_account.main.name
     name         = "benefits-config"
     type         = "AzureBlob"
-    share_name   = azurerm_storage_container.config_prod.name
+    share_name   = azurerm_storage_container.config.name
     mount_path   = "/home/calitp/app/config"
   }
 
   lifecycle {
     prevent_destroy = true
-    ignore_changes  = [app_settings, tags]
+    # app_settings are managed manually through the portal since they contain secrets
+    ignore_changes = [app_settings, sticky_settings, tags]
   }
 }
 
-resource "azurerm_app_service_custom_hostname_binding" "prod" {
-  hostname            = "benefits.calitp.org"
+resource "azurerm_app_service_custom_hostname_binding" "main" {
+  hostname            = local.hostname
   app_service_name    = azurerm_linux_web_app.main.name
-  resource_group_name = data.azurerm_resource_group.prod.name
-
-  # Ignore ssl_state and thumbprint as they are managed externally
-  lifecycle {
-    ignore_changes = [ssl_state, thumbprint]
-  }
-}
-
-## DEV ##
-
-resource "azurerm_linux_web_app_slot" "dev" {
-  name                      = "dev"
-  https_only                = true
-  app_service_id            = azurerm_linux_web_app.main.id
-  virtual_network_subnet_id = data.azurerm_subnet.main.id
-
-  site_config {
-    ftps_state             = "Disabled"
-    vnet_route_all_enabled = true
-    application_stack {
-      docker_image     = "ghcr.io/cal-itp/benefits"
-      docker_image_tag = "dev"
-    }
-  }
-
-  identity {
-    identity_ids = []
-    type         = "SystemAssigned"
-  }
-
-  logs {
-    detailed_error_messages = false
-    failed_request_tracing  = false
-
-    http_logs {
-      file_system {
-        retention_in_days = 99999
-        retention_in_mb   = 100
-      }
-    }
-  }
-
-  lifecycle {
-    prevent_destroy = true
-    ignore_changes  = [app_settings, tags]
-  }
-
-  # setup files
-  storage_account {
-    access_key   = azurerm_storage_account.main.primary_access_key
-    account_name = azurerm_storage_account.main.name
-    # use the same name
-    name       = azurerm_storage_container.config_dev.name
-    type       = "AzureBlob"
-    share_name = azurerm_storage_container.config_dev.name
-    mount_path = "/home/calitp/app/config"
-  }
-}
-
-resource "azurerm_app_service_slot_custom_hostname_binding" "dev" {
-  app_service_slot_id = azurerm_linux_web_app_slot.dev.id
-  hostname            = "dev-benefits.calitp.org"
-}
-
-## TEST ##
-
-resource "azurerm_linux_web_app_slot" "test" {
-  name                      = "test"
-  https_only                = true
-  app_service_id            = azurerm_linux_web_app.main.id
-  virtual_network_subnet_id = data.azurerm_subnet.main.id
-
-  site_config {
-    ftps_state             = "Disabled"
-    vnet_route_all_enabled = true
-    application_stack {
-      docker_image     = "ghcr.io/cal-itp/benefits"
-      docker_image_tag = "test"
-    }
-  }
-
-  identity {
-    identity_ids = []
-    type         = "SystemAssigned"
-  }
-
-  logs {
-    detailed_error_messages = false
-    failed_request_tracing  = false
-
-    http_logs {
-      file_system {
-        retention_in_days = 99999
-        retention_in_mb   = 100
-      }
-    }
-  }
-
-  lifecycle {
-    prevent_destroy = true
-    ignore_changes  = [app_settings, tags]
-  }
-
-  # setup files
-  storage_account {
-    access_key   = azurerm_storage_account.main.primary_access_key
-    account_name = azurerm_storage_account.main.name
-    # use the same name
-    name       = azurerm_storage_container.config_test.name
-    type       = "AzureBlob"
-    share_name = azurerm_storage_container.config_test.name
-    mount_path = "/home/calitp/app/config"
-  }
-}
-
-resource "azurerm_app_service_slot_custom_hostname_binding" "test" {
-  app_service_slot_id = azurerm_linux_web_app_slot.test.id
-  hostname            = "test-benefits.calitp.org"
+  resource_group_name = data.azurerm_resource_group.main.name
 }
 
 # migrations
 
 moved {
-  from = azurerm_app_service_custom_hostname_binding.main
-  to   = azurerm_app_service_custom_hostname_binding.prod
+  from = azurerm_app_service_custom_hostname_binding.prod
+  to   = azurerm_app_service_custom_hostname_binding.main
 }