Deploy your instance of Cromwell on Azure

Deploy your instance of Cromwell on Azure

Please see the deployer command-line options guide.

Prerequisites

1. You will need an Azure Subscription to deploy Cromwell on Azure.
2. You must have the proper Azure role assignments to deploy Cromwell on Azure. To check your current role assignments, please follow these instructions. You must have one of the following combinations of role assignments:
   1. Owner of the subscription
      
   2. Contributor and User Access Administrator of the subscription
   3. Owner of the resource group. Note: you must specify the resource group name during deployment with this level of access.
   4. Note: if you only have Service Administrator as a role assignment, please assign yourself as Owner of the subscription.
3. Install the Azure Command Line Interface (az cli), a command line experience for managing Azure resources.
4. Install Helm, a package manager for Kubernetes
5. Run az login to authenticate with Azure.

Download the deployment executable

Download the required executable from Releases. Choose the runtime of your choice from win-x64, linux-x64, osx-x64. On Windows machines, we recommend using the win-x64 runtime (deployment using the linux-x64 runtime via the Windows Subsystem for Linux is not supported).

Run the deployment executable

1. Linux and OS X only: assign execute permissions to the file by running the following command on the terminal:
   chmod +x <fileName>. Replace <fileName> with the correct name: deploy-cromwell-on-azure-linux or deploy-cromwell-on-azure-osx.app
2. You must specify the following parameters:
   1. SubscriptionId (required)
      1. This can be obtained by navigating to the subscriptions blade in the Azure portal
   2. RegionName (required)
      1. Specifies the region you would like to use for your Cromwell on Azure instance. To find a list of all available regions, run az account list-locations on the command line or in PowerShell and use the desired region's "name" property for RegionName.
   3. HelmBinaryPath (required)
      1. Location of the Helm binary path on your machine (e.g. /usr/local/bin/helm or C:\ProgramData\chocolatey\bin\helm.exe).
   4. MainIdentifierPrefix (optional)
      1. This string will be used to prefix the name of your Cromwell on Azure resource group and associated resources. If not specified, the default value of "coa" followed by random characters is used as a prefix for the resource group and all Azure resources created for your Cromwell on Azure instance. After installation, you can search for your resources using the MainIdentifierPrefix value.
         
   5. ResourceGroupName (optional, required when you only have owner-level access of the resource group)
      1. Specifies the name of a pre-existing resource group that you wish to deploy into.
   6. Environment variable named GITHUB_TOKEN (optional)
      1. This reduces the chance you are rate limited by GitHub. Create a PAT and save it in the environment variable.

Run the following at the command line or terminal after navigating to where your executable is saved:

deploy-cromwell-on-azure-win.exe --SubscriptionId <Your subscription ID> --RegionName <Your region> --HelmBinaryPath <Helm binary path> --MainIdentifierPrefix <Your string> 

Example (Linux):

deploy-cromwell-on-azure-linux --SubscriptionId 00000000-0000-0000-0000-000000000000 --HelmBinaryPath /usr/local/bin/helm --RegionName eastus --MainIdentifierPrefix coa 

Example (Windows):

deploy-cromwell-on-azure-win.exe --SubscriptionId 00000000-0000-0000-0000-000000000000 --HelmBinaryPath C:\ProgramData\chocolatey\bin\helm.exe --RegionName eastus --MainIdentifierPrefix coa 

A test workflow is run to ensure successful deployment. If your Batch account does not have enough resource quotas, you will see the error while deploying. You can request more quotas by following these instructions.

Deployment, including a small test workflow can take up to 25 minutes to complete.

Prepare, start or abort a workflow using instructions here.

Run the deployer in update mode

First, download the latest binary deployer from the releases page. You can then update a Cromwell on Azure instance using the --update true flag. As of version 5.1.0 you no longer need the aksclustername (see below if you are still on 4.x.x). The GITHUB_TOKEN environment variable is recommended (see above).

The following are bash (Linux) and Powershell (Windows) commands to help you run the deployer by using the Azure CLI to retrieve the AKS Cluster Name. These assume you have recently used az login with the appropriate tenant to login.

Example bash (Linux):

AZURE_SUB_ID="00000000-0000-0000-0000-000000000000"
AZURE_RESOURCE_GROUP_NAME="my-cromwellonazure-rg"
echo -e "Running updater on\n\t Subscription:\t\t $AZURE_SUB_ID\n\t Resource Group:\t $AZURE_RESOURCE_GROUP_NAME\n"
deploy-cromwell-on-azure-linux --SubscriptionId "$AZURE_SUB_ID" --HelmBinaryPath /usr/local/bin/helm --ResourceGroupName "$AZURE_RESOURCE_GROUP_NAME" --update true

Example PowerShell (Windows):*

$AZURE_SUB_ID="00000000-0000-0000-0000-000000000000"
$AZURE_RESOURCE_GROUP_NAME="my-cromwellonazure-rg"
echo "Running updater on`n`t Subscription:`t`t $AZURE_SUB_ID`n`t Resource Group:`t $AZURE_RESOURCE_GROUP_NAME`n"
deploy-cromwell-on-azure-win.exe --SubscriptionId "$AZURE_SUB_ID" --HelmBinaryPath "$env:UserProfile\scoop\shims\helm.exe" --ResourceGroupName "$AZURE_RESOURCE_GROUP_NAME" --update true

Run the deployer in update mode [prior to 5.x.x]

First, download the latest binary deployer from the releases page. You can then update a Cromwell on Azure instance using the --update true flag. As of version 4.0.0 you must also provide the --aksclustername.

The following are bash (Linux) and Powershell (Windows) commands to help you run the deployer by using the Azure CLI to retrieve the AKS Cluster Name. These assume you have recently used az login with the appropriate tenant to login.

Example bash (Linux) for 4.x.x versions:

AZURE_SUB_ID="00000000-0000-0000-0000-000000000000"
AZURE_RESOURCE_GROUP_NAME="my-cromwellonazure-rg"
AKS_CLUSTER_NAME=$(az aks list --resource-group $AZURE_RESOURCE_GROUP_NAME --subscription $AZURE_SUB_ID --query '[].name' --output tsv)
echo -e "Running updater on\n\t Subscription:\t\t $AZURE_SUB_ID\n\t Resource Group:\t $AZURE_RESOURCE_GROUP_NAME\n\t AKS Cluster Name:\t $AKS_CLUSTER_NAME\n"
deploy-cromwell-on-azure-linux --SubscriptionId "$AZURE_SUB_ID" --HelmBinaryPath /usr/local/bin/helm --ResourceGroupName "$AZURE_RESOURCE_GROUP_NAME" --aksclustername "$AKS_CLUSTER_NAME" --update true

Example PowerShell (Windows) for 4.x.x versions:*

$AZURE_SUB_ID="00000000-0000-0000-0000-000000000000"
$AZURE_RESOURCE_GROUP_NAME="my-cromwellonazure-rg"
$AKS_CLUSTER_NAME=$(az aks list --resource-group $AZURE_RESOURCE_GROUP_NAME --subscription $AZURE_SUB_ID --query '[].name' --output tsv)
echo "Running updater on`n`t Subscription:`t`t $AZURE_SUB_ID`n`t Resource Group:`t $AZURE_RESOURCE_GROUP_NAME`n`t AKS Cluster Name:`t $AKS_CLUSTER_NAME`n"
deploy-cromwell-on-azure-win.exe --SubscriptionId "$AZURE_SUB_ID" --HelmBinaryPath C:\ProgramData\chocolatey\bin\helm.exe --ResourceGroupName "$AZURE_RESOURCE_GROUP_NAME" --aksclustername "$AKS_CLUSTER_NAME" --update true

Cromwell on Azure deployed resources

Once deployed, Cromwell on Azure configures the following Azure resources:

• Azure Kubernetes Service - runs Cromwell and TES and ancillary required containers Learn more
• Batch account - The Azure Batch account is used by TES to spin up the virtual machines that run each task in a workflow. After deployment, create an Azure support request to increase your core quotas if you plan on running large workflows. Learn more
• Storage account - The Azure Storage account is mounted to the host VM using blobfuse, which enables Azure Block Blobs to be mounted as a local file system available to the four containers running in Docker. By default, it includes the following Blob containers - configuration, cromwell-executions, cromwell-workflow-logs, inputs, outputs, and workflows.
• Application Insights - This contains logs from TES and the Trigger Service to enable debugging.
• Azure Database for PostgreSQL - Flexible Server - one database is created for Cromwell and another database is created for TES, which includes information and metadata about each TES task that is run as part of a workflow.

All of these resources will be grouped under a single resource group in your account, which you can view on the Azure Portal. Note that your specific resource group name, host VM name and host VM password for username "vmadmin" are printed to the screen during deployment. You can store these for your future use, or you can reset the VM's password at a later date via the Azure Portal.

You can follow these steps if you wish to mount a different Azure Storage account that you manage or own, to your Cromwell on Azure instance.

Connect to existing Azure resources I own that are not part of the Cromwell on Azure instance by default

Cromwell on Azure uses managed identities to allow the host VM to connect to Azure resources in a simple and secure manner.

At the time of installation, a user-assigned managed identity is used for IAM between Cromwell, TES, and the Azure resources.

To allow TES to connect to custom Azure resources like another Storage Account you can use the Azure Portal or Azure CLI the user-managed identity and add it as a Contributor to the required Azure resource.

For convenience, some configuration files are hosted on your Cromwell on Azure Storage account, in the "configuration" container - containers-to-mount, and cromwell-application.conf. You can modify and save these file using Azure Portal UI "Edit Blob" option or simply upload a new file to replace the existing one.

See this section in the advanced configuration on details of how to connect a different storage account, batch account, or a private Azure Container Registry.

Hello World WDL test

As part of the Cromwell on Azure deployment, a "Hello World" workflow is automatically run as a check. The input files for this workflow are found in the inputs container, and the output files can be found in the cromwell-executions container of your default storage account. Once it runs to completion you can find the trigger JSON file that started the workflow in the workflows container in the succeeded directory, if it ran successfully.

Hello World WDL file:

task hello {
  String name

  command {
    echo 'Hello ${name}!'
  }
  output {
	File response = stdout()
  }
  runtime {
	docker: 'ubuntu:16.04'
  }
}

workflow test {
  call hello
}

Hello World inputs.json file:

{
  "test.hello.name": "World"
}

Hello World trigger JSON file as seen in your storage account's workflows container in the succeeded directory:

{
  "WorkflowUrl": "/<storageaccountname>/inputs/test/test.wdl",
  "WorkflowInputsUrl": "/<storageaccountname>/inputs/test/test.json",
  "WorkflowOptionsUrl": null,
  "WorkflowDependenciesUrl": null,
  "WorkflowLabelsUrl": null
}

If your "Hello-World" test workflow or other workflows consistently fail, make sure to check your Azure Batch account quotas.