runtime/race: 20x perf slowdown when program built with the race detector

[kevinburke]

Terraform is a tool for provisioning clusters of servers. You define
configuration files in a configuration language called "hcl". Terraform parses
these files and applies updates to your servers based on the configuration.

Terraform also ships with a tool called "fmt", that works similarly to "go fmt".
You apply it to hcl files, it parses them and then writes the updated file to
disk.

I was interested that when you build terraform with the race detector enabled,
it runs about 20 times slower than with the race detector disabled.

0s kevin at warthog in ~/src/...
± (ads-3*) $ time ~/bin/terraform fmt dev/terraform.tfvars
~/bin/terraform fmt dev/terraform.tfvars  0.02s user 0.02s system 52% cpu 0.085 total

With the race detector enabled:

0s kevin at warthog in ~/src/...
± (ads-3*) $ time ~/bin/terraform fmt dev/terraform.tfvars
~/bin/terraform fmt dev/terraform.tfvars  0.09s user 0.07s system 7% cpu 2.164 total

This is the configuration file I'm trying to format:

# Terragrunt is a thin wrapper for Terraform that provides extra tools for working with multiple Terraform modules,
# remote state, and locking: https://github.com/gruntwork-io/terragrunt
terragrunt = {
  # Configure Terragrunt to automatically store tfstate files in an S3 bucket
  remote_state {
    backend = "s3"
    config {
      encrypt = true
      bucket = "mybucket"
      key = "${path_relative_to_include()}/terraform.tfstate"
      region = "us-west-2"
      dynamodb_table = "dynamo-table"
    }
  }
}

I don't know if we have explicit performance benchmarks for running code with
the race detector enabled, but a 20x slowdown on a popular-ish program seems
bad.

I've been enabling the race detector on more Go programs now in the hopes of
catching more errors and stressing programs but maybe it's not appropriate in
all situations.

I'm using the latest version of terraform (hashicorp/terraform@5fb1fe9) and the current Go tip.

[egonelbre]

This is to be expected https://golang.org/doc/articles/race_detector.html#Runtime_Overheads.

[ianlancetaylor]

Using the race detector for testing should always be appropriate.

Nobody is expected to use the race detector in production.

[kevinburke]

OK, closing, since a 20x slowdown is within the bounds of acceptable performance.