Terraform init Client.Timeout exceeded while awaiting headers

[scanlonjts]

MacBook Pro (13-inch, 2020, Four Thunderbolt 3 ports), Processor: 2 GHz Quad-Core Intel Core i5
No VPN running, No proxy set

Simple tf file:
main.tf
provider "aws"{}

Terraform v0.14.3
go version go1.13.15 darwin/amd64
> terraform init
Initializing the backend...

Initializing provider plugins...

• Finding latest version of hashicorp/aws...

Error: Failed to query available provider packages

Could not retrieve the list of available versions for provider hashicorp/aws:
could not query provider registry for registry.terraform.io/hashicorp/aws: the
request failed after 2 attempts, please try again later: Get
"https://registry.terraform.io/v1/providers/hashicorp/aws/versions": net/http:
request canceled while waiting for connection (Client.Timeout exceeded while
awaiting headers)

trying again with debug enabled
jscanlon@jscanlon-mac xxxxx $ TF_LOG=trace terraform init
2020/12/28 10:06:35 [INFO] Terraform version: 0.14.3
2020/12/28 10:06:35 [INFO] Go runtime version: go1.15.2
2020/12/28 10:06:35 [INFO] CLI args: []string{"/Users/jscanlon/bin/terraform", "init"}
2020/12/28 10:06:35 [DEBUG] Attempting to open CLI config file: /Users/jscanlon/.terraformrc
2020/12/28 10:06:35 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2020/12/28 10:06:35 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2020/12/28 10:06:35 [DEBUG] ignoring non-existing provider search directory /Users/jscanlon/.terraform.d/plugins
2020/12/28 10:06:35 [DEBUG] ignoring non-existing provider search directory /Users/jscanlon/Library/Application Support/io.terraform/plugins
2020/12/28 10:06:35 [DEBUG] ignoring non-existing provider search directory /Library/Application Support/io.terraform/plugins
2020/12/28 10:06:35 [INFO] CLI command args: []string{"init"}

Initializing the backend...
2020/12/28 10:06:35 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
2020/12/28 10:06:35 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2020/12/28 10:06:35 [DEBUG] New state was assigned lineage "b3e48b4e-aa2d-fe59-402f-6645dad28ca4"
2020/12/28 10:06:35 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2020/12/28 10:06:35 [TRACE] Meta.Backend: instantiated backend of type
2020/12/28 10:06:35 [DEBUG] checking for provisioner in "."
2020/12/28 10:06:35 [DEBUG] checking for provisioner in "/Users/jscanlon/bin"
2020/12/28 10:06:35 [INFO] Failed to read plugin lock file .terraform/plugins/darwin_amd64/lock.json: open .terraform/plugins/darwin_amd64/lock.json: no such file or directory
2020/12/28 10:06:35 [TRACE] Meta.Backend: backend does not support operations, so wrapping it in a local backend
2020/12/28 10:06:35 [TRACE] backend/local: state manager for workspace "default" will:

• read initial snapshot from terraform.tfstate
• write new snapshots to terraform.tfstate
• create any backup at terraform.tfstate.backup
  2020/12/28 10:06:35 [TRACE] statemgr.Filesystem: reading initial snapshot from terraform.tfstate
  2020/12/28 10:06:35 [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that's okay
  2020/12/28 10:06:35 [TRACE] statemgr.Filesystem: read nil snapshot

Initializing provider plugins...
2020/12/28 10:06:35 [DEBUG] Service discovery for registry.terraform.io at https://registry.terraform.io/.well-known/terraform.json

• Finding latest version of hashicorp/aws...
  2020/12/28 10:06:35 [TRACE] HTTP client GET request to https://registry.terraform.io/.well-known/terraform.json
  2020/12/28 10:06:38 [ERR] Checkpoint error: Get "https://checkpoint-api.hashicorp.com/v1/check/terraform?arch=amd64&os=darwin&signature=2d9cfee6-e85b-19db-793a-5f30ab0ebcfe&version=0.14.3": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
  2020/12/28 10:06:46 [DEBUG] GET https://registry.terraform.io/v1/providers/hashicorp/aws/versions
  2020/12/28 10:06:46 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/hashicorp/aws/versions
  2020/12/28 10:06:56 [ERR] GET https://registry.terraform.io/v1/providers/hashicorp/aws/versions request failed: Get "https://registry.terraform.io/v1/providers/hashicorp/aws/versions": context deadline exceeded
  2020/12/28 10:06:56 [DEBUG] GET https://registry.terraform.io/v1/providers/hashicorp/aws/versions: retrying in 1s (1 left)
  2020/12/28 10:06:57 [INFO] Previous request to the remote registry failed, attempting retry.
  2020/12/28 10:06:57 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/hashicorp/aws/versions
  2020/12/28 10:07:07 [ERR] GET https://registry.terraform.io/v1/providers/hashicorp/aws/versions request failed: Get "https://registry.terraform.io/v1/providers/hashicorp/aws/versions": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

Error: Failed to query available provider packages

Could not retrieve the list of available versions for provider hashicorp/aws:
could not query provider registry for registry.terraform.io/hashicorp/aws: the
request failed after 2 attempts, please try again later: Get
"https://registry.terraform.io/v1/providers/hashicorp/aws/versions": net/http:
request canceled while waiting for connection (Client.Timeout exceeded while
awaiting headers)

[scanlonjts]

after researching thru the other reports looked at my DNS settings - using Eero networking had local DNS caching enabled -- when I disable the local cache it all works fine - --

re-opened - this did not solve the issue

[danieldreier]

@scanlonjts thanks for reporting this. My default assumption is that OS X network issues are related to #3536. We are planning on shipping a CGO-enabled Terraform build for OS X to work around that issue. One potential workaround, or test for whether this would fix your particular issue, is to try installing Terraform via homebrew, because they are already enable cgo already.

Please let us know if this works. If so, the fix to #3536 should also fix the issue you're reporting. If not, further investigation is needed.

[scanlonjts]

workaround: add: export TF_REGISTRY_CLIENT_TIMEOUT=20 to profile

[scanlonjts]

Tested - removing terraform and TF_REGISTRY_CLIENT_TIMEOUT - installed via homebrew - that works - and is SO MUCH FASTER than changing the default timeout settings

[danieldreier]

that is great news! Thank you for the quick test and response here. I encourage you to keep using the homebrew version (or your own cgo-enabled build) to work around this, for now. I am hopeful that 0.15 will be cgo-enabled but can't quite promise it yet.

In the meantime, I'm going to close this issue because I believe the #3536 issue basically tracks the issue you're seeing, in that the work to fix it is already queued up. Thanks again for reporting this!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.