Terraform 0.14 dependency lock file unusable because of platform mismatch

[Nosmoht]

We develop Terraform code mainly on Mac OSX but execute it on Linux systems, therefore provider hashes written to .terraform.lock.hcl do not match. Because of this, every Pipeline fails with error message doesn't match any of the checksums previously recorded in the dependency lock file.

Terraform Version

Terraform v0.14.0

Terraform Configuration Files

terraform {
  required_providers {
    null = {
      source  = "hashicorp/null"
      version = "= 3.0.0"
    }
  }
  required_version = ">= 0.14"
}

resource "null_resource" "test" {}

Debug Output

Provider hash for null provider on Mac OSX

provider "registry.terraform.io/hashicorp/null" {
  version     = "3.0.0"
  constraints = "3.0.0"
  hashes = [
    "h1:V1tzrSG6t3e7zWvUwRbGbhsWU2Jd/anrJpOl9XM+R/8=",
  ]
}

Provider hash for null provider on amd64 Linux system

provider "registry.terraform.io/hashicorp/null" {
  version     = "3.0.0"
  constraints = "3.0.0"
  hashes = [
    "h1:ysHGBhBNkIiJLEpthB/IVCLpA1Qoncp3KbCTFGFZTO0=",
    "zh:05fb7eab469324c97e9b73a61d2ece6f91de4e9b493e573bfeda0f2077bc3a4c",
    "zh:1688aa91885a395c4ae67636d411475d0b831e422e005dcf02eedacaafac3bb4",
    "zh:24a0b1292e3a474f57c483a7a4512d797e041bc9c2fbaac42fe12e86a7fb5a3c",
    "zh:2fc951bd0d1b9b23427acc93be09b6909d72871e464088171da60fbee4fdde03",
    "zh:6db825759425599a326385a68acc6be2d9ba0d7d6ef587191d0cdc6daef9ac63",
    "zh:85985763d02618993c32c294072cc6ec51f1692b803cb506fcfedca9d40eaec9",
    "zh:a53186599c57058be1509f904da512342cfdc5d808efdaf02dec15f0f3cb039a",
    "zh:c2e07b49b6efa676bdc7b00c06333ea1792a983a5720f9e2233db27323d2707c",
    "zh:cdc8fe1096103cf5374751e2e8408ec4abd2eb67d5a1c5151fe2c7ecfd525bef",
    "zh:dbdef21df0c012b0d08776f3d4f34eb0f2f229adfde07ff252a119e52c0f65b7",
  ]
}

Crash Output

Expected Behavior

Pipeline does not fail as .terraform.lock.hcl contains hashes of the provider for all supported platforms (?)

Actual Behavior

Pipeline fails on any system other than Mac OSX as provider hashes do not match

Steps to Reproduce

• Take provided config file
• Run terraform init on OSX
• Checkin .terraform.lock.hcl
• Checkout the same code on Linux system (or any other)
• Execute terraform plan

Additional Context

References

[slessardjr]

Use the following command on the directory to generate the lockfile for all platforms. See this documented here

terraform providers lock -platform=windows_amd64 -platform=darwin_amd64 -platform=linux_amd64

[Nosmoht]

Thanks @slessardjr , i didn't saw this.

[pkolyvas]

@slessardjr - thanks for linking to the docs!

I'm going to close this as a dupe of #26901

Here's a link to Martin's comment in the above issue with some additional explanation: #26901 (comment)

One question I'd have here though is exploring how your OS X workstation created a lock file with only a single hash. Are you using a local filesystem mirror or network mirror by any chance? Those are the cases where we expect to see a lock file automatically populated with only the current-platform's hashes.

[Nosmoht]

Hi @pkolyvas ,

i've TF_PLUGIN_CACHE_DIR set as environment variable. That might be the reason why i've only a single hash.

[shamil]

Same as for @Nosmoht , if I've TF_PLUGIN_CACHE_DIR, then lock file populated only with single hash.
But having TF_PLUGIN_CACHE_DIR unset causing plugins to be re-downloaded each time I run init.

[pdecat]

@pkolyvas Is setting TF_PLUGIN_CACHE_DIR the real cause of the lock file being created with only a single hash? Is this expected or is this a bug? The documentation does not mention this.

[pdecat]

If I understand the comment here:

  			// NOTE: The behavior here is unfortunate when a particular
  			// provider version was already cached on the first time
  			// the current configuration requested it, because that
  			// means we don't currently get the opportunity to fetch
  			// and verify the checksums for the new package from
  			// upstream. That's currently unavoidable because upstream
  			// checksums are in the "ziphash" format and so we can't
  			// verify them against our cache directory's unpacked
  			// packages: we'd need to go fetch the package from the
  			// origin and compare against it, which would defeat the
  			// purpose of the global cache.

Maybe the underlying issue is caused by a pre-filled cache when the lock file was created?

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.