Do not download and index already mirrored providers

[secustor]

Current Terraform Version

terraform version: v0.14.9

Use-cases

If terraform providers mirror terraform providers mirror -platform=linux_amd64 -platform=darwin_amd64 -platform=windows_amd64 /tmp//cache/terraform/providers is run mutliple times, the mirror command downloads and regenerate hashes every time. This is far from ideal as this leads to a lot of network traffic if regularly the mirrors are updated.

Attempted Solutions

I haven't find any option which should help here.

Proposal

Save the zh: hashes in the local index files and compare this hashes against the signature files provided by releases.hashicorp.com. If they are the same skip the whole provider, if not continue the current behavior for the provider.

References

• Feature request: Generate .terraform.lock.hcl including zh and h1 hash values for given platforms from required_providers block without downloading providers and modules #27264

[secustor]

closing as duplicate of #27811

[secustor]

Reopening as TRACE logs suggest that this is a bug:

2021/04/10 11:12:22 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/azurerm v2.55.0 for linux_amd64 at /tmp/renovate/cache/terraform/providers/registry.terraform.io/hashicorp/azurerm/terraform-provider-azurerm_2.55.0_linux_amd64.zip
2021/04/10 11:12:22 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/azurerm v2.55.0 for windows_amd64 at /tmp/renovate/cache/terraform/providers/registry.terraform.io/hashicorp/azurerm/terraform-provider-azurerm_2.55.0_windows_amd64.zip
2021/04/10 11:12:22 [WARN] ignoring file "registry.terraform.io/hashicorp/helm/1.2.4.json" as possible package for registry.terraform.io/hashicorp/helm: filename lacks expected prefix "terraform-provider-helm_"
2021/04/10 11:12:22 [WARN] ignoring file "registry.terraform.io/hashicorp/helm/1.3.2.json" as possible package for registry.terraform.io/hashicorp/helm: filename lacks expected prefix "terraform-provider-helm_"
2021/04/10 11:12:22 [WARN] ignoring file "registry.terraform.io/hashicorp/helm/2.0.2.json" as possible package for registry.terraform.io/hashicorp/helm: filename lacks expected prefix "terraform-provider-helm_"
2021/04/10 11:12:22 [WARN] ignoring file "registry.terraform.io/hashicorp/helm/2.1.0.json" as possible package for registry.terraform.io/hashicorp/helm: filename lacks expected prefix "terraform-provider-helm_"
2021/04/10 11:12:22 [WARN] ignoring file "registry.terraform.io/hashicorp/helm/index.json" as possible package for registry.terraform.io/hashicorp/helm: filename lacks expected prefix "terraform-provider-helm_"
2021/04/10 11:12:22 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/helm v1.2.4 for darwin_amd64 at /tmp/renovate/cache/terraform/providers/registry.terraform.io/hashicorp/helm/terraform-provider-helm_1.2.4_darwin_amd64.zip
2021/04/10 11:12:22 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/helm v1.2.4 for linux_amd64 at /tmp/renovate/cache/terraform/providers/registry.terraform.io/hashicorp/helm/terraform-provider-helm_1.2.4_linux_amd64.zip

[apparentlymart]

Hi @secustor,

That's not a bug, just a result of Terraform trying to use a directory with network mirror metadata as if it is a filesystem mirror. Those index files are intended to be used if you publish this directory on an HTTP server, but the local mirrors mechanism just scans the local filesystem directory directly and therefore doesn't have any need for the separate indices.

If your goal is only to create a filesystem mirror then you can just download the necessary packages and extract them into that layout, and skip creating the .json files. This command is primarily here to make it easier to construct a network mirror, by writing those index files automatically.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.