feat(terraform): update terraform lock files

[secustor]

Changes:

This PR adds a post update hook, similar to NPM to update Terraform lock files.

A Terraform binary ( >= 0.14.0 ) has to be in PATH for this to work.

Context:

Prerequisites: renovatebot/docker-renovate-full#256
Closes #7895

Documentation (please check one with an [x])

• I have updated the documentation, or
• No documentation update is required

How I've tested my work (please tick one)

I have verified these changes via:

• Code inspection only, or
• Newly added unit tests, or
• Unit tests + ran on a real repository

https://github.com/secustor/renovate_terraform_lockfile

[viceice]

Please do not follow npm implementation, use artifact handling like in helmv3 manager.

renovate/lib/manager/helmv3/artifacts.ts

Line 23 in 3ad5f98

export async function updateArtifacts({

[viceice]

This needs some unit tests

[secustor]

I have copied the helmv3 unit tests and adapted them to Terraform, but I`m not sure if they are correct like they are right now.

[Djiit]

Hey guys, can we help with this in any way? Would be awesome to see this on master. Cheers

[secustor]

Because of hashicorp/terraform#28333 and as there is no movement regarding the Hashicorp Registry improvements, I will investigate more in the direction to handle the lock file directly without using the terraform binary

My initial thoughts are:

• add capabilities to datasources that multiple hashes per release can be returned as we need this to handle a hash per platform
• Download binaries, create hashes using sha256 https://pkg.go.dev/golang.org/x/mod/sumdb/dirhash && https://github.com/hashicorp/terraform/blob/main/internal/getproviders/hash.go#L349
  This can be tricky but should be possible
• heavily cache the results
• update lock file as artifact.

Any thoughts from you guys?

[viceice]

Sounds good. Hashes should be safe to store in global cache for long time as they shouldn't ever change. They are also not privacy relevant.

[secustor]

changed the approach and dropped the hash generation at the datasource as the intial sync would take about 1-2h per provider.
I have implemented it now at the artifact layer as we here already know which exact version is needed.

[secustor]

@viceice Any chance you find some time this week to take a look?

[JamieMagee]

Merged latest main into this branch, just to be sure that there were no conflicts with #10362.

[secustor]

There have been conflicts, but they have been resolved in 051506c

[JamieMagee]

@secustor Ah, I'm sorry! When I initially looked I was only concerned with git conflicts, and didn't look for implementation conflicts. I hope it wasn't too much difficulty.

[rarkins]

So the first use case for this will be self-hosted only, right? Because it's a env variable then we app users can't opt-in individually. Could be good to have @secustor battle test it a little first anyway :)

[secustor]

@JamieMagee No problem! I have found this out by accident as I have been merging the main.

@rarkins Yes that would work only on self hosted instances. Regarding battle testing, I'm not using lock files as we pin the deps to exact versions. But I'm sure there will be people using it as it has been the most upvoted issue.

[rarkins]

I think the tests may fail on Windows:

(we only run Windows tests on main, not every PR)

We'll need to fix ASAP or revert.

[renovate-release]

🎉 This PR is included in version 25.43.0 🎉

The release is available on:

• GitHub release
• 25.43.0

Your semantic-release bot 📦🚀