Handling of .terraform.lock.hcl

[ChristophShyper]

Will there be any way of upgrading providers' locks in batches?
I mean something that will run terraform init -upgrade for the whole Terragrunt module tree.
Even first generating of .terraform.lock.hcl files is not straightforward and I needed to use terragrunt output-all to make it faster.
Imagine having hundreds of Terragrunt modules in your tree, that's my use case.

[jsnod]

I just achieved this yesterday by doing the following:

Add -upgrade flag to all init commands:

terraform {
  extra_arguments "init_args" {
    commands = [
      "init"
    ]

    arguments = [
      "-upgrade",
    ]
  }
}

Then simply run plan on all your modules:

terragrunt plan-all

This will force update all your lock files.

[ChristophShyper]

I achieved it with running output-all, but this is just a workaround and produces unnecessary overhead for getting state file and checking for outputs. But even that takes a long time and doesn't make any sense on a daily basis.
But currently **/.terraform.lock.hcl in .gitignore is just disabling the issue :)

Ideally creating init-all would solve those issues.

[brikis98]

Seems like #492 would solve this, right?

[ChristophShyper]

Seems like #492 would solve this, right?

Yes, if it'll ever get done.

[eperdeme]

Hey guys.

I'm just wondering what people are doing around Terragrunt and these 'helpful' new lockfiles. So say you have the structure in terragrunt like;

env/region/stack

./prod/eu-west-1/consume-ami
./nonprod/eu-west-1/consume-ami
./dev/eu-west-1/consume-ami

It would make sense to me that all 3 envs (dev/nonprod/prod) have the same lockfile contents, if you are promoting a release(git tag/commit/etc) through your environments. To do this it seems I'd need to in the first path generate the new .terraform.lock.hcl then manually copy it to the other two environments paths. That seems open to human error especially when we're talking about 100's of stacks and just messy and not very DRY.

I guess ideally it'd be good if .terraform.lock.hcl was outside of the env/region path and bound to all stacks containing the same name, any idea how that would be possible using Terragrunt ?

The other option I thought is in CI during the DEV planb/apply stage we pass the .terraform.lock.hcl file on through the pipeline, but then if you attempt to re-run the same TAG X days later and a new provider downloads results could differ.

[ChristophShyper]

That's not SDLC. First you update it on dev and test if it's ok.
Then you run on staging, test it and then on production.
With lock files you see at any given point in time which specific provider version was used exactly where.

If you don't commit lock files you will achieve what you mentioned - it will always look for a provider using defined chain and version restrictions.

[eperdeme]

We do that but in a different way. We test it on dev env(lets say feature branch), then if passed we merge+tag the repo and promote the tag upwards, but if we need to manually copy the file to lockfile to ./nonprod/eu-west-1/consume-ami
&& ./dev/eu-west-1/consume-ami then humans will forget/make a mistake and then nonprod/prod could result in different providers in the lockfile (earlier lockfile contents). So it'd be better if all ENV stacks used a single source lockfile so when it passes test in dev env and merged+tagged the other envs references it.

[lorengordon]

I'm also struggling with the new lockfile. Seems like a similar use case to @eperdeme. For us it's one module, many many implementations of the same module managing a baseline set of resources across all accounts of a given "env" or "stage" (so, all "dev" accounts, for example). The new lockfile is quite painful to update across all of the terragrunt directories, and would basically involve a human or CI process running init for all of them, committing the change, opening a new pr, etc.

The only advantage I can see to the lockfile is the hashes. The versions can already get pinned in the terraform config block. So for now, my workaround is basically just to git-ignore the lockfile. 🤷‍♂️ I also already construct the plugin-cache with known providers. So, pin provider versions, use terraform-bundle to create the plugin-cache with those versions, and host that for the build system. Build system retrieves the bundle, sets up the plugin-cache so only those versions are used/available, runs terragrunt. Also added an after-hook in the terragrunt config to remove .terraform.lock.hcl, to reduce the clutter when I am working locally.

  after_hook "terraform_lock" {
    commands = get_terraform_commands_that_need_locking()
    execute  = ["rm", "-f", "${get_terragrunt_dir()}/.terraform.lock.hcl"]
  }

I'm considering keeping a more central copy of the lockfile in one place in the project, and using a before-hook to copy it into the terragrunt working dir before any command. I'd still remove it from the working dir after, and separately manage updates to that central copy. Edit: I suppose instead of copying the file, I could use generate to put it in the right place...

[brikis98]

Thanks for providing more context everyone. A few thoughts:

• A single, central lock file for each module doesn't really work as a general-purpose solution, as you may be using different versions of that module in different environments, and each version may have a different lock file.
• So I guess for now, the recommended solution is, when updating the version number on a module, you run init, and commit the lock file too. That said, I agree this is a lot of annoying overhead.
• I wonder if the lock file should be versioned with the module itself? That is, you create the module in, say, an infrastructure-modules repo, and in that repo, you run init to generate the lock file, and commit it. Then, whenever you use that module with Terragrunt in, say, an infrastructure-live repo, the lock file will be downloaded with the module code. You can then safely ignore the lock file Terragrunt copies for you next to the terragrunt.hcl file; perhaps we can even add a flag to disable that copying behavior?

[ChristophShyper]

• So I guess for now, the recommended solution is, when updating the version number on a module, you run init, and commit the lock file too. That said, I agree this is a lot of annoying overhead.

I guess I can make script that will run init in each directory with terragrunt.hcl. But having init-all built in would be more elegant solution.

• I wonder if the lock file should be versioned with the module itself? That is, you create the module in, say, an infrastructure-modules repo, and in that repo, you run init to generate the lock file, and commit it. Then, whenever you use that module with Terragrunt in, say, an infrastructure-live repo, the lock file will be downloaded with the module code. You can then safely ignore the lock file Terragrunt copies for you next to the terragrunt.hcl file; perhaps we can even add a flag to disable that copying behavior?

It will not work in my case for example. My modules' monorepo uses the newest providers, as there is only terraform validate running in its CI system and they are rarely updated.
But then in live infra repo I'm using version constraints for each module, and they can be different, depending on SDLC environment, type, etc. That way I can promote versions of providers without the need of making releases on modules' monorepo.

Also wouldn't the file be overwritten automatically by Terragrunt if its coming from module's repo?

[brikis98]

• So I guess for now, the recommended solution is, when updating the version number on a module, you run init, and commit the lock file too. That said, I agree this is a lot of annoying overhead.

I guess I can make script that will run init in each directory with terragrunt.hcl. But having init-all built in would be more elegant solution.

Agreed. PRs welcome!

• I wonder if the lock file should be versioned with the module itself? That is, you create the module in, say, an infrastructure-modules repo, and in that repo, you run init to generate the lock file, and commit it. Then, whenever you use that module with Terragrunt in, say, an infrastructure-live repo, the lock file will be downloaded with the module code. You can then safely ignore the lock file Terragrunt copies for you next to the terragrunt.hcl file; perhaps we can even add a flag to disable that copying behavior?

It will not work in my case for example. My modules' monorepo uses the newest providers, as there is only terraform validate running in its CI system and they are rarely updated.
But then in live infra repo I'm using version constraints for each module, and they can be different, depending on SDLC environment, type, etc. That way I can promote versions of providers without the need of making releases on modules' monorepo.

Hm, we typically have all of our automated testing in the modules repo... So bumping provider version numbers there makes a lot more sense to me, as then it goes through the entire test suite. And if that test suite passes, we release a new version of the modules repo to indicate that provider versions were upgraded and tested. The live repo should mostly be to define environments and set environment-specific variables for them. Using live to test out different provider versions seems more risky, no?

Also wouldn't the file be overwritten automatically by Terragrunt if its coming from module's repo?

If you had the lock file checked into the live repo too, then yes, Terragrunt would copy it and overwrite the one from the modules repo. But my point is that you wouldn't check the lock file into your live repo at all.

[yorinasub17]

I guess I can make script that will run init in each directory with terragrunt.hcl. But having init-all built in would be more elegant solution.

I have a PR that implements #492 in #1520, which should allow you to do init-all by running terragrunt run-all init -upgrade. You can subscribe to that PR to be notified when we merge + release it.

[yorinasub17]

#1520 is now merged and released as https://github.com/gruntwork-io/terragrunt/releases/tag/v0.28.1

[ChristophShyper]

Love it! Tomorrow I'll test it. Thank you all.