allow -target to accept globs

[tamsky]

Suppose one declares several concurrent environments in a single tf config.

To limit cli actions to a single environment at a time, it would be convenient if a way existed, via -target to glob-match modules, module_paths, resource_types and resource_names:

terraform apply -target=aws*.staging* # target aws* resource_types

or similarly, for all modules:

terraform apply -target=*.*.staging* # target all modules and resource_types

or similarly, for all module_paths:

terraform apply -target=**.staging* # star star = target all modules and submodules

[scalp42]

👍 would love to be able to use as opposed to writing CLIs to wrap Terraform

[joshmullis]

There are other requests I've seen for "exclude" logic, so it may be useful to have this use regex instead of glob to take care of both requests.

[mario21ic]

Where is the issue for "exclude" logic? that it'll resolve me many problems 👍, maybe like target:
$ terraform plan -exclude=mymodule.resource_not_important

[ljsommer]

Wow this is a very old request for a very useful feature.
Great idea, our use case would be to use target to specify the creation of template resources to create fully populated python files for Lambdas which have interpolated their vars from terraform so that we can do local unit and lint testing before interacting with the AWS API.

[idjaw]

Would it be correct to relate with Issue #2253

The feature request is for a different option parameter, but the nature of the feature to have an "exclude" pattern instead of just "apply" could be interesting.

Considering they are both trying to achieve somewhat similar results. By supporting also the exclude and this pattern feature, it would make it that much more flexible and powerful.

[apparentlymart]

Please do not post "+1" comments here, since it creates noise for others watching the issue and ultimately doesn't influence our prioritization because we can't actually report on these. Instead, react to the original issue comment with 👍, which we can and do report on.

[JnMik]

Not only the option target, the "Data" ressource block as well should be able to target a ressource name="prefix-*". Ressources often needs to have dynamic name to be able to spawn in parallel to have zero downtime with lifecycle "create_before_destroy", it makes them pretty hard to target sometimes.

[vincentkwok]

Any schedule on this feature? It helps on managing multiple modules. E.g. modify resources under specific module instead of all resources among modules

[sdickhoven]

we are managing 700+ domains (plus records) with terraform in cloudflare. our resource naming is perfect for prefix wildcarding (if that's easier to implement than full-on globbing):

terraform apply -target=cloudflare_record.my_domain_com__*

some zones have hundreds of records so it would be incredibly tedious to list them all as targets. but all the records are prefixed with the zone name.

so definitely 👍 on this feature.

[teamterraform]

Please do not post "+1" comments here, since it creates noise for others watching the issue and ultimately doesn't influence our prioritization because we can't actually report on these. Instead, react to the original issue comment with +1, which we can and do report on.

[jomeier]

Hi,

this request is almost four years old and seems to be rather important. Any progress on that?

Thanks and best regards,

Josef

[schneidexe]

There were at least two implemented/mentioned by @phzietsman and me a few comments above. But I guess there is more out there in the wild...

[majormoses]

Not sure as of what version but you can terraform plan -target=module.my-module which will plan for all resources within the module. Not quite the same thing but its at least a step in the right direction. This did solve most of my use cases and can live with until there is more featureful pattern matching.

[schneidexe]

That only gets you so far though. Imagine you have a complex setup containing modules for k8s clusters on AWS with lambdas, ELBs, databases, DNS records, etc. and just wanna update your ELBs. Then you need scope it to something like terraform plan -target=**.aws_elb.*. In such a case you'd now need to search for all aws_elb entries in your state and then add them as individual -targets which can become quite cumbersome.

[majormoses]

I hear, at work we use CI (jenkins) to help alleviate some of the tedious bits associated. A state list job allows an optional grep term to filter. Our plan jobs allows a comma separated target list. Not perfect, a bit of CI glue can reduce the pain until its enhanced.

[schneidexe]

Yeah, exactly that was also my approach and it's pretty much what my little wrapper project does (see above).

[whiskeysierra]

@schneidexe Instead of parsing the human-readable console output of terraform plan, you could render the plan to a file and then use terraform show on it. With some jq magic, you'd have something less brittle. Although the terraform output is most likely not going to change anytime soon.

[alahijani]

Here's my low-tech workaround. Add this function to your .bashrc:

terraform-targets () {
  sed 's/\x1b\[[0-9;]*m//g' | grep -o '# [^( ]* ' | sed " s/^# /-target '/; s/ $/'/; "
}

You can use it it in a variety of ways:

terraform plan | terraform-targets
terraform plan | terraform-targets | grep 'some pattern'
terraform plan | terraform-targets | grep 'some pattern' | xargs -r terraform apply

(The -r flag to xargs means don't run the command if the list is empty.)

[iramello]

Thanks @alahijani . I've added an additional grep to the line because we have bash scripts with comments, and those comments are incorrectly shown as -target otherwise.

terraform-targets () {
  sed 's/\x1b\[[0-9;]*m//g' | grep -o '# [^( ]* ' | grep '\.' | sed " s/^# /-target '/; s/ $/'/; "
}

[DanPhonovation]

@alahijani

terraform plan | terraform-targets | grep 'some pattern' | xargs -r terraform apply

When I try your last command for the "yes" prompt, something else gets passed and I get action cancelled. Am I doing something wrong?

[GeorgiKarapetrov]

@alahijani

terraform plan | terraform-targets | grep 'some pattern' | xargs -r terraform apply

When I try your last command for the "yes" prompt, something else gets passed and I get action cancelled. Am I doing something wrong?

I run terraform plan | terraform-targets | grep 'some pattern' | xargs -r terraform apply -auto-approve when I am positive it's what I want.

[suryastef]

I have another workaround for this, but I think this will apply only for resource that already in state

for i in $(terraform state list | grep "some-pattern");
do
  terraform apply -target=$i -auto-approve;
done

note: be careful with -auto-approve, do terraform plan first insteaad of terraform apply directly

[schneidexe]

@suryastef keep in mind that will run a separate plan/apply for each resource matching which could take a while esp. if you have e.g. a big remote state file in s3. Also in some cases the state might chnage and resources in the list would be removed as dependecies to others... so it would be beneficial to chain the -target .... -target for each $i and run the command only once.

[suryastef]

@schneidexe thank you for your concern, I didn't think that way before

Also in some cases the state might chnage

[wwalker]

The problem is the lock acquisition and release time for each call, and if they were route53 targets every call could take 3 minutes. (grumble about route53 (not terraform) taking so long to do ANYTHING...). so if I wanted to destroy 120 route53 entries
(which I actually do at least weekly), the loop method could take up to 6 hours!!!

I have another workaround for this, but I think this will apply only for resource that already in state

for i in $(terraform state list | grep "some-pattern");
do
  terraform apply -target=$i -auto-approve;
done

note: be careful with -auto-approve, do terraform plan first insteaad of terraform apply directly

[cusspvz]

Based on the solution provided by @suryastef at #2182 (comment), I think this makes more sense as it only makes use of one terraform call. (terraform allows us to define multiple targets at the same apply call)

targets=$(for x in `terraform state list | grep -e "azurerm_key_vault*.*"`; do echo "-target=$x "; done);
echo $targets | xargs terraform apply -var-file=... # add here your own options;

[schneidexe]

Unfort. that does not cover for targets that were not yet deployed and hence do not yet exist in the state...

[cusspvz]

@schneidexe figured that out after commenting it in here. 😬

[silopolis]

8 years and a half later... I hardly believe such a widespread need remains unconsidered! :|

Adding to the list of alternatives: https://github.com/future-architect/tftarget

[vanvuvuong]

I know this issue hadn't been resolved for long time, but maybe this will help someone. This is another low-level option for my case, terraform apply for multiple target based on *tf file

How I used it with terraform/terragrunt:

terraform apply $(get_target "aws_subnet,aws_vpc" "module.network.")
terragrunt apply $(get_target "aws_subnet,aws_vpc" "module.network.")

get_target() {
    # $1: pattern of target resource you want to filter(based on your terraform files), e.g: aws_subnet
    # $2: prefix that you want to add to -target, e.g: module.test.
    # result: -target module.test.aws_subnet.subnet_name
    if [ $# -ne 2 ]; then
        printf "Please run this function like this example\n- general: \t\t\t get_target \"pattern\" \"prefix\"\n- 1 pattern resource:\t\t get_target \"aws_s3\" \"module.frontend.\"\n- multiple pattern resource:\t get_target \"aws_s3,aws_subnet,aws_route\" \"module.frontend.\" "
        return 1
    fi
    array_pattern=($(echo "$1" | tr ',' '\n'))
    prefix=$2
    output_pattern=""
    apply_targets=""
    for pattern in "${array_pattern[@]}"; do
        output_pattern="^resource.*$pattern|$output_pattern"
    done
    output_pattern=${output_pattern%?}
    echo $output_pattern
    for target in $(grep -hRE "$output_pattern" | sed -e 's/resource "//g' | sed 's/" {//g' | sed 's/" "/./g'); do apply_targets="$apply_targets -target $prefix$target"; done
    echo $apply_targets
}

---

updated function for multiple pattern in 15/4/2024

[Positronico]

That's what I came up with, and it's helping me a lot. I'm using regex on resource names from the tf source and state files. Just added an exclude option to skip some of the previously selected targets for finer control.

Single python script, easy to integrate in a pipeline.

https://github.com/Positronico/terratarget

[kimllee]

Thank you @Positronico !

Looks like it can't handle the following context :

terratarget  plan -var-file values.tfvars -target="google_logging_project_sink.*"

Warning: No targets matched the wildcard patterns.

Planned command: terraform plan -var-file values.tfvars

Do you want to execute this command? [y/N]: N
Execution aborted by the user.