azurerm_policy_set_definition/azurerm_policy_definition - Reverse the order of policies lookup to favour builtin

[Nepomuceno]

This one would solve #18337

It changed the plan time from over 2 minutes to less than 40 seconds fot the sample case

terraform {
}

provider "azurerm" {
  features {}
}

data "azurerm_policy_set_definition" "policy_set" {
  name = "179d1daa-458f-4e47-8086-2a68d0d6c38f"
}

data "azurerm_policy_definition" "for_each_pol" {
    count = length(data.azurerm_policy_set_definition.policy_set.policy_definition_reference)
    name = data.azurerm_policy_set_definition.policy_set.policy_definition_reference[0].reference_id
}

There could be a solution long term for you to sepecify the policy type when creating a data block but those would probablt be corner cases.

[katbyte]

Thanks @Nepomuceno - however i don't think we can just change the order without impacting existing users (and the ability to lookup a custom policy with the same name as a built in policy)

instead how about we add 2 new properties? a name_builtin and name_user_defined which allows a user to specify the type? this will allow you to save the time while also not impacting existing users. WDYT?

[Nepomuceno]

This would have no change pratically for who it is using the provider already. It would just change when those throtololing would happen. for the vast majority of people their tplan would just become a lot faster.

This is not a breaking change.

We could add just a flag built_in and keep just the name property if this flag was not set I would look at both if this flag was set I woudl look only on the built in ones or I would not look in the built in ones if it was set but false.

Burt even adding this property does not change the fact that whejn looking at both it is more efficient to look first in the built in ones.

[wuxu92]

what about using a parallel query: one for built-in policy and another for custom policy and then joining both results? there will be no scheme update, and users will not aware of it.

[Nepomuceno]

This would just increase the problem. The problem right now it is that if you have too many policies to lookup you go over the 120 queries per minute limit and needs to keep waiting the throttling. If you execute both queries always you will reach this even earlier.

[katbyte]

@Nepomuceno - on internal discussion the consensus is yes it could affect users in the case their custom policy is named the same as the builtin. I think the best way is to just add two new properties that users can switch to depending on what type of policy they are looking for & document this can save a lot of time for built-in policies (as well provide a way to limit the search)

[mbfrahry]

LGTM! After another internal discussion, we should be ok here as @katbyte was thinking about display_name versus name. Thanks for getting this written out @Nepomuceno

[Nepomuceno]

Yeah calling this property "name" it is quite misleading. It should have been called ID

[github-actions]

This functionality has been released in v3.30.0 of the Terraform Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.