[FEAT]: Reduce API throttling by reading all env variables per env at once.

[DavidWHarvey]

Describe the need

When running plans against 64 repos, we hit API throttling at about an order of magnitude earlier due to the reading of env variables one at a time rather than one API call to get all of them (the data source can read them all in one call). WRT number of resources per repo, env variables currently dominate, since there may be 10 or 50 per environment.

We are re-planning repos and Azure resources (using a matrix per repo - so one plan per repo in parallel), and the process takes about 8 minutes on 64 repos, consuming about 60% of the budget, i.e., we cannot run this twice in the same hour.

I have identified 3 possible solutions:

• on a read, read all values for the environment and cache them in memory. This is simple, but I'm unclear about how this might affect terraform.
• create a new resource that holds the env variable map for the entire environment, similar to the data source.
• Work around this with a provisioner and the data source to trigger the provisioner to run. The provisioner would sync a per env map with the contents of the environment.

SDK Version

No response

API Version

No response

Relevant log output

No response

Code of Conduct

• I agree to follow this project's Code of Conduct