hashicorp/go-azure-sdk refactoring for azurerm_subscriptions - pagination and modelling issue

[jschoombee]

Is there an existing issue for this?

• I have searched the existing issues

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.7.5

AzureRM Provider Version

3.92.0

Affected Resource(s)/Data Source(s)

azurerm_subscriptions

Terraform Configuration Files

data "azurerm_subscriptions" "all" {
   //searching with a few thousand subscriptions in tenant 
}

Debug Output/Panic Output

Less subscriptions in the tf state instances than expected.

Expected Behaviour

No response

Actual Behaviour

No response

Steps to Reproduce

Use a version prior to 3.90.0 and all subscriptions (>1000) are returned. This PR seemed to have created the issue

I debugged the provider locally and managed to resolve the issue by dropping the @odata prefix for nextLink here

Further count in the subscriptions response needed changing as it is not of type int. Below is what I ended up with which did the trick for me:

// OData is used to unmarshal OData metadata from an API response.
type OData struct {
	Context      *string `json:"@odata.context"`
	MetadataEtag *string `json:"@odata.metadataEtag"`
	Type         *Type   `json:"@odata.type"`
	Count        *Count  `json:"count"`
	NextLink     *Link   `json:"nextLink"`
	Delta        *Link   `json:"@odata.delta"`
	DeltaLink    *Link   `json:"@odata.deltaLink"`
	Id           *Id     `json:"@odata.id"`
	EditLink     *Link   `json:"@odata.editLink"`
	Etag         *string `json:"@odata.etag"`

	Error *Error `json:"-"`

	Value interface{} `json:"value"`
}

type Count struct {
	Type  *string `json:"type"`
	Value *int    `json:"value"`
}

Important Factoids

Have a local branch to fix this issue and happy to submit but not sure if other resources/data sources depending on this package might be affected

References

PR

[wuxu92]

Hi @jschoombee ,

Thank you for reporting this issue. It appears that the OData structure lost some information during unmarshalling. Could you please provide the API response that caused the issue? (I don't have an account with multiple subscriptions to reproduce the problem.)

[jschoombee]

Hi @wuxu92,

I logged the raw responses when testing, below is a redacted snippet of the output. I haven't worked with the MS graph APIs much but it seems like OData was not implemented for subscriptions in azurerm - api ref

{
    "value": [ 
        {
            "id": "/subscriptions/11111111-9f71-4226-ae1b-7995c4af4ccc",
            "authorizationSource": "RoleBased",
            "managedByTenants": [],
            "tags": {
                "redacted": 1
            },
            "subscriptionId": "11111111-9f71-4226-ae1b-7995c4af4ccc",
            "tenantId": "11111111-a3da-442a-1111-235cac24cd5c",
            "displayName": "redacted",
            "state": "Enabled",
            "subscriptionPolicies": {
                "locationPlacementId": "Public_2014-09-01",
                "quotaId": "EnterpriseAgreement_2014-09-01",
                "spendingLimit": "Off"
            }
        }
    ],
    "nextLink": "https://management.azure.com/subscriptions?$skipToken=redacted%3d&api-version=2022-12-01",
    "count": {
        "type": "Total",
        "value": 5297
    }
}

[BigFrog-coding]

Is there any progress on this? This is a blocking issue.

[jschoombee]

Confirm this bug still exists in 3.110.0

[BigFrog-coding]

Confirm this bug still exists in 3.110.0

They dont seem to care or believe us

[wuxu92]

@tombuildsstuff This issue should remain open for now. The Pandora PR only fixes the SDK generation problem, and AzureRM needs to update its SDK to address the current issue.

[jschoombee]

can we re-open?

[jschoombee]

Can this be re-opened?

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.