v1.30.4 does not work on Azure Stack Hub

[jsafrane]

What happened:
The CSI driver v1.30.4, when used on Azure Stack Hub(ASH), fails provisioning a volume with:

E1024 05:36:01.335536       1 utils.go:110] GRPC error: rpc error: code = Internal desc = PUT https://censored.com/subscriptions/de7e09c3-b59a-4c7d-9c77-439c11b92879/resourceGroups/ci-op-wv5kxjrl-cc5c6/providers/Microsoft.Compute/disks/pvc-854653a6-6107-44ff-95e3-a6d588864420
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: NoRegisteredProviderFound
--------------------------------------------------------------------------------
{
  "error": {
    "code": "NoRegisteredProviderFound",
    "message": "No registered resource provider found for location 'censored' and API version '2023-10-02' for type 'disks'. The supported api-versions are '2017-03-30, 2018-04-01, 2018-06-01, 2018-09-30, 2019-03-01, 2019-07-01, 2019-11-01'. The supported locations are 'censored'."
  }
}

What you expected to happen:
The driver provisions a volume

Anything else we need to know?:

• Seems related to a recent Azure SDK bump, I haven't noticed anything in the driver itself requiring API version 2023-10-02.
• As far I understand ASH (which is not much), there is no ASH version that would provide disk API 2023-10-02, this says that the last API versions on ASH are 2020-09-01.

[andyzhangx]

from csi driver v1.30, we have already migrated to using sdk track2 which hardcodes the api-version as 2023-10-02, not sure whether sdk track2 supports old api-version any more. is it possible that we only run v1.29 or earlier version on Azure Stack Hub?
@jsafrane

[andyzhangx]

I filed a feature request here: Azure/azure-sdk-for-go#23686
cc @MartinForReal

[MartinForReal]

apiVersion can be overrrided here https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/[email protected]/policy#ClientOptions