chore: Replace deprecated autorest SDK with azidentity

[shahramk64]

Description

What this PR does / why we need it:

Azure authentication currently uses go-autorest SDK. It's now deprecated and using MSAL and azidentity SDKs are now recommended. This PR replaces the deprecated SDK with azidentity.

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):

Fixes #1630

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Helm Chart Change (any edit/addition/update that is necessary for changes merged to the main branch)
• This change requires a documentation update

How Has This Been Tested?

This tutorial was followed to test authenticating AKS with AKV and retrieval of certificates.
1- KeyManagementProvider scenario: After deploying the following kmp resource using kubectl apply -f kmp_config.yaml, the Ratify pod log shows successful retrieval of the certificate.

kmp_config.yaml:

apiVersion: config.ratify.deislabs.io/v1beta1
kind: KeyManagementProvider
metadata:
  name: keymanagementprovider-akv
spec:
  type: azurekeyvault
  parameters:
    vaultURI: https://skal21kv.vault.azure.net/
    keys:
      - name: cosign
        version: 16c798cd4be049908bdcd841d20c19cd
    certificates:
      - name: wabbit-networks-io
        version: 3d6a30dfb52645ebb307d99e900c10ad
    tenantID: 72f988bf-86f1-41af-91ab-2d7cd011db47
    clientID: 6f77181a-6ca6-404d-accc-703b3ae18d03

Ratify logs:

time=2024-11-05T04:27:27.618019657Z level=info msg=provider.cloudEnv.KeyVaultEndpoint https://vault.azure.net/, provider.vaultURI https://skal21kv.vault.azure.net/ component-type=keyManagementProvider go.version=go1.22.8
time=2024-11-05T04:27:27.618039936Z level=info msg=kvEndpoint https://skal21kv.vault.azure.net component-type=keyManagementProvider go.version=go1.22.8
time=2024-11-05T04:27:27.618466847Z level=info msg=azsecrets kvclient created successfully component-type=keyManagementProvider go.version=go1.22.8
time=2024-11-05T04:27:27.618525541Z level=info msg=fetching secret from key vault, certName wabbit-networks-io, certVersion %!v(MISSING) component-type=keyManagementProvider go.version=go1.22.8
time=2024-11-05T04:27:27.901644022Z level=info msg=1 certificate(s) & 1 key(s) fetched for key management provider keymanagementprovider-akv

It was also verified by the following command:
kubectl describe KeyManagementProvider keymanagementprovider-akv

Name:         keymanagementprovider-akv
Namespace:    
Labels:       <none>
Annotations:  <none>
API Version:  config.ratify.deislabs.io/v1beta1
Kind:         KeyManagementProvider
Metadata:
  Creation Timestamp:  2024-11-05T04:27:27Z
  Generation:          1
  Resource Version:    1802478
  UID:                 889e589a-f696-4410-bd01-73bfa8a330b3
Spec:
  Parameters:
    Certificates:
      Name:     wabbit-networks-io
      Version:  3d6a30dfb52645ebb307d99e900c10ad
    Client ID:  6f77181a-6ca6-404d-accc-703b3ae18d03
    Keys:
      Name:          cosign
      Version:       16c798cd4be049908bdcd841d20c19cd
    Tenant ID:       72f988bf-86f1-41af-91ab-2d7cd011db47
    Vault URI:       https://skal21kv.vault.azure.net/
  Refresh Interval:  
  Type:              azurekeyvault
Status:
  Issuccess:        true
  Lastfetchedtime:  2024-11-05T04:27:27Z
  Properties:
    Certificates:
      Last Refreshed:  2024-11-05T04:27:27Z
      Name:            wabbit-networks-io
      Version:         3d6a30dfb52645ebb307d99e900c10ad
    Keys:
      Last Refreshed:  2024-11-05T04:27:27Z
      Name:            cosign
      Version:         16c798cd4be049908bdcd841d20c19cd
Events:                <none>

2- CertificateStore scenario: After deploying the following certificate store resource using kubectl apply -f cs_config.yaml, the Ratify pod log shows successful retrieval of the certificate.

cs_config.yaml:

apiVersion: config.ratify.deislabs.io/v1beta1
kind: CertificateStore
metadata:
  name: certstore-akv
spec:
  provider: azurekeyvault
  parameters:
    vaultURI: https://skal21kv.vault.azure.net/
    certificates: |
      array:
        - |
          certificateName: wabbit-networks-io
          certificateVersion: 3d6a30dfb52645ebb307d99e900c10ad
    tenantID: 72f988bf-86f1-41af-91ab-2d7cd011db47
    clientID: 6f77181a-6ca6-404d-accc-703b3ae18d03

Ratify logs:

time=2024-11-11T10:26:40.87690862Z level=info msg=reconciling certificate store 'gatekeeper-system/certstore-akv'
time=2024-11-11T10:26:41.589902887Z level=info msg=1 certificates fetched for certificate store gatekeeper-system/certstore-akv

It was also verified by the following command:
kubectl describe CertificateStore certstore-akv

Name:         certstore-akv
Namespace:    gatekeeper-system
Labels:       <none>
Annotations:  <none>
API Version:  config.ratify.deislabs.io/v1beta1
Kind:         CertificateStore
Metadata:
  Creation Timestamp:  2024-11-11T10:26:40Z
  Generation:          1
  Resource Version:    4036265
  UID:                 a9f89508-e7a0-45a4-855f-b1e7ec0ee13e
Spec:
  Parameters:
    Certificates:  array:
  - |
    certificateName: wabbit-networks-io
    certificateVersion: 3d6a30dfb52645ebb307d99e900c10ad

    Client ID:  6f77181a-6ca6-404d-accc-703b3ae18d03
    Tenant ID:  72f988bf-86f1-41af-91ab-2d7cd011db47
    Vault URI:  https://skal21kv.vault.azure.net/
  Provider:     azurekeyvault
Status:
  Issuccess:        true
  Lastfetchedtime:  2024-11-11T10:26:40Z
  Properties:
    Certificates:
      Certificate Name:  wabbit-networks-io
      Last Refreshed:    2024-11-11T10:26:41Z
      Version:           3d6a30dfb52645ebb307d99e900c10ad
Events:                  <none>

Checklist:

• Does the affected code have corresponding tests?
• Are the changes documented, not just with inline documentation, but also with conceptual documentation such as an overview of a new feature, or task-based documentation like a tutorial? Consider if this change should be announced on your project blog.
• Does this introduce breaking changes that would require an announcement or bumping the major version?
• Do all new files have appropriate license header?

Post Merge Requirements

• MAINTAINERS: manually trigger the "Publish Package" workflow after merging any PR that indicates Helm Chart Change

[codecov]

Codecov Report

Attention: Patch coverage is 80.00000% with 21 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...kg/keymanagementprovider/azurekeyvault/provider.go	80.95%	13 Missing and 3 partials ⚠️
pkg/certificateprovider/azurekeyvault/provider.go	76.19%	5 Missing ⚠️

Files with missing lines	Coverage Δ
pkg/certificateprovider/azurekeyvault/auth.go	65.21% <ø> (+31.12%)	⬆️
pkg/keymanagementprovider/azurekeyvault/auth.go	65.21% <ø> (+8.39%)	⬆️
pkg/certificateprovider/azurekeyvault/provider.go	63.58% <76.19%> (+5.47%)	⬆️
...kg/keymanagementprovider/azurekeyvault/provider.go	84.24% <80.95%> (-0.61%)	⬇️

... and 1 file with indirect coverage changes

---

🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests

[susanshi]

Thanks for the PR @shahramk64 , looks great overall. Would you be able to include all automated and manual validation in the PR description? our automated test may have switched to using kmp, we might need to manually validate certificateprovider path to ensure this provider continues to be functional.

[shahramk64]

Manual Validation test failed. I'll continue working on this PR and will update you once it's fixed
cc: @susanshi @akashsinghal @binbin-li

[shahramk64]

@susanshi @binbin-li @akashsinghal
I fixed a bug, manual testing succeeded and I updated the PR description to include the testing evidence.

[susanshi]

Thankyou @shahramk64 ! code change looks good to me. I have two ask:

• thank you for providing the validation result for kmp CR, since this change also updated certificate provider CR, would you be able to validate certificateprovider CR and paste the result?
• would you be to review @duffney 's PR to help understand if there will be any updates required to resolve any merge conflict.

[binbin-li]

lgtm overall, left minor comments.

[duffney]

@duffney Can you please have a look at this PR and see if the merge makes sense to you? Thanks

LGTM 👍

[binbin-li]

lgtm, thanks for the update!

[junczhu]

LGTM. Thank you for the changes.