Microsoft Azure blob storage doesn't allow managed identity connection in ObjectStore (but does in hook)

[tomrutter]

Apache Airflow Provider(s)

microsoft-azure

Versions of Apache Airflow Providers

No response

Apache Airflow version

9.0.1

Operating System

linux (standard airflow slim images extended with custom code running on kubernetes)

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

What happened

We use airflow connections to azure blob storage using managed identity. We would like to use the ObjectStore but this doesn't currently work.

What you think should happen instead

I propose to update the provider code to pass the required arguments to adlfs.AzureBlobFileSystem. The aim is consistent handling of the airflow connection data between WasbHook and the ObjectStoragePath for "abfs://" paths.

How to reproduce

For an existing azure blob storage container, add to airflow a connection of type "wasb" containing extra fields "account_name", "managed_identity_client_id", "workload_identity_tenant_id".

Compare results of:
WasbHook(conn_id).get_blobs_list(container_name)
to
ObjectStoragePath(f"abfs://{container_name}/", conn_id=conn_id).iterdir()

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct