Status of testing Providers that were prepared on October 28, 2023

[eladkal]

Body

I have a kind request for all the contributors to the latest provider packages release.
Could you please help us to test the RC versions of the providers?

The guidelines on how to test providers can be found in

Verify providers by contributors

Let us know in the comment, whether the issue is addressed.

Those are providers that require testing as there were some substantial changes introduced:

Provider amazon: 8.10.0rc1

• AWS ECS Executor (#34381): @o-nikolas
• AIP-58: Add Airflow ObjectStore (AFS) (#34729): @bolkedebruin
• Add http to s3 operator (#35176): @simonprydden
• Enable encryption in S3 download_files() hook. (#35037): @ferruzzi
• Use base aws classes in Amazon AppFlow Operators (#35082): @Taragolis
• Use base aws classes in Amazon Athena Operators/Sensors/Triggers (#35133): @Taragolis
• Use base classes for AWS Lambda Operators/Sensors (#34890): @Taragolis
• Use base aws classes in Amazon S3 Glacier Operators/Sensors (#35108): @Taragolis
• Expose catalog parameter in AthenaOpeartor (#35103): @eladkal

Provider apache.spark: 4.3.0rc1

• Add use_krb5ccache option to SparkSubmitHook (#34386): @zeotuan

Provider cncf.kubernetes: 7.8.0rc1

• Added to the rendering of KubernetesOperator V1VolumeMount, sub_path (#35129): @mescanne
• feat: add hostAliases to pod spec in KubernetesPodOperator (#35063): @imurchie
• Replace blocking IO with async IO in AsyncKubernetesHook (#35162): @functicons
• Consolidate the warning stacklevel in KubernetesPodTrigger (#35079): @anukrati1507

Provider common.io: 1.0.0rc1

• AIP-58: Add Airflow ObjectStore (AFS) (#34729): @bolkedebruin

Provider databricks: 4.7.0rc1

• Add operator to create jobs in Databricks (#35156): @tatiana

Provider google: 10.11.0rc1

• AIP-58: Add Airflow ObjectStore (AFS) (#34729): @bolkedebruin
• Make Dataprep system test self-sufficient (#34880): @moiseenkov
• Added overrides parameter to CloudRunExecuteJobOperator (#34874): @ChloeSheasby

Provider microsoft.azure: 8.1.0rc1

• AIP-58: Add Airflow ObjectStore (AFS) (#34729): @bolkedebruin

Provider mongo: 3.4.0rc1

• Add distinct function to MongoHook in apache-airflow-providers-mongo (#34466): @octaviolage

Provider openlineage: 1.2.0rc1

• openlineage, common.sql: send column lineage from SQL operators. (#34843): @JDarDagran

Provider presto: 5.2.1rc1

• Bump min version of presto client to bring better dbapi compatibility (#35132): @potiuk

Provider slack: 8.3.0rc1

• Pass additional arguments from Slack's Operators/Notifiers to Hooks (#35039): @Taragolis

Provider smtp: 1.4.1rc1

• Make cc and bcc templated fields in EmailOperator (#35235): @Joffreybvn

Provider ssh: 3.8.1rc1

• Consolidate stacklevel in ssh operator warning (#35151): @shubhamraj-git

All users involved in the PRs:
@bolkedebruin @mescanne @shubhamraj-git @octaviolage @potiuk @zeotuan @moiseenkov @o-nikolas @ChloeSheasby @ferruzzi @Joffreybvn @anukrati1507 @eladkal @JDarDagran @Taragolis @tatiana @imurchie
@functicons @simonprydden

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[simonprydden]

#35176 HttpToS3Operator tested and working as expected

[potiuk]

Min version for presto is nicely bumped :) #35132

[bolkedebruin]

I have tested, S3 particularly, for #34729. However, this only becomes relevant for Airflow 2.8 - integrations until that time won't be active / work.

[eladkal]

I have tested, S3 particularly, for #34729. However, this only becomes relevant for Airflow 2.8 - integrations until that time won't be active / work.

Yes this is expected

[Taragolis]

• Pass additional arguments from Slack's Operators/Notifiers to Hooks #35039 : test different combination and it looks good
• Use base aws classes in Amazon AppFlow Operators #35082, Use base aws classes in Amazon Athena Operators/Sensors/Triggers #35133, Use base classes for AWS Lambda Operators/Sensors #34890, Use base aws classes in Amazon S3 Glacier Operators/Sensors #35108 : I've primary test Lambda and Athena Operators/Sensors, hardly possible to test all combination but I wan't to believe that unit tests cover other cases and Appflow and Glacier oeprators

[Taragolis]

In addition test #35037, looks fine

[moiseenkov]

#34880 works as expected

[bolkedebruin]

I'm adding a small fix to get_fs for s3 to allow anonymous access. You might want to include this.

[imurchie]

Kubernetes host_aliases (#35063) works as expected.

[eladkal]

I'm adding a small fix to get_fs for s3 to allow anonymous access. You might want to include this.

This doesn't call for RC2 because the feature is useable only with Airflow>=2.8.0 thus there is no regression in Amazon provider

[ChloeSheasby]

I believe that #34874 is working as expected, but this is my first time as a contributor, so if someone else could also test it, I would really appreciate that!

[eladkal]

Thank you everyone.
Providers are released
I invite everyone to help improve providers for the next release, a list of open issues can be found here.