Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deprecate databricks async provider #1416

Closed
Lee-W opened this issue Jan 2, 2024 · 2 comments · Fixed by #1460
Closed

Deprecate databricks async provider #1416

Lee-W opened this issue Jan 2, 2024 · 2 comments · Fixed by #1460
Assignees
@pankajkoti
Labels
feature New feature or request
Milestone
1.19.0

Comments

@Lee-W
Copy link
Contributor

Lee-W commented Jan 2, 2024 •
edited by pankajkoti
Loading

Is your feature request related to a problem? Please describe.

as most of the functionality has been contributed back to OSS airflow, we're going to leverage the code there and deprecate this project

Describe the solution you'd like

  1. Contribute the delta to the OSS provider
  2. Point the astronomer provider to the OSS provider
  3. Do the release and wait for a month or so before we remove the astronomer provider in a major release

Describe alternatives you've considered

Additional context

Current Progress

  • Number of operator changes on OSS airflow needed: 2
  • Number of operators to deprecate: 2
  1. DatabricksSubmitRunOperatorAsync
  2. DatabricksRunNowOperatorAsync
@Lee-W Lee-W added the feature New feature or request label Jan 2, 2024
@Lee-W Lee-W self-assigned this Jan 2, 2024
@Lee-W Lee-W changed the title Drepcreate databricks provider Deprecate databricks async provider Jan 2, 2024
@vatsrahul1001
Copy link
Contributor

Raised OSS PR

@Lee-W Lee-W added this to the 1.19.0 milestone Jan 19, 2024
pankajkoti added a commit that referenced this issue Jan 29, 2024
@pankajkoti
Deprecate DatabricksSubmitRunOperatorAsync and DatabricksRunNowOperat…
7def2bc 
…orAsync

This PR deprecates the operators DatabricksSubmitRunOperatorAsync
and DatabricksRunNowOperatorAsync from the databricks provider
by proxying them to their Airflow OSS provider's counterpart.

closes: #1416
@pankajkoti pankajkoti mentioned this issue Jan 29, 2024
Merged
@pankajkoti
Copy link
Collaborator

Created deprecation PR: #1460

pankajkoti added a commit that referenced this issue Jan 29, 2024
@pankajkoti
Deprecate DatabricksSubmitRunOperatorAsync and DatabricksRunNowOperat…
ef4e3d8 
…orAsync

This PR deprecates the operators DatabricksSubmitRunOperatorAsync
and DatabricksRunNowOperatorAsync from the databricks provider
by proxying them to their Airflow OSS provider's counterpart.

closes: #1416
pankajkoti added a commit that referenced this issue Jan 29, 2024
@pankajkoti
Deprecate DatabricksSubmitRunOperatorAsync and DatabricksRunNowOperat…
84cfc6b 
…orAsync (#1460)

* Deprecate DatabricksSubmitRunOperatorAsync and DatabricksRunNowOperatorAsync

This PR deprecates the operators DatabricksSubmitRunOperatorAsync
and DatabricksRunNowOperatorAsync from the databricks provider
by proxying them to their Airflow OSS provider's counterpart.

closes: #1416
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pankajkoti pankajkoti

Labels
feature New feature or request
Projects
None yet
Milestone
1.19.0
3 participants
@Lee-W @pankajkoti @vatsrahul1001