Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Installed code via python callable string #221

Merged
merged 3 commits into from
Oct 16, 2024

Conversation

john-drews
Copy link
Contributor

i want to be able to define a path to a custom function that i am installing into my airflow environment. This will allow me to test my dag locally without having to define a path and simply refer to the package path instead.

Example

        branch_by_bq_load_status:
            operator: airflow.operators.python.BranchPythonOperator
            python_callable: custom.path.to.func

allow for installable code to be passed as python callable
added test for callable str
@tatiana tatiana added this to the DAG Factory 0.20.0 milestone Oct 7, 2024
@pankajastro
Copy link
Collaborator

@john-drews, Thank you very much for the contribution, could you please fix the CI issue?

@pankajastro
Copy link
Collaborator

Could you please rebase it

@john-drews
Copy link
Contributor Author

hi @pankajastro will do.

my work computer does not let me push changes back to git hub so i will need to setup my personal computer to do this work.

@codecov-commenter
Copy link

codecov-commenter commented Oct 16, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.28%. Comparing base (e474a3a) to head (7b35330).
Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #221      +/-   ##
==========================================
+ Coverage   87.24%   87.28%   +0.04%     
==========================================
  Files           6        6              
  Lines         580      582       +2     
==========================================
+ Hits          506      508       +2     
  Misses         74       74              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@pankajastro pankajastro merged commit bf9272c into astronomer:main Oct 16, 2024
35 checks passed
@tatiana tatiana mentioned this pull request Oct 17, 2024
tatiana added a commit that referenced this pull request Oct 22, 2024
### Added
- Support using envvar in config YAML by @tatiana in #236
- **Callback improvements**
- Support installed code via python callable string by @john-drews in
#221
- Add `callback_file` & `callback_name` to `default_args` DAG level by
@subbota19 in #218
- Cast callbacks to functions when set with `default_args` on TaskGroups
by @Baraldo and @pankajastro in #235

- **Telemetry**
- For more information, please, read the [Privacy
Notice](https://github.com/astronomer/dag-factory/blob/main/PRIVACY_NOTICE.md#collection-of-data).
  - Add scarf to readme for website analytics by @cmarteepants in #219
- Support telemetry during DAG parsing emitting data to Scarf by
@tatiana in #250.

### Fixed
- Build DAGs when tehre is an invalid YAML in the DAGs folder by @quydx
and @tatiana in #184

### Others
- Development tools
  - Fix make docker-run by @pankajkoti in #249
  - Add vim dot files to .gitignore by @tatiana in #228
  - Use Hatchling to modern package building by @kaxil in #208
- CI
  - Fix static check failures in PR #218 by @pankajkoti in #251
  - Fix pre-commit checks by @tatiana in #247
  - Remove tox and corresponding build jobs in CI by @pankajkoti in #248
- Install Airflow with different versions in the CI by @pankajkoti in
#237
  - Run pre-commit hooks on all existing files by @pankajkoti in #245
  - Add Python 3.11 and 3.12 to CI test pipeline by @pankajkoti in #229
- Tests
  - Fix duplicate test name by @pankajastro in #234
  - Add static check by @pankajastro in #231
  - Fix running tests locally (outside the CI) by @tatiana in #227
  - Add the task_2 back to dataset example by @cmarteepants in #204
  - Remove unnecessary config line by @jlaneve in #202
- Documentation
  - Update the license from MIT to Apache 2.0 by @pankajastro in #191
- Add registration icon and links to Airflow references by @cmarteepants
in #190
  - Update quickstart and add feature examples by @cmarteepants #189

### Breaking changes
- Removed support for Python 3.7
- The license was changed from MIT to Apache 2.0

Closes: #217
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants