From 681ef8967dc6a111ddc289507ebd5ae45963e0c5 Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Thu, 17 Oct 2024 09:52:58 +0100 Subject: [PATCH 01/13] Release 0.2.0 --- CHANGELOG.md | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2749b1e4..f3c94733 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,43 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed - Removed support for Python 3.7 +## [0.20.0a1] - 2024-08-17 + +### Added +- Support using envvar in config YAML by @tatiana in #236 +- **Callback improvements** + - Support installed code via python callable string by @john-drews in #221 + - Add `callback_file` & `callback_name` to `default_args` DAG level by @subbota19 in #218 + - Cast callbacks to functions when set with `default_args` on TaskGroups by @Baraldo and @pankajastro in #235 + +- **Telemetry** + - For more information, please, read the [Privacy Notice](https://github.com/astronomer/dag-factory/blob/main/PRIVACY_NOTICE.md#collection-of-data). + - Add scarf to readme for website analytics by @cmarteepants in #219 + - Support telemetry during DAG parsing emitting data to Scarf by @tatiana in #250. + +### Fixed +- Build DAGs when tehre is an invalid YAML in the DAGs folder by @quydx and @tatiana in #184 + +### Others +- Fix static check failures in PR #218 by @pankajkoti in #251 +- Fix make docker-run by @pankajkoti in #249 +- Fix pre-commit checks by @tatiana in #247 +- Remove tox and corresponding build jobs in CI by @pankajkoti in #248 +- Install Airflow with different versions in the CI by @pankajkoti in #237 +- Run pre-commit hooks on all existing files by @pankajkoti in #245 +- Fix duplicate test name by @pankajastro in #234 +- Add static check by @pankajastro in #231 +- Add Python 3.11 and 3.12 to CI test pipeline by @pankajkoti in #229 +- Add vim dot files to .gitignore by @tatiana in #228 +- Fix running tests locally (outside the CI) by @tatiana in #227 +- Use Hatchling to modern package building by @kaxil in #208 +- Add the task_2 back to dataset example by @cmarteepants in #204 +- Remove unnecessary config line by @jlaneve in #202 +- Update the license from MIT to Apache 2.0 by @pankajastro in #191 +- Add registration icon and links to Airflow references by @cmarteepants in #190 +- Update quickstart and add feature examples by @cmarteepants #189 + + ## [0.19.0] - 2023-07-19 ### Added - Support for Airflow Datasets (data-aware scheduling) From 1dfdbe8d479af5101eaeea0d0b3c3034ec55a996 Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Thu, 17 Oct 2024 09:53:35 +0100 Subject: [PATCH 02/13] Fix date --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f3c94733..7a977611 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,7 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed - Removed support for Python 3.7 -## [0.20.0a1] - 2024-08-17 +## [0.20.0a1] - 2024-10-17 ### Added - Support using envvar in config YAML by @tatiana in #236 From 7cb55297a94ac11983d4494caafd64d266b7c7f9 Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Thu, 17 Oct 2024 09:57:31 +0100 Subject: [PATCH 03/13] Add breaking changes, group release notes --- CHANGELOG.md | 45 ++++++++++++++++++++++++++------------------- 1 file changed, 26 insertions(+), 19 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7a977611..617c3da2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,11 +5,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] -### Changed -- Removed support for Python 3.7 ## [0.20.0a1] - 2024-10-17 + ### Added - Support using envvar in config YAML by @tatiana in #236 - **Callback improvements** @@ -26,23 +25,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Build DAGs when tehre is an invalid YAML in the DAGs folder by @quydx and @tatiana in #184 ### Others -- Fix static check failures in PR #218 by @pankajkoti in #251 -- Fix make docker-run by @pankajkoti in #249 -- Fix pre-commit checks by @tatiana in #247 -- Remove tox and corresponding build jobs in CI by @pankajkoti in #248 -- Install Airflow with different versions in the CI by @pankajkoti in #237 -- Run pre-commit hooks on all existing files by @pankajkoti in #245 -- Fix duplicate test name by @pankajastro in #234 -- Add static check by @pankajastro in #231 -- Add Python 3.11 and 3.12 to CI test pipeline by @pankajkoti in #229 -- Add vim dot files to .gitignore by @tatiana in #228 -- Fix running tests locally (outside the CI) by @tatiana in #227 -- Use Hatchling to modern package building by @kaxil in #208 -- Add the task_2 back to dataset example by @cmarteepants in #204 -- Remove unnecessary config line by @jlaneve in #202 -- Update the license from MIT to Apache 2.0 by @pankajastro in #191 -- Add registration icon and links to Airflow references by @cmarteepants in #190 -- Update quickstart and add feature examples by @cmarteepants #189 +- Development tools + - Fix make docker-run by @pankajkoti in #249 + - Add vim dot files to .gitignore by @tatiana in #228 + - Use Hatchling to modern package building by @kaxil in #208 +- CI + - Fix static check failures in PR #218 by @pankajkoti in #251 + - Fix pre-commit checks by @tatiana in #247 + - Remove tox and corresponding build jobs in CI by @pankajkoti in #248 + - Install Airflow with different versions in the CI by @pankajkoti in #237 + - Run pre-commit hooks on all existing files by @pankajkoti in #245 + - Add Python 3.11 and 3.12 to CI test pipeline by @pankajkoti in #229 +- Tests + - Fix duplicate test name by @pankajastro in #234 + - Add static check by @pankajastro in #231 + - Fix running tests locally (outside the CI) by @tatiana in #227 + - Add the task_2 back to dataset example by @cmarteepants in #204 + - Remove unnecessary config line by @jlaneve in #202 +- Documentation + - Update the license from MIT to Apache 2.0 by @pankajastro in #191 + - Add registration icon and links to Airflow references by @cmarteepants in #190 + - Update quickstart and add feature examples by @cmarteepants #189 + +### Breaking changes +- Removed support for Python 3.7 +- The license was changed from MIT to Apache 2.0 ## [0.19.0] - 2023-07-19 From fd406f7d3b5d66e6984570174c1c46ce44d90802 Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Thu, 17 Oct 2024 10:40:35 +0100 Subject: [PATCH 04/13] Update release notes with PyPI package --- CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 617c3da2..ece74fb4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -36,6 +36,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Install Airflow with different versions in the CI by @pankajkoti in #237 - Run pre-commit hooks on all existing files by @pankajkoti in #245 - Add Python 3.11 and 3.12 to CI test pipeline by @pankajkoti in #229 +- Release + - Configure GitHub to automate publishing DAG Factory in PyPI by @tatiana in #255 - Tests - Fix duplicate test name by @pankajastro in #234 - Add static check by @pankajastro in #231 From 00ce223420094a1e79b18504afcb2acc9da13d9e Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Thu, 17 Oct 2024 11:47:36 +0100 Subject: [PATCH 05/13] Update release notes after rebase --- CHANGELOG.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ece74fb4..41e3063c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] -## [0.20.0a1] - 2024-10-17 +## [0.20.0a2] - 2024-10-17 ### Added @@ -22,7 +22,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Support telemetry during DAG parsing emitting data to Scarf by @tatiana in #250. ### Fixed -- Build DAGs when tehre is an invalid YAML in the DAGs folder by @quydx and @tatiana in #184 +- Build DAGs when there is an invalid YAML in the DAGs folder by @quydx and @tatiana in #184 ### Others - Development tools @@ -44,11 +44,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Fix running tests locally (outside the CI) by @tatiana in #227 - Add the task_2 back to dataset example by @cmarteepants in #204 - Remove unnecessary config line by @jlaneve in #202 + - Fix Pytest fixture that changed DAG YAML file by @tatiana in #256 +- Refactor + - Refactor poor exception handling by @tatiana in #259 - Documentation - Update the license from MIT to Apache 2.0 by @pankajastro in #191 - Add registration icon and links to Airflow references by @cmarteepants in #190 - Update quickstart and add feature examples by @cmarteepants #189 - + - Fix `README.md` badges by @tatiana in #260 + - ### Breaking changes - Removed support for Python 3.7 - The license was changed from MIT to Apache 2.0 From 18b29f2fa06b9256526162a4ace400bb98541c69 Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Fri, 18 Oct 2024 09:20:20 +0100 Subject: [PATCH 06/13] Rebase, update release notes --- CHANGELOG.md | 4 +++- dagfactory/__init__.py | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 41e3063c..832a52af 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -36,8 +36,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Install Airflow with different versions in the CI by @pankajkoti in #237 - Run pre-commit hooks on all existing files by @pankajkoti in #245 - Add Python 3.11 and 3.12 to CI test pipeline by @pankajkoti in #229 -- Release + - Fix release action and overall CI jobs dependencies by @tatiana in #261 +- Packaging & Release - Configure GitHub to automate publishing DAG Factory in PyPI by @tatiana in #255 + - Update pyproject classifiers for Python 3.11 and 3.12 by @pankajastro in #262 - Tests - Fix duplicate test name by @pankajastro in #234 - Add static check by @pankajastro in #231 diff --git a/dagfactory/__init__.py b/dagfactory/__init__.py index 29ef06b9..1b84ff08 100644 --- a/dagfactory/__init__.py +++ b/dagfactory/__init__.py @@ -2,7 +2,7 @@ from .dagfactory import DagFactory, load_yaml_dags -__version__ = "0.20.0a2" +__version__ = "0.20.0a3" __all__ = [ "DagFactory", "load_yaml_dags", From b79ecf659df2797ae268badf7c119abc7281edcc Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Tue, 22 Oct 2024 08:42:16 +0100 Subject: [PATCH 07/13] Update changelog after rebase --- CHANGELOG.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 832a52af..ad2eac95 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -29,6 +29,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Fix make docker-run by @pankajkoti in #249 - Add vim dot files to .gitignore by @tatiana in #228 - Use Hatchling to modern package building by @kaxil in #208 + - Cleanup dependabot, MANIFEST.in and Makefile by @pankajastro in #268 + - Add Astro CLI project to validate DAG Factory by @pankajastro in #267 - CI - Fix static check failures in PR #218 by @pankajkoti in #251 - Fix pre-commit checks by @tatiana in #247 @@ -40,6 +42,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Packaging & Release - Configure GitHub to automate publishing DAG Factory in PyPI by @tatiana in #255 - Update pyproject classifiers for Python 3.11 and 3.12 by @pankajastro in #262 + - Update http sensor example to Airflow 2.0 by @pankajastro in #265 - Tests - Fix duplicate test name by @pankajastro in #234 - Add static check by @pankajastro in #231 @@ -47,6 +50,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Add the task_2 back to dataset example by @cmarteepants in #204 - Remove unnecessary config line by @jlaneve in #202 - Fix Pytest fixture that changed DAG YAML file by @tatiana in #256 + - Run integration tests in CI by @pankajkoti in #266 + - Improve test coverage by @pankajastro in #258 - Refactor - Refactor poor exception handling by @tatiana in #259 - Documentation @@ -54,7 +59,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Add registration icon and links to Airflow references by @cmarteepants in #190 - Update quickstart and add feature examples by @cmarteepants #189 - Fix `README.md` badges by @tatiana in #260 - - + ### Breaking changes - Removed support for Python 3.7 - The license was changed from MIT to Apache 2.0 From d724a112252b9ea8f3bedc0eed0b31a1b284fa9a Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Tue, 22 Oct 2024 08:42:54 +0100 Subject: [PATCH 08/13] Release 0.20.0rc1 --- CHANGELOG.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ad2eac95..342d6993 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,8 +6,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] -## [0.20.0a2] - 2024-10-17 - +## [0.20.0a3] - 2024-10-18 ### Added - Support using envvar in config YAML by @tatiana in #236 From 17bc79d1502c1c848c6fd56657daa2fb6e80974e Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Tue, 22 Oct 2024 08:50:55 +0100 Subject: [PATCH 09/13] Release 0.20.0rc1 --- CHANGELOG.md | 2 +- dagfactory/__init__.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 342d6993..2f488e1d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] -## [0.20.0a3] - 2024-10-18 +## [0.20.0rc1] - 2024-10-22 ### Added - Support using envvar in config YAML by @tatiana in #236 diff --git a/dagfactory/__init__.py b/dagfactory/__init__.py index 1b84ff08..69d4aa11 100644 --- a/dagfactory/__init__.py +++ b/dagfactory/__init__.py @@ -2,7 +2,7 @@ from .dagfactory import DagFactory, load_yaml_dags -__version__ = "0.20.0a3" +__version__ = "0.20.0rc1" __all__ = [ "DagFactory", "load_yaml_dags", From 43233fba4a4b5f1548e9f47afbc49cecb16c018c Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Tue, 22 Oct 2024 11:36:46 +0100 Subject: [PATCH 10/13] Release 0.20.0 --- CHANGELOG.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2f488e1d..b682484e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] -## [0.20.0rc1] - 2024-10-22 +## [0.20.0] - 2024-10-22 ### Added - Support using envvar in config YAML by @tatiana in #236 @@ -30,6 +30,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Use Hatchling to modern package building by @kaxil in #208 - Cleanup dependabot, MANIFEST.in and Makefile by @pankajastro in #268 - Add Astro CLI project to validate DAG Factory by @pankajastro in #267 + - Fix Makefile to run make docker-run by @tatiana in #271 - CI - Fix static check failures in PR #218 by @pankajkoti in #251 - Fix pre-commit checks by @tatiana in #247 @@ -53,6 +54,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Improve test coverage by @pankajastro in #258 - Refactor - Refactor poor exception handling by @tatiana in #259 + - Remove off looking start-date value in example_dag yaml config by @pankajkoti in #273 - Documentation - Update the license from MIT to Apache 2.0 by @pankajastro in #191 - Add registration icon and links to Airflow references by @cmarteepants in #190 From 574560c4eaaf7e3f71380e5c729c6e4ac9d55204 Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Tue, 22 Oct 2024 12:22:01 +0100 Subject: [PATCH 11/13] Update CHANGELOG.md Co-authored-by: Pankaj Koti --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index b682484e..f6a5e548 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -27,6 +27,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Development tools - Fix make docker-run by @pankajkoti in #249 - Add vim dot files to .gitignore by @tatiana in #228 + - Add local airflow setup files to .gitignore by @pankajkoti in #246 - Use Hatchling to modern package building by @kaxil in #208 - Cleanup dependabot, MANIFEST.in and Makefile by @pankajastro in #268 - Add Astro CLI project to validate DAG Factory by @pankajastro in #267 From 80088afaeb00fcd9b7fbac1dde91025eefea75ea Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Tue, 22 Oct 2024 12:22:33 +0100 Subject: [PATCH 12/13] Update dagfactory/__init__.py Co-authored-by: Pankaj Singh <98807258+pankajastro@users.noreply.github.com> --- dagfactory/__init__.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dagfactory/__init__.py b/dagfactory/__init__.py index 69d4aa11..f6569d23 100644 --- a/dagfactory/__init__.py +++ b/dagfactory/__init__.py @@ -2,7 +2,7 @@ from .dagfactory import DagFactory, load_yaml_dags -__version__ = "0.20.0rc1" +__version__ = "0.20.0" __all__ = [ "DagFactory", "load_yaml_dags", From f9996c8c2f53d6ddb0cce3572330143730323573 Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Tue, 22 Oct 2024 12:22:42 +0100 Subject: [PATCH 13/13] Update CHANGELOG.md Co-authored-by: Pankaj Koti --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index f6a5e548..e28a915b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -61,6 +61,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Add registration icon and links to Airflow references by @cmarteepants in #190 - Update quickstart and add feature examples by @cmarteepants #189 - Fix `README.md` badges by @tatiana in #260 + - Remove duplicated operator in `README.md` by @brair in #263 ### Breaking changes - Removed support for Python 3.7