Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplified configuration parsing in Apprise Connection #37202

Merged
merged 12 commits into from
Feb 8, 2024

Conversation

infohash
Copy link
Contributor

@infohash infohash commented Feb 6, 2024

Closes #37170

Previously

$ export AIRFLOW_CONN_APPRISE_DEFAULT='{"extra": {"config": "{\"path\": \"https://hooks.slack.com/services/T1JJ3T3L2/A1BRTD4JD/TIiajkdnlazkcOXrIdevi7F\", \"tags\": \"alert\"}"}}'

>>> from airflow.providers.apprise.hooks.apprise import AppriseHook
>>> hook = AppriseHook()
>>> hook.get_config_from_conn()

{"path": "https://hooks.slack.com/services/T1JJ3T3L2/A1BRTD4JD/TIiajkdnlazkcOXrIdevi7F", "tags": "alert"}

Now

No need to escape quotes in the environment variable.

$ export AIRFLOW_CONN_APPRISE_DEFAULT='{"extra": {"config": {"path": "https://hooks.slack.com/services/T1JJ3T3L2/A1BRTD4JD/TIiajkdnlazkcOXrIdevi7F", "tags": "alert"}}}'


>>> from airflow.providers.apprise.hooks.apprise import AppriseHook
>>> hook = AppriseHook()
>>> hook.get_config_from_conn()

{"path": "https://hooks.slack.com/services/T1JJ3T3L2/A1BRTD4JD/TIiajkdnlazkcOXrIdevi7F", "tags": "alert"}

Copy link

boring-cyborg bot commented Feb 6, 2024

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: [email protected]
    Slack: https://s.apache.org/airflow-slack

Copy link
Contributor

@Taragolis Taragolis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to check Static Checks, i would recommend install pre-commit, it would save a lot of time

It would be also nice if we have tests cases that string also works as expected

dirrao

This comment was marked as duplicate.

Copy link
Contributor

@dirrao dirrao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a breaking change. It's worth to mention it in the provider release doc.

@infohash
Copy link
Contributor Author

infohash commented Feb 7, 2024

It's backward compatible. See this line:

return json.loads(config) if isinstance(config, str) else config

and the test case which checks for both ways (as JSON string or as dict) config can be provided.

@pytest.mark.parametrize(
"config",
[
{"path": "http://some_path_that_dont_exist/", "tag": "alert"},
'{"path": "http://some_path_that_dont_exist/", "tag": "alert"}',
],
)
def test_get_config_from_conn(self, config):
extra = {"config": config}
with patch.object(
AppriseHook,
"get_connection",
return_value=Connection(conn_type="apprise", extra=extra),
):
hook = AppriseHook()
assert hook.get_config_from_conn() == (json.loads(config) if isinstance(config, str) else config)

@Taragolis Taragolis changed the title Simplified Parsing of apprise_default Connection ID Simplified configuration parsing in Apprise Connection Feb 7, 2024
@dirrao
Copy link
Contributor

dirrao commented Feb 8, 2024

config

Does airflow consumers needs to update the environment variables without escaping the double quotes?

@infohash
Copy link
Contributor Author

infohash commented Feb 8, 2024

No, they don't have to which is why backward compatibility is there.

Until now, there was only one way:

$ export AIRFLOW_CONN_APPRISE_DEFAULT='{"extra": {"config": "{\"path\": \"https://hooks.slack.com/services/T1JJ3T3L2/A1BRTD4JD/TIiajkdnlazkcOXrIdevi7F\", \"tags\": \"alert\"}"}}'

But now, you can also set its env var like this:

$ export AIRFLOW_CONN_APPRISE_DEFAULT='{"extra": {"config": {"path": "https://hooks.slack.com/services/T1JJ3T3L2/A1BRTD4JD/TIiajkdnlazkcOXrIdevi7F", "tags": "alert"}}}'

It should have been like this from the start because airfow.models.Connection accepts JSON string in env var so there was never any need to escape nested JSON.

@hussein-awala hussein-awala merged commit 250f29f into apache:main Feb 8, 2024
56 checks passed
Copy link

boring-cyborg bot commented Feb 8, 2024

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

@hussein-awala
Copy link
Member

Congrats on your first commit 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Simplify Parsing of apprise_default Connection ID
5 participants