-
Notifications
You must be signed in to change notification settings - Fork 14.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add init_command
parameter to MySqlHook
#33359
Conversation
I have ran the tests and have tested a DAG locally with a MySQL connection created with the extra set to {
"init_command": "SET time_zone = '+00:00';"
} and a task to output the SQLExecuteQueryOperator(
task_id='check_time_zone',
conn_id='mysql_default',
sql=r"""SELECT @@SESSION.time_zone;""",
autocommit=True,
show_return_value_in_logs=True
) with the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Main reason we are not adding all the potential options to extras is potential problem with security.
Allowing to execute arbitrary command via "extra" is potential security risk. This gives the user who can edit connections more priviliedges than needed. We had a number of security vulnerabilities, where we had to remove features like that from connection extras.
While we have modified our security model description https://airflow.apache.org/docs/apache-airflow/stable/security/index.html and hopefully informed our users that users who have access to edit connection can have more priviledges, adding a new option for that knowingly is a bad idea.
The "extra" parameters in ideal world should only be limited to declarative parameters, that merrely configure options of the underlying driver. passing arbitrary code to execute is a no-go.
It's quite ok to add an init_command when you initilalize the Hook - and with sql operator you can pass such parameter via hook_params, so that DAG authors could modify it. so if you can chang e it so that init command canot be passed by extra, but by hook parameter, then I'd be perfectly fine with accepting such PR.
Thank you for the explanation. I see your point and will revise this PR to turn this into a hook parameter. |
b8bccd6
to
8fcc169
Compare
Ok, this is changed to a |
Oh, I didn't even know about the |
8fcc169
to
ea17f4d
Compare
8fdf187
to
43bd7e1
Compare
43bd7e1
to
f388dbe
Compare
@potiuk this is now ready for review again |
f388dbe
to
a0ee2b3
Compare
init_command
parameter to MySqlHook
This allows the addition of `init_command` as a `MySqlHook` parameter. For example, to set the MySQL session's `time_zone` to UTC you can set the `init_command` to `SET time_zone = '+00:00';`.
a0ee2b3
to
749f322
Compare
@potiuk it still says "change requested" although I already made the requested change, can you review it again? |
Cool. Just got to it now. Please exercise a bit of patience next time. No need to hurry things like that. |
This allows the addition of
init_command
as aMySqlHook
parameter.init_command
is a connection argument to set an initial command to issue to the MySQL server upon connection. It is available as both amysqlclient
connection attribute (https://mysqlclient.readthedocs.io/user_guide.html?highlight=init_command#functions-and-attributes) and amysql-connector-python
connection attribute (https://dev.mysql.com/doc/connector-python/en/connector-python-connectargs.html).For example, to set the session's
time_zone
to UTC you can set in aMySqlHook
aninit_command
parameter as shown:or in a SQL operator shown using
hook_params
(which should return "+00:00"):closes: #33300
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in newsfragments.