Suppress jaydebeapi.Error when setAutoCommit or getAutoCommit is unsupported by JDBC driver #38707
Conversation
…it is being invoked as some JDBC drivers don't support this operation
…ying the exception to be suppressed and log a warning
There was a problem hiding this comment.
There is a problem with back-compatibility. Our providers work for Airflow >= 2.6.0 - see the errrors raised in tests - it cannot be imported from Airlfow until we keep 2.10.0 support - you will need to duplicate it in the provider and fall-back to it and mark it as "remove after min-airflow version is set to 2.10.
Another option (probably better) is to add it to common.sql and use from there (and add common.sql >= NEXT MINOR VERSION in provider.yaml.
Yes I saw that and I have same issue I think with MSGraphOperator but I don't fully understand how I can fix this :( |
Simply - You can't use code from For SQL providers, a simpler way around it is to add a code to common.sql as a new feature and use There was a discussion to have |
Ok got it, thank you for the explantion, indeed I saw that discussion of the common.util. At the moment I just can move that suppress_and_warn context manager to the sql provider as it isn't used elsewhere, but I put it in airflow itself which indeed started the dependency issue. We can always move it afterwards if it's needed elsewhere, like for example to a common util. |
Ok thanks @potiuk will do that 👍 Conflict has already been resolved |
|
@potiuk You're sure I need to update the dependency in all providers? Because I checked and I see not all providers depend on the same version. I just ask to be sure so I don't break any provider. |
|
Yes. But don't do it yet indeed. We have no policy for that and this is something I just realised we will need to fix - becuase some of the providers might depend on features that were released since their min version. In most installation they already have latest common.sql because it is set like that in constraints in the new |
…sql dependency to be at least 1.12.0 so the jdbc provider is able to use the newly added suppress_and_warn method" This reverts commit 569ef2c.
….0 to 2.7.0 for microsoft-azure provider in provider_dependencies.json
|
Also |
…from 2.6.0 to 2.7.0 for microsoft-azure provider in provider_dependencies.json" This reverts commit 63982cb.
Yes I thought I reverted that one as I've adapted it in the wrong branch, it's reverted now |
We experienced unexpected exceptions being raised by the JdbcHook as it is invoking the setAutoCommit and getAutoCommit methods on the JDBC driver, but in our case as we are using the SAS JDBC driver, those methods throw an UnsupportedOperationException in Java. Even though those methods make part of the JDBC spec, it isn't always implemented by the driver hence why the exception. So to avoid those exceptions to be propagated, it's better to suppress them.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.