BigQueryCheckOperator skips value and error check when in deferrable mode but not deferred #37885
Closed
2 tasks done
Labels
area:providers
good first issue
kind:bug
This is a clearly a bug
provider:google
Google (including GCP) related issues
Apache Airflow Provider(s)
google
Versions of Apache Airflow Providers
10.2.0 and above
Apache Airflow version
main branch
Operating System
MacOS
Deployment
Other
Deployment details
BREEZE
What happened
This PR optimized the defferable mode in
BigQueryCheckOperator
and introduced this bug. When the operator is in defferable mode, but the job finishes quickly enough and is not deffered (it does not fall intoif job.running()
condition), no error is raised when the job fails and no value check is performed.Similar optimization has been made here and then was fixed here.
What you think should happen instead
Operator should raise an error when job fails and check te value returned by the job, even without being deffered.
I think something like this similar to what was made in
BigQueryValueCheckOperator
should be enough:How to reproduce
Run a really quick query in defferable mode, if it fails no error will be raised, if not, the values will not be checked.
Anything else
I'll make a PR in free time, but maybe somebody will pick it up by then.
Also, there are no tests checking that operator (both CheckOperator and CheckValueOperator) raised an error, when in defferable mode but not deffered, so something like this should be added:
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: