fix: use bigquery_client for gcs_stage methods #70
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When using
gcs_stage
method,self.client
attribute ofBaseBigQuerySink
is actually astorage.Client
instance, notbigquery.Client
instance. This makes theclean_up
fail ifoverwrite
orupsert
is used, becausestorage.Client
has noquery
method.
The solution here was to create a
bigquery.Client
using the existing factory when needed.The downside of this is that the bq client may actually be needed elsewhere, also it could be reused but instead
we create a new instance for each table.
I'll let you consider whether to pull or close and consider this a hotfix.
Closes #69, IMO.