Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 BigQuery destination: always run GCS destination check for GCS staging method #13579

Merged
merged 8 commits into from
Jun 8, 2022

Conversation

tuliren
Copy link
Contributor

@tuliren tuliren commented Jun 7, 2022

What

How

  • Always run GCS destination check, even when the service account has all the required permissions.

🚨 User Impact 🚨

  • Currently users can enter the wrong HMAC keys and their BigQuery destination check will pass.
  • Once this PR is merged and published, this false positive result will be gone. The connection check will fail if the HMAC keys are incorrect.

@github-actions github-actions bot added the area/connectors Connector related issues label Jun 7, 2022
@tuliren tuliren changed the title Run gcs destination check all the time 🐛 BigQuery destination: always run GCS destination check for GCS staging method Jun 7, 2022
@tuliren tuliren requested a review from girarda June 7, 2022 21:59
@tuliren
Copy link
Contributor Author

tuliren commented Jun 7, 2022

/test connector=connectors/destination-bigquery

🕑 connectors/destination-bigquery https://github.com/airbytehq/airbyte/actions/runs/2457854072
✅ connectors/destination-bigquery https://github.com/airbytehq/airbyte/actions/runs/2457854072
Python tests coverage:

Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
normalization/transform_config/__init__.py                            2      0   100%
normalization/transform_catalog/reserved_keywords.py                 13      0   100%
normalization/transform_catalog/__init__.py                           2      0   100%
normalization/destination_type.py                                    13      0   100%
normalization/__init__.py                                             4      0   100%
normalization/transform_catalog/destination_name_transformer.py     155      8    95%
normalization/transform_config/transform.py                         159     31    81%
normalization/transform_catalog/table_name_registry.py              174     34    80%
normalization/transform_catalog/utils.py                             38      9    76%
normalization/transform_catalog/dbt_macro.py                         22      7    68%
normalization/transform_catalog/catalog_processor.py                147     80    46%
normalization/transform_catalog/transform.py                         61     38    38%
normalization/transform_catalog/stream_processor.py                 543    352    35%
-------------------------------------------------------------------------------------
TOTAL                                                              1333    559    58%

Build Passed

Test summary info:

All Passed

@tuliren
Copy link
Contributor Author

tuliren commented Jun 7, 2022

/publish connector=connectors/destination-bigquery-denormalized

🕑 connectors/destination-bigquery-denormalized https://github.com/airbytehq/airbyte/actions/runs/2458114448
🚀 Successfully published connectors/destination-bigquery-denormalized
🚀 Auto-bumped version for connectors/destination-bigquery-denormalized
✅ connectors/destination-bigquery-denormalized https://github.com/airbytehq/airbyte/actions/runs/2458114448

@tuliren
Copy link
Contributor Author

tuliren commented Jun 7, 2022

/publish connector=connectors/destination-bigquery

🕑 connectors/destination-bigquery https://github.com/airbytehq/airbyte/actions/runs/2458115565
🚀 Successfully published connectors/destination-bigquery
🚀 Auto-bumped version for connectors/destination-bigquery
✅ connectors/destination-bigquery https://github.com/airbytehq/airbyte/actions/runs/2458115565

@octavia-squidington-iii octavia-squidington-iii temporarily deployed to more-secrets June 8, 2022 00:20 Inactive
@octavia-squidington-iii octavia-squidington-iii temporarily deployed to more-secrets June 8, 2022 00:24 Inactive
@tuliren tuliren merged commit e996c2c into master Jun 8, 2022
@tuliren tuliren deleted the liren/patch-bigquery-destination branch June 8, 2022 00:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants