BigQuery API Deprecation Warning

[drewmcdonald]

Describe the feature

The BigQueryConnectionManager.dataset method is firing a deprecation warning due to its call to bigquery.Client.dataset:

[...]
2021-12-30 18:02:49.877082 (ThreadPoolExecutor-1_9): Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
[...]

From the current API docs:

Client.dataset
Deprecated since version 1.24.0: Construct a DatasetReference using its constructor or use a string where previously a reference object was used.

Suggested Work

Given this and other updates to the BigQuery API, I think it would be healthy to do two things:

• Simplify BigQueryConnectionManager.dataset to directly create a bigquery.DatasetReference.
  • optionally, rename this method to dataset_ref to be clear about what's being returned
• Simplify BigQueryConnectionManager.table_ref to directly create a bigquery.TableReference; make it a static method.

With these changes, we can quit passing a conn around just to deliver a reference, and we'll meet (my interpretation of) BQ API best practice by working with unconnected references where possible and deferring 'connected' operations to a client.

Describe alternatives you've considered

1. do nothing / kick the can down the road on this deprecation
2. fix only the call triggering the deprecation warning, leave the rest as-is
3. cry

Additional context

n/a

Who will this benefit?

This will benefit maintainers by simplifying the signatures of two core methods.

Are you interested in contributing this feature?

Put me in, coach!

[McKnight-42]

@drewmcdonald thank you for pointing this out seems like a fantastic idea and would love to be of any help I can be on this as you work on it.