BigQuery: Make RowIterator._to_dataframe_iterable public

[tswast]

Follow-up to #7293

There are use cases where it's not ideal to download a whole table at once, but a pandas DataFrame is still desired.

Potential solution:

Make the pages member of RowIterator return a generator of a subclass of google.api_core.page_iterator.Page with a to_dataframe() method.

[agsha]

Any updates on this?

[tswast]

We have a private method RowIterator._to_dataframe_iterable that we could make public.

google-cloud-python/bigquery/google/cloud/bigquery/table.py

Line 1557 in 13bd849

def _to_dataframe_iterable(self, bqstorage_client=None, dtypes=None):

When I originally filed this FR, I thought a per-page to_dataframe would be more appropriate, but it's actually quite difficult to override the page class with how the superclasses are structured. Plus, it wouldn't be able to take advantage of the BQ Storage API if we used the pages property.

[tswast]

@HemangChothani I think this is a good task you all. #7339 (comment)

In addition to making it public, let's make sure we have some unit tests for using the function directly.