bigquery: consider moving from getQueryResults to listTableData

[pongad]

The current query method uses getQueryResults RPC to read rows of the results. However, benchmark shows that listTableData run significantly faster. Experiments in Go shows getQueryResults take 70% longer.

The choice of RPC is an implementation detail and can be changed whenever. However, listTableData returns fewer fields. In Java, we decided to tweak the API surface to make it easier to change RPC later. Of course, the change to API needs to happen before GA.

Will PHP require any API change?

[pongad]

I believe this should be P1. If surface change is required, it needs to happen before GA.

[dwsupplee]

Hey @pongad,

It looks like the original direction we received from the BigQuery team was in fact to do exactly what you have suggested - however, that requirement was later dropped with a note stating getQueryResults is planned to be ported to one platform along with listTableData and that there would no longer be a required change. Do you know if this may have changed?

[pongad]

It's not a required change in the sense that things will still work if we don't change.

However, according to BigQuery team

getQueryResults will stay slower than listTableData for the near future

A part of this is somewhat fundamental: getQueryResults needs to do more work and therefore slower.

Let's imagine that

• the client surface heavily relies on fields available to getQueryResults but not listTableData
• we declare GA

We might want to make queries faster a few months from now, but then we can't since moving to listTableData would break people.

The P1 issue is to verify that we can swap the RPC later on. The actual swap doesn't need to happen right now.

Am I making any sense?

[dwsupplee]

Yes, makes sense. Thanks for the extra explanation :).

If we move to using listTableData, is it safe to assume we retain the initial getQueryResults call in order to have access to the schema? If so, I believe we require no surface changes.

[pongad]

No problem!

Yes, we can keep using getQueryResults initially. I think we need to do that to wait for query to finish anyway.

Thank you for checking. Since this no longer blocks GA, I'll move this to P2 since we'll probably want to move at some point.

[jdpedrie]

Moving this to the new Feature Request wiki page.