Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bigquery: job.wait inner backoff configuration is to slow #10555

Closed
idan3 opened this issue Jul 16, 2024 · 0 comments · Fixed by #10558
Closed

bigquery: job.wait inner backoff configuration is to slow #10555

idan3 opened this issue Jul 16, 2024 · 0 comments · Fixed by #10558
Assignees
Labels
api: bigquery Issues related to the BigQuery API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@idan3
Copy link

idan3 commented Jul 16, 2024

I'm using job.wait to determine when the job is complete. The internal configuration for the job.wait function is located within waitForQuery and is as follows:

backoff := gax.Backoff{
		Initial:    1 * time.Second,
		Multiplier: 2,
		Max:        60 * time.Second,
	}

It is a significant problem because the initial delay of 1 second is too slow when performing a large number of queries at a high rate. Can you please fix this or provide an option to modify this configuration?

Something like :

backoff := gax.Backoff{
		Initial:    80 * time.Millisecond,
		Multiplier: 1.1,
		Max:        60 * time.Second,
	}
@idan3 idan3 added the triage me I really want to be triaged. label Jul 16, 2024
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the BigQuery API. label Jul 16, 2024
@alvarowolfx alvarowolfx self-assigned this Jul 16, 2024
@alvarowolfx alvarowolfx added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. priority: p2 Moderately-important priority. Fix may not be included in next release. and removed triage me I really want to be triaged. labels Jul 16, 2024
shollyman added a commit to shollyman/google-cloud-go that referenced this issue Jul 17, 2024
This PR reduces the initial backoff used during job polling from 1s to
50ms.  In practice most time is spent in the hanging poll on the server
side in GetQueryResults, so this reduces opportunities for wasted
waiting.

Fixes: googleapis#10555
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the BigQuery API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants