Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storage: Batch._do_request does not accept timeout kwarg #9951

Closed
kgriffs opened this issue Dec 10, 2019 · 1 comment
Closed

Storage: Batch._do_request does not accept timeout kwarg #9951

kgriffs opened this issue Dec 10, 2019 · 1 comment
Assignees
Labels
api: storage Issues related to the Cloud Storage API. type: process A process-related concern. May include testing, release, or the like.

Comments

@kgriffs
Copy link

kgriffs commented Dec 10, 2019

Environment details

Arch Linux (Docker; tag 20191205; https://hub.docker.com/_/archlinux)
PyPy3 7.2
google-cloud-storage-1.23.0

Steps to reproduce

  1. Use the batch() context manager to create a batch of blob deletes
  2. Execute the batch

Code example

def delete_all_by_prefix(self, prefix):
    blobs_to_delete = [blob for blob in self._bucket.list_blobs(prefix=prefix)]
    if not blobs_to_delete:
        return

    batch_size = 500
    for i in range(0, len(blobs_to_delete), batch_size):
        batch = blobs_to_delete[i:(i + batch_size)]

        with self._client.batch():
            for b in batch:
                b.delete()

Stack trace

Traceback (most recent call last):

  ...

  File "/opt/pypy3/site-packages/google/cloud/storage/blob.py", line 554, in delete
    self.name, client=client, generation=self.generation
  File "/opt/pypy3/site-packages/google/cloud/storage/bucket.py", line 1100, in delete_blob
    _target_object=None,
  File "/opt/pypy3/site-packages/google/cloud/_http.py", line 417, in api_request
    timeout=timeout,
  File "/opt/pypy3/site-packages/google/cloud/_http.py", line 275, in _make_request
    method, url, headers, data, target_object, timeout=timeout
    TypeError: _do_request() got an unexpected keyword argument 'timeout'

Other info

I was able to fix the issue by monkey-patching google.cloud.storage.batch.Batch._do_request.

@emar-kar emar-kar added api: storage Issues related to the Cloud Storage API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. type: process A process-related concern. May include testing, release, or the like. and removed type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. labels Dec 10, 2019
@HemangChothani HemangChothani self-assigned this Dec 10, 2019
@HemangChothani
Copy link
Contributor

Duplicate issue of 10009 and resolved in PR 10010

atalyaalon added a commit to data-for-change/anyway that referenced this issue Mar 6, 2020
@sagivharel - fix for timeout parameter: we had to make sure the google-cloud-core version and google-api-core versions are updated
(issue: googleapis/google-cloud-python#9951 pr: googleapis/google-cloud-python#10010)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the Cloud Storage API. type: process A process-related concern. May include testing, release, or the like.
Projects
None yet
3 participants