Broken Pipe exceptions raised after idle periods #218

tbonfort · 2016-04-18T11:41:39Z

I'm using long-lived client objects like so:

client = discovery.build(....)
task = client.get_something(....).execute(num_retries=5)
some_long_running_process(task)
client.do_another_thing().execute(num_retries=5)

And running into exceptions on the calls to do_another_thing():

Traceback (most recent call last):
  File "/opt/app/gcp_helper.py", line 168, in get_size
    instanceGroupManager=instance_group).execute(num_retries=5)
  File "/opt/venv/local/lib/python2.7/site-packages/oauth2client/util.py", line 137, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/opt/venv/local/lib/python2.7/site-packages/googleapiclient/http.py", line 755, in execute
    method=str(self.method), body=self.body, headers=self.headers)
  File "/opt/venv/local/lib/python2.7/site-packages/googleapiclient/http.py", line 93, in _retry_request
    resp, content = http.request(uri, method, *args, **kwargs)
  File "/opt/venv/local/lib/python2.7/site-packages/oauth2client/client.py", line 633, in new_request
    self._refresh(request_orig)
  File "/opt/venv/local/lib/python2.7/site-packages/oauth2client/contrib/gce.py", line 129, in _refresh
    META, headers={'Metadata-Flavor': 'Google'})
  File "/opt/venv/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1609, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/opt/venv/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1351, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/opt/venv/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1273, in _conn_request
    conn.request(method, request_uri, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 1001, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 1035, in _send_request
    self.endheaders(body)
  File "/usr/lib/python2.7/httplib.py", line 997, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 850, in _send_output
    self.send(msg)
  File "/usr/lib/python2.7/httplib.py", line 826, in send
    self.sock.sendall(data)
  File "/usr/lib/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
error: [Errno 32] Broken pipe"

I would expect the num_retries argument to handle these exceptions in https://github.com/google/google-api-python-client/blob/master/googleapiclient/http.py#L145, but for the time being I have to implement my own retry logic around all API calls to overcome this.

The text was updated successfully, but these errors were encountered:

theacodes · 2016-04-18T17:32:36Z

It's likely that c6425a0 fixed this, but we haven't cut a new release. Can you try the HEAD version and see if you still see this issue?

tbonfort · 2016-04-18T20:03:36Z

Still happening on master:

Traceback (most recent call last):
  File "/opt/app/gcp_helper.py", line 168, in get_size
    instanceGroupManager=instance_group).execute(num_retries=5)
  File "/opt/venv/local/lib/python2.7/site-packages/oauth2client/util.py", line 135, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/opt/venv/local/lib/python2.7/site-packages/googleapiclient/http.py", line 820, in execute
    method=str(self.method), body=self.body, headers=self.headers)
  File "/opt/venv/local/lib/python2.7/site-packages/googleapiclient/http.py", line 147, in _retry_request
    resp, content = http.request(uri, method, *args, **kwargs)
  File "/opt/venv/local/lib/python2.7/site-packages/oauth2client/client.py", line 632, in new_request
    self._refresh(request_orig)
  File "/opt/venv/local/lib/python2.7/site-packages/oauth2client/contrib/gce.py", line 129, in _refresh
    META, headers={'Metadata-Flavor': 'Google'})
  File "/opt/venv/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1609, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/opt/venv/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1351, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/opt/venv/local/lib/python2.7/site-packages/httplib2/__init__.py", line 1273, in _conn_request
    conn.request(method, request_uri, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 1001, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 1035, in _send_request
    self.endheaders(body)
  File "/usr/lib/python2.7/httplib.py", line 997, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 850, in _send_output
    self.send(msg)
  File "/usr/lib/python2.7/httplib.py", line 826, in send
    self.sock.sendall(data)
  File "/usr/lib/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
error: [Errno 32] Broken pipe"

theacodes · 2016-04-18T20:54:41Z

Is that a socket.error or an IO/OSError? Seems to be an IOError - we may just have to add some additional logic here. If you want to make a PR, I'll be happy to review it.

nathanielmanistaatgoogle · 2016-04-28T20:32:51Z

Fixed in 713b1de.

kapi90 · 2016-05-03T21:01:43Z

Hi, I'm using the Cloud-Vision API in python with similar conditions (a long-lived object and long idle sessions between requests)

I get the same error message a lot, therefore I wonder if this could be the same problem mentioned above, so pulling the latest commit should fix it.

I copy the related part from traceback here:

in detect_text responses = request.execute(num_retries=num_retries) File "/opt/python/run/venv/lib/python2.7/site-packages/oauth2client/util.py", line 135, in positional_wrapper return wrapped(_args, *_kwargs) File "/opt/python/run/venv/lib/python2.7/site-packages/googleapiclient/http.py", line 755, in execute method=str(self.method), body=self.body, headers=self.headers) File "/opt/python/run/venv/lib/python2.7/site-packages/googleapiclient/http.py", line 93, in _retry_request resp, content = http.request(uri, method, _args, *_kwargs) File "/opt/python/run/venv/lib/python2.7/site-packages/oauth2client/client.py", line 621, in new_request redirections, connection_type) File "/opt/python/run/venv/lib/python2.7/site-packages/httplib2/init.py", line 1609, in request (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey) File "/opt/python/run/venv/lib/python2.7/site-packages/httplib2/init.py", line 1351, in _request (response, content) = self._conn_request(conn, request_uri, method, body, headers) File "/opt/python/run/venv/lib/python2.7/site-packages/httplib2/init.py", line 1273, in _conn_request conn.request(method, request_uri, body, headers) File "/usr/lib64/python2.7/httplib.py", line 1001, in request self._send_request(method, url, body, headers) File "/usr/lib64/python2.7/httplib.py", line 1035, in _send_request self.endheaders(body) File "/usr/lib64/python2.7/httplib.py", line 997, in endheaders self._send_output(message_body) File "/usr/lib64/python2.7/httplib.py", line 850, in _send_output self.send(msg) File "/usr/lib64/python2.7/httplib.py", line 826, in send self.sock.sendall(data) File "/usr/lib64/python2.7/ssl.py", line 701, in sendall v = self.send(data[count:]) File "/usr/lib64/python2.7/ssl.py", line 667, in send v = self._sslobj.write(data) error: [Errno 32] Broken pipe

theacodes · 2016-05-03T21:05:20Z

Let us know if it fixes you.

@nathanielmanistaatgoogle should we cut another minor release?

nathanielmanistaatgoogle · 2016-05-04T14:47:49Z

@jonparrott Probably! Have you any time in the next few days?

theacodes · 2016-05-04T16:51:03Z

Maybe Friday. I'll make a bug for it.

klday · 2016-08-24T18:29:59Z

I'm having this issue using the google bigquery API. I'm pulling data from Azure SQL Server in batches and sending to bigquery. I have to use batches with some of the tables because of their size. Because of filtering, ordering and batching of some of the tables, the pulls from Azure take longer than pulls from other tables. I'm getting the broken pipe error on tables with longer pulls.

I've tried several different batch sizes with the same result.

Any advice would be greatly appreciated.

theacodes · 2016-08-24T18:31:46Z

@klday are you experiencing this with the latest release?

klday · 2016-08-24T18:32:02Z

Sorry, I left out some important information:

I'm using pandas to_gbq() to import the data to BigQuery.

Here is my terminal output for the traceback:
Streaming Insert is 100.0% Complete
Traceback (most recent call last):
File "datascience/app_sql_to_bq/main.py", line 146, in
logging_level = args.logging_level)
File "datascience/app_sql_to_bq/main.py", line 83, in main
table_time = bq.sql_data_to_bq_table(sql_resource,bq_resource,sql_table_name,bq_table_id,sql_schema_columns,sql_query,sql_columns,sql_filtered_size_query,step_size=step_size)
File "./datascience/utils/bigqueryUtils.py", line 630, in sql_data_to_bq_table
clean_df.to_gbq(bq_dataset_table_id, bq_project_id, if_exists='append', private_key=key)
File "/opt/conda/lib/python3.5/site-packages/pandas/core/frame.py", line 899, in to_gbq
if_exists=if_exists, private_key=private_key)
File "/opt/conda/lib/python3.5/site-packages/pandas/io/gbq.py", line 724, in to_gbq
connector.load_data(dataframe, dataset_id, table_id, chunksize)
File "/opt/conda/lib/python3.5/site-packages/pandas/io/gbq.py", line 453, in load_data
body=body).execute()
File "/opt/conda/lib/python3.5/site-packages/oauth2client/util.py", line 137, in positional_wrapper
return wrapped(_args, *_kwargs)
File "/opt/conda/lib/python3.5/site-packages/googleapiclient/http.py", line 833, in execute
method=str(self.method), body=self.body, headers=self.headers)
File "/opt/conda/lib/python3.5/site-packages/googleapiclient/http.py", line 173, in _retry_request
raise exception
File "/opt/conda/lib/python3.5/site-packages/googleapiclient/http.py", line 160, in _retry_request
resp, content = http.request(uri, method, _args, *_kwargs)
File "/opt/conda/lib/python3.5/site-packages/oauth2client/transport.py", line 169, in new_request
redirections, connection_type)
File "/opt/conda/lib/python3.5/site-packages/httplib2/init.py", line 1314, in request
(response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
File "/opt/conda/lib/python3.5/site-packages/httplib2/init.py", line 1064, in _request
(response, content) = self._conn_request(conn, request_uri, method, body, headers)
File "/opt/conda/lib/python3.5/site-packages/httplib2/init.py", line 988, in _conn_request
conn.request(method, request_uri, body, headers)
File "/opt/conda/lib/python3.5/http/client.py", line 1106, in request
self._send_request(method, url, body, headers)
File "/opt/conda/lib/python3.5/http/client.py", line 1151, in _send_request
self.endheaders(body)
File "/opt/conda/lib/python3.5/http/client.py", line 1102, in endheaders
self._send_output(message_body)
File "/opt/conda/lib/python3.5/http/client.py", line 936, in _send_output
self.send(message_body)
File "/opt/conda/lib/python3.5/http/client.py", line 908, in send
self.sock.sendall(data)
File "/opt/conda/lib/python3.5/ssl.py", line 891, in sendall
v = self.send(data[count:])
File "/opt/conda/lib/python3.5/ssl.py", line 861, in send
return self._sslobj.write(data)
File "/opt/conda/lib/python3.5/ssl.py", line 586, in write
return self._sslobj.write(data)
BrokenPipeError: [Errno 32] Broken pipe

klday · 2016-08-24T21:05:05Z

The latest release of ...?
I just installed the api tools so I believe I am using the latest release of those.

theacodes · 2016-08-24T21:05:43Z

This library, 1.5.2?

klday · 2016-08-24T21:07:40Z

Honestly, I have no idea. I just downloaded the google api tools for the cloud platform. Those are the most recent tools. And, I installed those using the command line. I'm not sure what library you're referring to, but I'm also relatively new to programming.

theacodes · 2016-08-24T21:13:24Z

@klday no worries. Can you open up python and run the following and give me the output?

>>> import googleapiclient
>>> googleapiclient.__version__
'1.5.1'
>>> import oauth2client
>>> print oauth2client.__version__
2.2.0

klday · 2016-08-24T21:33:31Z

Yes, thank you. The output is:

import googleapiclient
googleapiclient.version
'1.5.2'
import oauth2client
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named 'oauthclient'

I am working in a docker container. I have been creating a google cloud resource to work with the bigquery api tools using a key (pretty sure I'm explaining that correctly).

theacodes · 2016-08-24T21:36:21Z

well, in that case I don't know. It could be an underlying issue with httplib2.

In which case you can try using httplib2shim to see if it fixes it.

klday · 2016-08-24T21:36:45Z

Ok, thank you. I'll give that a shot.

LordAro · 2016-10-23T17:41:31Z

FWIW, I just got this with 1.5.3 (From FreeBSD 10.1, python 3.5.1 if that matters). I guess there's no progress with it?

Have updated to 1.5.4 (w/ oauth2client 4.0.0) now, will see what happens

theacodes · 2016-10-24T16:35:27Z

@LordAro this should definitely be occurring less often. If you can provide a full stacktrace that will be helpful.

LordAro · 2016-10-24T19:56:00Z

Sure

[2016/10/23 18:31:15] (I:googleapiclient.discovery) URL being requested: GET https://www.googleapis.com/youtube/v3/videos?id=5_LxyhCJp
sM&part=snippet%2CcontentDetails%2Cstatistics&alt=json&key=<snip>
[2016/10/23 18:31:15] (E:asyncio) Unhandled exception in event task
future: <Task finished coro=<LinkInfo.scan_privmsg() done, defined at /home/urybot/.pyenv/versions/3.5.1/lib/python3.5/asyncio/corouti
nes.py:204> exception=BrokenPipeError(32, 'Broken pipe')>
Traceback (most recent call last):
  File "/home/urybot/.pyenv/versions/3.5.1/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "/home/urybot/.pyenv/versions/3.5.1/lib/python3.5/asyncio/coroutines.py", line 206, in coro
    res = func(*args, **kw)
  File "/home/urybot/csbot/csbot/plugins/linkinfo.py", line 155, in scan_privmsg
    result = self.get_link_info(part)
  File "/home/urybot/csbot/csbot/plugins/linkinfo.py", line 189, in get_link_info
    result = h.handler(url, match)
  File "/home/urybot/csbot/csbot/plugins/youtube.py", line 153, in page_handler
    response = self._yt(url)
  File "/home/urybot/csbot/csbot/plugins/youtube.py", line 86, in _yt
    json = self.get_video_json(vid_id)
  File "/home/urybot/csbot/csbot/plugins/youtube.py", line 73, in get_video_json
    response = self.client.videos().list(id=id, part='snippet,contentDetails,statistics').execute(http=self.http)
  File "/home/urybot/.pyenv/versions/3.5.1/lib/python3.5/site-packages/oauth2client/util.py", line 137, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/home/urybot/.pyenv/versions/3.5.1/lib/python3.5/site-packages/googleapiclient/http.py", line 833, in execute
    method=str(self.method), body=self.body, headers=self.headers)
  File "/home/urybot/.pyenv/versions/3.5.1/lib/python3.5/site-packages/googleapiclient/http.py", line 173, in _retry_request
    raise exception
  File "/home/urybot/.pyenv/versions/3.5.1/lib/python3.5/site-packages/googleapiclient/http.py", line 160, in _retry_request
    resp, content = http.request(uri, method, *args, **kwargs)
  File "/home/urybot/.pyenv/versions/3.5.1/lib/python3.5/site-packages/httplib2/__init__.py", line 1314, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/home/urybot/.pyenv/versions/3.5.1/lib/python3.5/site-packages/httplib2/__init__.py", line 1064, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/home/urybot/.pyenv/versions/3.5.1/lib/python3.5/site-packages/httplib2/__init__.py", line 988, in _conn_request
    conn.request(method, request_uri, body, headers)
  File "/home/urybot/.pyenv/versions/3.5.1/lib/python3.5/http/client.py", line 1083, in request
    self._send_request(method, url, body, headers)
  File "/home/urybot/.pyenv/versions/3.5.1/lib/python3.5/http/client.py", line 1128, in _send_request
    self.endheaders(body)
  File "/home/urybot/.pyenv/versions/3.5.1/lib/python3.5/http/client.py", line 1079, in endheaders
    self._send_output(message_body)
  File "/home/urybot/.pyenv/versions/3.5.1/lib/python3.5/http/client.py", line 911, in _send_output
    self.send(msg)
  File "/home/urybot/.pyenv/versions/3.5.1/lib/python3.5/http/client.py", line 885, in send
    self.sock.sendall(data)
  File "/home/urybot/.pyenv/versions/3.5.1/lib/python3.5/ssl.py", line 886, in sendall
    v = self.send(data[count:])
  File "/home/urybot/.pyenv/versions/3.5.1/lib/python3.5/ssl.py", line 856, in send
    return self._sslobj.write(data)
  File "/home/urybot/.pyenv/versions/3.5.1/lib/python3.5/ssl.py", line 581, in write
    return self._sslobj.write(data)
BrokenPipeError: [Errno 32] Broken pipe

Bobspadger · 2016-10-25T15:09:52Z

I'm also getting the broken pipe on BatchHttpRequest()

I can get through thousands of calls to the shopping api, but if we have a quiet period, then I get the broken pipe.

theacodes · 2016-10-26T16:59:24Z

This issue is most likely due to httplib2's connection pooling logic. I'd suggest trying httplib2shim.

Bobspadger · 2016-11-02T16:04:18Z

@jonparrott currently testing this and it seems to have fixed the problem

Is it worth making this the required method for connecting ?

theacodes · 2016-11-02T16:26:39Z

@Bobspadger probably not, but we're soon replacing oauth2client in this library with google-auth, which will allow us to use urllib3 and requests without needing httplib2shim.

Bobspadger · 2016-11-03T11:31:58Z

@jonparrott excellent, requests makes life so much easier!

dhyon · 2016-11-14T19:52:56Z

@jonparrott Any timeline on this replacement endeavor? It seems like a great idea and this bug is truly annoying.

SkYNewZ · 2020-02-06T14:31:14Z

Always the issue

Bobspadger · 2020-02-07T08:57:59Z

This broken pipe issue has occurred again for me now I'm moving to googles v2.1 content API and i'm using a service account (no longer httplib2shim as I've followed the instructions from your example and using all the google libraries.

Is the answer to go back to using httplib2shim.

I too have a long life client - does this need rebuilding before every run ?

SkYNewZ · 2020-02-17T16:25:24Z

?

ianseyer · 2020-05-26T16:06:19Z

This is now happening to me via a Cloud Run execution that happens every 30 minutes. What's the status of this fix?

Why was the issue closed when there is no solution or fix in place?

HaririAli · 2020-06-20T22:10:45Z

Hi,

I'm having the same issue via Cloud Run also. I randomly get "Broken Pipe" or "connection reset by peer" errors.
I am using the latest version of the Google API client so it seems this has not been fixed yet.

HaririAli · 2020-06-21T13:34:24Z

OK this is just a FYI in case someone is having the same problem. I am getting the Broken Pipe error while using Firebase messaging via Cloud Run.
As @theacodes mentioned, this has been fixed in google-cloud-python, which includes APIs for many cloud services, but it doesn't include an API for Firebase messaging.
It seems that this issue will not be fixed soon for Firebase messaging, so try PyFCM. It is based on requests module, so I assume it should not have this issue.

Fix for exception: See also googleapis/google-api-python-client#218 (comment) Traceback (most recent call last): File "/work/src/experiment/scheduler.py", line 602, in schedule_loop trial_instance_manager.handle_preempted_trials() File "/work/src/experiment/scheduler.py", line 538, in handle_preempted_trials if not delete_instances(instances, self.experiment_config): File "/work/src/experiment/scheduler.py", line 112, in delete_instances i for i in gce.get_instances(cloud_project, cloud_compute_zone) File "/work/src/experiment/scheduler.py", line 112, in <listcomp> i for i in gce.get_instances(cloud_project, cloud_compute_zone) File "/work/src/common/gce.py", line 46, in get_instances for instance in _get_instance_items(project, zone): File "/work/src/common/gce.py", line 37, in _get_instance_items response = request.execute() File "/work/.venv/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper return wrapped(*args, **kwargs) File "/work/.venv/lib/python3.7/site-packages/googleapiclient/http.py", line 901, in execute headers=self.headers, File "/work/.venv/lib/python3.7/site-packages/googleapiclient/http.py", line 204, in _retry_request raise exception File "/work/.venv/lib/python3.7/site-packages/googleapiclient/http.py", line 177, in _retry_request resp, content = http.request(uri, method, *args, **kwargs) File "/work/.venv/lib/python3.7/site-packages/oauth2client/transport.py", line 186, in new_request credentials._refresh(orig_request_method) File "/work/.venv/lib/python3.7/site-packages/oauth2client/client.py", line 749, in _refresh self._do_refresh_request(http) File "/work/.venv/lib/python3.7/site-packages/oauth2client/client.py", line 780, in _do_refresh_request body=body, headers=headers) File "/work/.venv/lib/python3.7/site-packages/oauth2client/transport.py", line 282, in request connection_type=connection_type) File "/work/.venv/lib/python3.7/site-packages/httplib2/__init__.py", line 1994, in request cachekey, File "/work/.venv/lib/python3.7/site-packages/httplib2/__init__.py", line 1651, in _request conn, request_uri, method, body, headers File "/work/.venv/lib/python3.7/site-packages/httplib2/__init__.py", line 1558, in _conn_request conn.request(method, request_uri, body, headers) File "/usr/local/lib/python3.7/http/client.py", line 1252, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/local/lib/python3.7/http/client.py", line 1298, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/local/lib/python3.7/http/client.py", line 1247, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/local/lib/python3.7/http/client.py", line 1065, in _send_output self.send(chunk) File "/usr/local/lib/python3.7/http/client.py", line 987, in send self.sock.sendall(data) File "/usr/local/lib/python3.7/ssl.py", line 1034, in sendall v = self.send(byte_view[count:]) File "/usr/local/lib/python3.7/ssl.py", line 1003, in send return self._sslobj.write(data) BrokenPipeError: [Errno 32] Broken pipe Message: Error occurred during scheduling."

numbsafari · 2021-01-26T01:30:02Z

We are also experiencing this issue after upgrading to the latest version.

Would it be appropriate to re-open this ticket, or should I introduce a new one?

pilipenok · 2021-01-29T06:14:34Z

We have the same issue when periodically call status of long ai platform training job from CE instance.
Pipe is broken after 1 hour (+/- minute) of using discovery for ml.v1
google-api-python-client==1.9.1

p.s. update to google.auth (1.24.0) did not help.
The workaround is to recreate api client and retry on broken pipe exception

gmlwns2000 · 2021-02-04T05:42:22Z

We have the same issue when call a long ai platform training job from CE instance.
Pipe is broken after 1 hour (+/- minute) of using discovery for ml.v1
google-api-python-client==1.9.1

p.s. update to google.auth (1.24.0) did not help

I am encountered same issue too.

dkajtoch · 2021-02-04T11:42:08Z

The same for me with ml engine train

Ark-kun · 2021-05-29T01:15:20Z

Our customers are getting this error when polling DataProc.

Unfortunately, this cannot completely replace the outer retry loop, which addresses an issue for long running client due to googleapis/google-api-python-client#218 . Updated comments for that instead. PiperOrigin-RevId: 381029373

Unfortunately, this cannot completely replace the outer retry loop, which addresses an issue for long running client due to googleapis/google-api-python-client#218 . Updated comments for that instead. PiperOrigin-RevId: 381075453

lzzy12 · 2021-07-12T16:05:09Z

I get this with the Google drive API as well :/

gregsheu · 2021-09-16T18:46:05Z

I got this error as well and it has been almost a year. I believe I have all latest versions.
google-api-core 2.0.1
google-api-python-client 2.21.0
google-auth 2.1.0
google-auth-httplib2 0.1.0
google-auth-oauthlib 0.4.6
googleapis-common-protos 1.53.0

ItayVortex · 2022-02-17T12:31:46Z

I get the same problem when calling the compute api

  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/functions_framework/__init__.py", line 171, in view_func
    function(data, context)
  File "/workspace/main.py", line 73, in periodic_check
    status = compute.instances().get(project=project, zone=zone, instance=instance).execute()['status']
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/googleapiclient/_helpers.py", line 131, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/googleapiclient/http.py", line 922, in execute
    resp, content = _retry_request(
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/googleapiclient/http.py", line 221, in _retry_request
    raise exception
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/googleapiclient/http.py", line 190, in _retry_request
    resp, content = http.request(uri, method, *args, **kwargs)
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google_auth_httplib2.py", line 218, in request
    response, content = self.http.request(
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/httplib2/__init__.py", line 1725, in request
    (response, content) = self._request(
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/httplib2/__init__.py", line 1441, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/httplib2/__init__.py", line 1364, in _conn_request
    conn.request(method, request_uri, body, headers)
  File "/opt/python3.9/lib/python3.9/http/client.py", line 1285, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/opt/python3.9/lib/python3.9/http/client.py", line 1331, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/opt/python3.9/lib/python3.9/http/client.py", line 1280, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/opt/python3.9/lib/python3.9/http/client.py", line 1040, in _send_output
    self.send(msg)
  File "/opt/python3.9/lib/python3.9/http/client.py", line 1001, in send
    self.sock.sendall(data)
  File "/opt/python3.9/lib/python3.9/ssl.py", line 1204, in sendall
    v = self.send(byte_view[count:])
  File "/opt/python3.9/lib/python3.9/ssl.py", line 1173, in send
    return self._sslobj.write(data)
BrokenPipeError: [Errno 32] Broken pipe

honjiaxuan · 2022-04-15T05:45:32Z

I am having this error when using Python Gmail API.

parthea · 2023-06-01T18:33:20Z

If anyone is experiencing the Broken pipe error, please can you share sample code? I'm having difficulty re-creating the issue.

nathanielmanistaatgoogle closed this as completed Apr 28, 2016

Bobspadger mentioned this issue Oct 26, 2016

Broken Pipe issue appears to be back. #302

Closed

theacodes reopened this Oct 26, 2016

yoshi-automation added 🚨 This issue needs some love. triage me I really want to be triaged. labels Apr 7, 2020

thomasdesr mentioned this issue Jul 20, 2020

[autoscaler] Autoscaler prints (harmless) errors every 30 mins and then kills the workers in GCP cluster ray-project/ray#9368

Open

2 tasks

inferno-chromium mentioned this issue Sep 16, 2020

Fix for oauth2client exception, use google-auth library. google/fuzzbench#742

Merged

JustinBeckwith assigned nathanielmanistaatgoogle Feb 1, 2021

dkajtoch mentioned this issue Mar 5, 2021

fix(components): add bypass for BrokenPipeError in gcp ml engine client. Fixes #5085 kubeflow/pipelines#5250

Merged

2 tasks

igorgad mentioned this issue Jun 18, 2021

[autoscaler][gcp][default] Autoscaler throws BrokenPipeError on gcp. ray-project/ray#16072

Closed

2 tasks

copybara-service bot mentioned this issue Jun 23, 2021

Make get_job() retry on transient errors for CAIP/uCAIP training. tensorflow/tfx#3957

Merged

JoshuaFox mentioned this issue Mar 12, 2023

Known errors in Google API Client Library doitintl/iris3#19

Closed

Broken Pipe exceptions raised after idle periods #218

Broken Pipe exceptions raised after idle periods #218

Comments

tbonfort commented Apr 18, 2016

theacodes commented Apr 18, 2016

tbonfort commented Apr 18, 2016

theacodes commented Apr 18, 2016

nathanielmanistaatgoogle commented Apr 28, 2016

kapi90 commented May 3, 2016

theacodes commented May 3, 2016

nathanielmanistaatgoogle commented May 4, 2016

theacodes commented May 4, 2016

klday commented Aug 24, 2016

theacodes commented Aug 24, 2016

klday commented Aug 24, 2016

klday commented Aug 24, 2016 • edited Loading

theacodes commented Aug 24, 2016

klday commented Aug 24, 2016

theacodes commented Aug 24, 2016

klday commented Aug 24, 2016

theacodes commented Aug 24, 2016

klday commented Aug 24, 2016

LordAro commented Oct 23, 2016 • edited Loading

theacodes commented Oct 24, 2016

LordAro commented Oct 24, 2016

Bobspadger commented Oct 25, 2016

theacodes commented Oct 26, 2016

Bobspadger commented Nov 2, 2016

theacodes commented Nov 2, 2016 • edited Loading

Bobspadger commented Nov 3, 2016

dhyon commented Nov 14, 2016

SkYNewZ commented Feb 6, 2020

Bobspadger commented Feb 7, 2020

SkYNewZ commented Feb 17, 2020

ianseyer commented May 26, 2020 • edited Loading

HaririAli commented Jun 20, 2020

HaririAli commented Jun 21, 2020 • edited Loading

numbsafari commented Jan 26, 2021

pilipenok commented Jan 29, 2021 • edited Loading

gmlwns2000 commented Feb 4, 2021

dkajtoch commented Feb 4, 2021

Ark-kun commented May 29, 2021

lzzy12 commented Jul 12, 2021

gregsheu commented Sep 16, 2021

ItayVortex commented Feb 17, 2022

honjiaxuan commented Apr 15, 2022

parthea commented Jun 1, 2023

klday commented Aug 24, 2016 •

edited

Loading

LordAro commented Oct 23, 2016 •

edited

Loading

theacodes commented Nov 2, 2016 •

edited

Loading

ianseyer commented May 26, 2020 •

edited

Loading

HaririAli commented Jun 21, 2020 •

edited

Loading

pilipenok commented Jan 29, 2021 •

edited

Loading