[BUG] Occasional (but disruptive) connection failure errors

[pradh]

The code snippet below runs into connection failures on my linux machine (and Fiona's too), but runs fine on colab. Not yet sure if its a lib/network issue.

import json                                                                      
import requests                                                                  
                                                                                 
resp = requests.get('https://api.datacommons.org/node/places-in?dcids=geoId/06&placeType=CensusTract')
ans = json.loads(json.loads(resp.content)['payload'])                            
for pair in ans:                                                                 
  resp = requests.get('https://api.datacommons.org/node/property-values?dcids=' + pair['place'] + '&property=geoJsonCoordinates')
  ans = json.loads(json.loads(resp.content)['payload'])                          
  if pair['place'] not in ans or 'out' not in ans[pair['place']]: continue       
  for val in ans[pair['place']]['out']:                                          
    print(val['value'])

[pradh]

Error message:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 159, in _new_conn
    conn = connection.create_connection(
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 308, in connect
    conn = self._new_conn()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 171, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f8107a4fd90>: Failed to establish a new connection: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 724, in urlopen
    retries = retries.increment(
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 439, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.datacommons.org', port=443): Max retries exceeded with url: /node/property-values?dcids=geoId/06001406201&property=geoJsonCoordinates (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8107a4fd90>: Failed to establish a new connection: [Errno 110] Connection timed out'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "dc_scrape.py", line 7, in <module>
    resp = requests.get('https://api.datacommons.org/node/property-values?dcids=' + pair['place'] + '&property=geoJsonCoordinates')
  File "/usr/lib/python3/dist-packages/requests/api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.datacommons.org', port=443): Max retries exceeded with url: /node/property-values?dcids=geoId/06001406201&property=geoJsonCoordinates (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8107a4fd90>: Failed to establish a new connection: [Errno 110] Connection timed out'))