Timeout issues with pip or pip3: TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'

[PieterGit]

Just hit this nice traceback with oref0 while install requests package with pip3.

...
added process://ns/nightscout/ns NIGHTSCOUT_HOST API_SECRET
process://ns/nightscout/ns https://happy-bee.herokuapp.com token=subject-hash
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libpython3-dev libpython3.5-dev python3-cffi-backend python3-crypto
  python3-cryptography python3-dev python3-idna python3-keyring
  python3-keyrings.alt python3-pkg-resources python3-pyasn1
  python3-secretstorage python3-setuptools python3-six python3-wheel
  python3-xdg python3.5-dev
Suggested packages:
  python3-crypto-dbg python-crypto-doc python-cryptography-doc
  python3-cryptography-vectors gnome-keyring libkf5wallet-bin
  gir1.2-gnomekeyring-1.0 python3-pykde4 doc-base python-secretstorage-doc
  python-setuptools-doc
The following NEW packages will be installed:
  libpython3-dev libpython3.5-dev python3-cffi-backend python3-crypto
  python3-cryptography python3-dev python3-idna python3-keyring
  python3-keyrings.alt python3-pip python3-pkg-resources python3-pyasn1
  python3-secretstorage python3-setuptools python3-six python3-wheel
  python3-xdg python3.5-dev
0 upgraded, 18 newly installed, 0 to remove and 0 not upgraded.
Need to get 38.6 MB of archives.
After this operation, 54.9 MB of additional disk space will be used.
Get:1 http://archive.raspberrypi.org/debian stretch/main armhf python3-pip all 9.0.1-2+rpt2 [142 kB]
Get:2 http://mirror.nl.leaseweb.net/raspbian/raspbian stretch/main armhf libpython3.5-dev armhf 3.5.3-1 [36.9 MB]
Get:13 http://raspbian.raspberrypi.org/raspbian stretch/main armhf python3-dev armhf 3.5.3-1 [1158 B]
Get:3 http://mirror.nl.leaseweb.net/raspbian/raspbian stretch/main armhf libpython3-dev armhf 3.5.3-1 [18.7 kB]
Get:4 http://mirror.nl.leaseweb.net/raspbian/raspbian stretch/main armhf python3-cffi-backend armhf 1.9.1-2 [61.3 kB]
Get:5 http://mirror.nl.leaseweb.net/raspbian/raspbian stretch/main armhf python3-crypto armhf 2.6.1-7 [252 kB]
Get:6 http://mirror.nl.leaseweb.net/raspbian/raspbian stretch/main armhf python3-idna all 2.2-1 [32.7 kB]
Get:7 http://mirror.nl.leaseweb.net/raspbian/raspbian stretch/main armhf python3-pyasn1 all 0.1.9-2 [34.5 kB]
Get:8 http://mirror.nl.leaseweb.net/raspbian/raspbian stretch/main armhf python3-pkg-resources all 33.1.1-1 [137 kB]
Get:9 http://mirror.nl.leaseweb.net/raspbian/raspbian stretch/main armhf python3-setuptools all 33.1.1-1 [215 kB]
Get:10 http://mirror.nl.leaseweb.net/raspbian/raspbian stretch/main armhf python3-six all 1.10.0-3 [14.4 kB]
Get:11 http://mirror.nl.leaseweb.net/raspbian/raspbian stretch/main armhf python3-cryptography armhf 1.7.1-3+b2 [173 kB]
Get:12 http://mirror.nl.leaseweb.net/raspbian/raspbian stretch/main armhf python3.5-dev armhf 3.5.3-1 [413 kB]
Get:14 http://mirror.nl.leaseweb.net/raspbian/raspbian stretch/main armhf python3-secretstorage all 2.3.1-2 [14.2 kB]
Get:15 http://mirror.nl.leaseweb.net/raspbian/raspbian stretch/main armhf python3-keyring all 10.1-1 [36.8 kB]
Get:16 http://mirror.nl.leaseweb.net/raspbian/raspbian stretch/main armhf python3-keyrings.alt all 1.3-1 [16.2 kB]
Get:17 http://mirror.nl.leaseweb.net/raspbian/raspbian stretch/main armhf python3-wheel all 0.29.0-2 [51.8 kB]
Get:18 http://mirror.nl.leaseweb.net/raspbian/raspbian stretch/main armhf python3-xdg all 0.25-4 [35.7 kB]
Fetched 38.6 MB in 13s (2941 kB/s)
Selecting previously unselected package libpython3.5-dev:armhf.
(Reading database ... 44166 files and directories currently installed.)
Preparing to unpack .../00-libpython3.5-dev_3.5.3-1_armhf.deb ...
Unpacking libpython3.5-dev:armhf (3.5.3-1) ...
Selecting previously unselected package libpython3-dev:armhf.
Preparing to unpack .../01-libpython3-dev_3.5.3-1_armhf.deb ...
Unpacking libpython3-dev:armhf (3.5.3-1) ...
Selecting previously unselected package python3-cffi-backend.
Preparing to unpack .../02-python3-cffi-backend_1.9.1-2_armhf.deb ...
Unpacking python3-cffi-backend (1.9.1-2) ...
Selecting previously unselected package python3-crypto.
Preparing to unpack .../03-python3-crypto_2.6.1-7_armhf.deb ...
Unpacking python3-crypto (2.6.1-7) ...
Selecting previously unselected package python3-idna.
Preparing to unpack .../04-python3-idna_2.2-1_all.deb ...
Unpacking python3-idna (2.2-1) ...
Selecting previously unselected package python3-pyasn1.
Preparing to unpack .../05-python3-pyasn1_0.1.9-2_all.deb ...
Unpacking python3-pyasn1 (0.1.9-2) ...
Selecting previously unselected package python3-pkg-resources.
Preparing to unpack .../06-python3-pkg-resources_33.1.1-1_all.deb ...
Unpacking python3-pkg-resources (33.1.1-1) ...
Selecting previously unselected package python3-setuptools.
Preparing to unpack .../07-python3-setuptools_33.1.1-1_all.deb ...
Unpacking python3-setuptools (33.1.1-1) ...
Selecting previously unselected package python3-six.
Preparing to unpack .../08-python3-six_1.10.0-3_all.deb ...
Unpacking python3-six (1.10.0-3) ...
Selecting previously unselected package python3-cryptography.
Preparing to unpack .../09-python3-cryptography_1.7.1-3+b2_armhf.deb ...
Unpacking python3-cryptography (1.7.1-3+b2) ...
Selecting previously unselected package python3.5-dev.
Preparing to unpack .../10-python3.5-dev_3.5.3-1_armhf.deb ...
Unpacking python3.5-dev (3.5.3-1) ...
Selecting previously unselected package python3-dev.
Preparing to unpack .../11-python3-dev_3.5.3-1_armhf.deb ...
Unpacking python3-dev (3.5.3-1) ...
Selecting previously unselected package python3-secretstorage.
Preparing to unpack .../12-python3-secretstorage_2.3.1-2_all.deb ...
Unpacking python3-secretstorage (2.3.1-2) ...
Selecting previously unselected package python3-keyring.
Preparing to unpack .../13-python3-keyring_10.1-1_all.deb ...
Unpacking python3-keyring (10.1-1) ...
Selecting previously unselected package python3-keyrings.alt.
Preparing to unpack .../14-python3-keyrings.alt_1.3-1_all.deb ...
Unpacking python3-keyrings.alt (1.3-1) ...
Selecting previously unselected package python3-pip.
Preparing to unpack .../15-python3-pip_9.0.1-2+rpt2_all.deb ...
Unpacking python3-pip (9.0.1-2+rpt2) ...
Selecting previously unselected package python3-wheel.
Preparing to unpack .../16-python3-wheel_0.29.0-2_all.deb ...
Unpacking python3-wheel (0.29.0-2) ...
Selecting previously unselected package python3-xdg.
Preparing to unpack .../17-python3-xdg_0.25-4_all.deb ...
Unpacking python3-xdg (0.25-4) ...
Setting up python3-cffi-backend (1.9.1-2) ...
Setting up python3-crypto (2.6.1-7) ...
Setting up python3-idna (2.2-1) ...
Setting up python3-xdg (0.25-4) ...
Setting up python3-keyrings.alt (1.3-1) ...
Setting up python3-six (1.10.0-3) ...
Setting up python3-wheel (0.29.0-2) ...
Setting up python3-pkg-resources (33.1.1-1) ...
Setting up libpython3.5-dev:armhf (3.5.3-1) ...
Setting up python3-pyasn1 (0.1.9-2) ...
Setting up python3-pip (9.0.1-2+rpt2) ...
Processing triggers for man-db (2.7.6.1-2) ...
Setting up python3-setuptools (33.1.1-1) ...
Setting up python3.5-dev (3.5.3-1) ...
Setting up libpython3-dev:armhf (3.5.3-1) ...
Setting up python3-cryptography (1.7.1-3+b2) ...
Setting up python3-dev (3.5.3-1) ...
Setting up python3-secretstorage (2.3.1-2) ...
Setting up python3-keyring (10.1-1) ...
Collecting requests
Exception:
Traceback (most recent call last):
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 594, in urlopen
    chunked=chunked)
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 350, in _make_request
    self._validate_conn(conn)
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 837, in _validate_conn
    conn.connect()
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connection.py", line 323, in connect
    ssl_context=context)
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/util/ssl_.py", line 324, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/lib/python3.5/ssl.py", line 385, in wrap_socket
    _context=self)
  File "/usr/lib/python3.5/ssl.py", line 760, in __init__
    self.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 996, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 641, in do_handshake
    self._sslobj.do_handshake()
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 353, in run
    wb.build(autobuilding=True)
  File "/usr/lib/python3/dist-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 554, in _prepare_file
    require_hashes
  File "/usr/lib/python3/dist-packages/pip/req/req_install.py", line 278, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 465, in find_requirement
    all_candidates = self.find_all_candidates(req.name)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 423, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
  File "/usr/lib/python3/dist-packages/pip/index.py", line 568, in _get_pages
    page = self._get_page(location)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 683, in _get_page
    return HTMLPage.get_page(link, session=self.session)
  File "/usr/lib/python3/dist-packages/pip/index.py", line 792, in get_page
    "Cache-Control": "max-age=600",
  File "/usr/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 501, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python3/dist-packages/pip/download.py", line 386, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/usr/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 630, in send
    history = [resp for resp in gen] if allow_redirects else []
  File "/usr/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 630, in <listcomp>
    history = [resp for resp in gen] if allow_redirects else []
  File "/usr/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 190, in resolve_redirects
    **adapter_kwargs
  File "/usr/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/adapter.py", line 47, in send
    resp = super(CacheControlAdapter, self).send(request, **kw)
  File "/usr/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/adapters.py", line 423, in send
    timeout=timeout
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 643, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/util/retry.py", line 315, in increment
    total -= 1
TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'
Can't add pip3 requests - error installing
root@pi0rig:~/src/oref0#

I occurs with fresh latest rasbian lite image on a RPI0.

# pip --version
pip 9.0.1 from /usr/lib/python2.7/dist-packages (python 2.7)
# pip3 --version
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.5)

Several users of Debian and Ubuntu are hitting this bug, e.g.

• https://stackoverflow.com/questions/37495375/python-pip-install-throws-typeerror-unsupported-operand-types-for-retry/37531821#37531821
• https://community.home-assistant.io/t/raspian-freshh-install-connectionreseterror-errno-104-connection-reset-by-peer/46834/4
• python-pip-whl: Incompatible versions of requests and urllib3 cause TypeError when index is unavailable

Rootcause seems to be a bug in the urllib3 library which has been resolved, but is not present in current distributions, see urllib3/urllib3#567

Several pages suggest to upgrade your system pip, setuptools and wheel, but this is not a good idea as described pypa/pip#5599 . If you do that you will break your pip and pip3 commands because the upgraded version don't match the system install. I did this myself and fixed it like this: pypa/pip#5599 (comment)

The best would be that Debian and thus raspbian update their python-pip and python-pip3 packages (see e.g. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=901393 )

Fortunately it seems that adding a larger timeout also works, e.g.
pip3 --default-timeout=1000 install requests seems to fix the issue.
(credits go @zbot473 and I found it here sammchardy/python-binance#245 (comment) )

[tzachi-dar]

If you have reached here from google, and your setup is failing, then here is what you should do:

The command that fails is
curl -s https://raw.githubusercontent.com/openaps/oref0/master/bin/openaps-packages.sh | bash -

What this command does is:
sudo pip install -U openaps-contrib &&
sudo openaps-install-udev-rules &&
sudo activate-global-python-argcomplete &&
sudo npm install -g json oref0 && \

manually run:
sudo pip install -U openaps --default-timeout=1000
sudo pip install -U openaps-contrib &&
sudo openaps-install-udev-rules &&
sudo activate-global-python-argcomplete &&
sudo npm install -g json oref0 && \

Make sure you run this repeatedly until it succeeds.

Than run:
curl -s https://raw.githubusercontent.com/openaps/oref0/$BRANCH/bin/openaps-packages.sh | bash -

This time it will work (I hope :-))

[scottleibrand]

@tzachi-dar you might want to edit your comment to replace $BRANCH with master or whichever branch you're suggesting people run, as they may not have the BRANCH variable defined in their shell.

[tzachi-dar]

Changed that to master.
Maybe we should also edit the file https://raw.githubusercontent.com/openaps/oref0/master/bin/openaps-packages.sh and do the changes there (also maybe do the retries there as well).

[scottleibrand]

@PieterGit @tzachi-dar is this still an issue?

[tzachi-dar]

On my last installs, this did not happen.

[scottleibrand]

Nor on mine. Closing: please reopen (or open a new issue/PR) for any further problems.

[tousif-cheetay]

I was having the same issue which i resolve by increasing default time out
pip install --upgrade pip -r requirements.txt
replace with
pip --default-timeout=1000 install --upgrade pip -r requirements.txt