pip.install sometimes fails to install dependencies when proxy is needed

[benfiedler]

Description of Issue

pip.install state or module fails to install cffi dependency when I try to do a pip.install of python-augeas

Setup

Have a CentOS/Debian minion in a proxied environment with python[2]-pip installed but without pip packages python-augeas or cffi

Steps to Reproduce Issue

$ salt 'minion01' pip.install python-augeas proxy=my.proxy:1234 no_cache_dir=True

retcode:
1
stderr:
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-JSJQHF/python-augeas/
You are using pip version 8.1.2, however version 20.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
stdout:
Collecting python-augeas
Using cached https://files.pythonhosted.org/packages/af/cc/5064a3c25721cd863e6982b87f10fdd91d8bcc62b6f7f36f5231f20d6376/python-augeas-1.1.0.tar.gz
Complete output from command python setup.py egg_info:
Download error on https://pypi.python.org/simple/cffi/: [Errno 101] Network is unreachable -- Some packages may not be found!
Couldn't find index page for 'cffi' (maybe misspelled?)
Download error on https://pypi.python.org/simple/: [Errno 101] Network is unreachable -- Some packages may not be found!
No local packages or download links found for cffi>=1.0.0
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-build-JSJQHF/python-augeas/setup.py", line 43, in
test_suite="test.test_augeas",
File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 265, in init
self.fetch_build_eggs(attrs.pop('setup_requires'))
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 289, in fetch_build_eggs
parse_requirements(requires), installer=self.fetch_build_egg
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 618, in resolve
dist = best[req.key] = env.best_match(req, self, installer)
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 862, in best_match
return self.obtain(req, installer) # try and download/install
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 874, in obtain
return installer(requirement)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 339, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 617, in easy_install
raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('cffi>=1.0.0')

Versions Report

Minion:
(behavior first noticed on 2019.2.5 minion, upped minion to latest 3000.3 but behavior persisted)

Salt Version:
Salt: 3000.3

Dependency Versions:
cffi: 1.14.0
cherrypy: Not Installed
dateutil: 2.8.0
docker-py: 4.2.2
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 2.7.2
libgit2: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.6.2
mysql-python: Not Installed
pycparser: 2.20
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.5 (default, Aug 7 2019, 00:51:29)
python-gnupg: Not Installed
PyYAML: 5.1.2
PyZMQ: 15.3.0
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.1.4

System Versions:
dist: centos 7.7.1908 Core
locale: UTF-8
machine: x86_64
release: 3.10.0-1062.1.1.el7.x86_64
system: Linux
version: CentOS Linux 7.7.1908 Core

Master:

Salt Version:
Salt: 2019.2.5

Dependency Versions:
cffi: 0.8.6
cherrypy: 3.5.0
dateutil: 2.2
docker-py: Not Installed
gitdb: 0.5.4
gitpython: 0.3.2 RC1
ioflo: Not Installed
Jinja2: 2.9.4
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.4.2
mysql-python: Not Installed
pycparser: 2.10
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.9 (default, Sep 14 2019, 20:00:08)
python-gnupg: Not Installed
PyYAML: 3.11
PyZMQ: 14.4.0
RAET: Not Installed
smmap: 0.8.2
timelib: Not Installed
Tornado: 4.4.3
ZMQ: 4.0.5

System Versions:
dist: debian 8.11
locale: ANSI_X3.4-1968
machine: x86_64
release: 3.16.0-4-amd64
system: Linux
version: debian 8.11

Workaround

Installing cffi first and then python-augeas works just fine, but it is very strange that I should have to do so. Trying to attempt both packages simulatenously (e.g. pip.install pkgs="['cffi', 'python-augeas']" ) will fail; they have to be sequential.

$ salt 'minion01' pip.install cffi proxy=my.proxy:1234 no_cache_dir=True

retcode:
    0
stderr:
    You are using pip version 8.1.2, however version 20.1.1 is available.
    You should consider upgrading via the 'pip install --upgrade pip' command.
stdout:
    Collecting cffi
      Downloading https://files.pythonhosted.org/packages/08/29/8001b940ef40e7a25ffe8f3188bc9b118934b513d64f769dbf461e46f4ed/cffi-1.14.0-cp27-cp27mu-manylinux1_x86_64.whl (387kB)
    Requirement already satisfied (use --upgrade to upgrade): pycparser in /usr/lib/python2.7/site-packages (from cffi)
    Installing collected packages: cffi
    Successfully installed cffi-1.14.0

$ salt 'minion01' pip.install python-augeas proxy=my.proxy:1234 no_cache_dir=True

retcode:
    0
stderr:
    You are using pip version 8.1.2, however version 20.1.1 is available.
    You should consider upgrading via the 'pip install --upgrade pip' command.
stdout:
    Collecting python-augeas
      Downloading https://files.pythonhosted.org/packages/af/cc/5064a3c25721cd863e6982b87f10fdd91d8bcc62b6f7f36f5231f20d6376/python-augeas-1.1.0.tar.gz (93kB)
    Requirement already satisfied (use --upgrade to upgrade): cffi>=1.0.0 in /usr/lib64/python2.7/site-packages (from python-augeas)
    Requirement already satisfied (use --upgrade to upgrade): pycparser in /usr/lib/python2.7/site-packages (from cffi>=1.0.0->python-augeas)
    Installing collected packages: python-augeas
      Running setup.py install for python-augeas: started
        Running setup.py install for python-augeas: finished with status 'done'
    Successfully installed python-augeas-1.1.0

[benfiedler]

My mistake, this is not a saltstack problem: the problem is with pip itself and it looks like I am a duplicated by pypa/pip#7805

$ pip install python-augeas --proxy my.proxy:1234

Collecting python-augeas
Using cached https://files.pythonhosted.org/packages/af/cc/5064a3c25721cd863e6982b87f10fdd91d8bcc62b6f7f36f5231f20d6376/python-augeas-1.1.0.tar.gz
Complete output from command python setup.py egg_info:
Download error on https://pypi.python.org/simple/cffi/: [Errno 101] Network is unreachable -- Some packages may not be found!
Couldn't find index page for 'cffi' (maybe misspelled?)
Download error on https://pypi.python.org/simple/: [Errno 101] Network is unreachable -- Some packages may not be found!
No local packages or download links found for cffi>=1.0.0
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-build-51_gEl/python-augeas/setup.py", line 43, in
test_suite="test.test_augeas",
File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 265, in init
self.fetch_build_eggs(attrs.pop('setup_requires'))
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 289, in fetch_build_eggs
parse_requirements(requires), installer=self.fetch_build_egg
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 618, in resolve
dist = best[req.key] = env.best_match(req, self, installer)
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 862, in best_match
return self.obtain(req, installer) # try and download/install
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 874, in obtain
return installer(requirement)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 339, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 617, in easy_install
raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('cffi>=1.0.0')

Closing this