InstallationError: Command "python setup.py egg_info" failed with error code 1 #738

y3g0r · 2019-02-20T13:21:03Z

When dependency is specified using -e [email protected]:MyProject#egg=MyProject form in .in file, download_dir is not set to None, which makes python setup.py egg_info fail, when
setuptools_scm is used.
Basically it's the old partially fixed issue #369 with one corner case not being fixed.

Environment Versions

Ubuntu 18.10
Python version: 3.5.6
pip version: 19.0.2
pip-tools version: 3.4.0

Steps to replicate

add requirement: echo '-e [email protected]:jazzband/pip-tools#egg=pip-tools' > requirements.in
try to compile: pip-compile requirements.in

Expected result

requirements.txt file generated (like it would if we used any other editable git dependency form, i.e. -e git+http://github.com/jazzband/pip-tools#egg=pip-tools)

Actual result

$ pip-compile requirements.in                                                    
Traceback (most recent call last):
  File "/home/yegor/.pyenv/versions/lab1/bin/pip-compile", line 11, in <module>
    load_entry_point('pip-tools', 'console_scripts', 'pip-compile')()
  File "/home/yegor/.pyenv/versions/35/envs/lab1/lib/python3.5/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/yegor/.pyenv/versions/35/envs/lab1/lib/python3.5/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/yegor/.pyenv/versions/35/envs/lab1/lib/python3.5/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/yegor/.pyenv/versions/35/envs/lab1/lib/python3.5/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/yegor/.pyenv/versions/3.5.6/envs/35/envs/lab1/src/pipcompiletest/piptools/scripts/compile.py", line 197, in cli
    results = resolver.resolve(max_rounds=max_rounds)
  File "/home/yegor/.pyenv/versions/3.5.6/envs/35/envs/lab1/src/pipcompiletest/piptools/resolver.py", line 101, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/home/yegor/.pyenv/versions/3.5.6/envs/35/envs/lab1/src/pipcompiletest/piptools/resolver.py", line 198, in _resolve_one_round
    for dep in self._iter_dependencies(best_match):
  File "/home/yegor/.pyenv/versions/3.5.6/envs/35/envs/lab1/src/pipcompiletest/piptools/resolver.py", line 272, in _iter_dependencies
    for dependency in self.repository.get_dependencies(ireq):
  File "/home/yegor/.pyenv/versions/3.5.6/envs/35/envs/lab1/src/pipcompiletest/piptools/repositories/local.py", line 65, in get_dependencies
    return self.repository.get_dependencies(ireq)
  File "/home/yegor/.pyenv/versions/3.5.6/envs/35/envs/lab1/src/pipcompiletest/piptools/repositories/pypi.py", line 225, in get_dependencies
    self._dependencies_cache[ireq] = self.resolve_reqs(download_dir, ireq, wheel_cache)
  File "/home/yegor/.pyenv/versions/3.5.6/envs/35/envs/lab1/src/pipcompiletest/piptools/repositories/pypi.py", line 191, in resolve_reqs
    results = resolver._resolve_one(reqset, ireq)
  File "/home/yegor/.pyenv/versions/35/envs/lab1/lib/python3.5/site-packages/pip/_internal/resolve.py", line 294, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/home/yegor/.pyenv/versions/35/envs/lab1/lib/python3.5/site-packages/pip/_internal/resolve.py", line 226, in _get_abstract_dist_for
    req, self.require_hashes, self.use_user_site, self.finder,
  File "/home/yegor/.pyenv/versions/35/envs/lab1/lib/python3.5/site-packages/pip/_internal/operations/prepare.py", line 382, in prepare_editable_requirement
    abstract_dist.prep_for_dist(finder, self.build_isolation)
  File "/home/yegor/.pyenv/versions/35/envs/lab1/lib/python3.5/site-packages/pip/_internal/operations/prepare.py", line 158, in prep_for_dist
    self.req.prepare_metadata()
  File "/home/yegor/.pyenv/versions/35/envs/lab1/lib/python3.5/site-packages/pip/_internal/req/req_install.py", line 530, in prepare_metadata
    self.run_egg_info()
  File "/home/yegor/.pyenv/versions/35/envs/lab1/lib/python3.5/site-packages/pip/_internal/req/req_install.py", line 609, in run_egg_info
    command_desc='python setup.py egg_info')
  File "/home/yegor/.pyenv/versions/35/envs/lab1/lib/python3.5/site-packages/pip/_internal/utils/misc.py", line 761, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pip._internal.exceptions.InstallationError: Command "python setup.py egg_info" failed with error code 1 in /tmp/tmpyhkzvnvasource/pip-tools/

The text was updated successfully, but these errors were encountered:

atugushev · 2019-02-20T13:55:31Z

Hello @y3g0r,

Thank you for the report! Funny to know that pip-tools can't compile itself :)

/cc @blueyed

blueyed · 2019-02-21T03:39:01Z

Basically it's the old partially fixed issue #369 with one corner case not being fixed.

Do you have a suggested fix? (sorry, have not investigated, but it sounds like you know the corner case from debugging already)

atugushev · 2019-02-22T09:48:38Z

I've tried to investigate this issue and found that the main problem is here, see a stacktrace:

  /Users/albert/Projects/pip-tools/.venv/bin/pip-compile(11)<module>()
     10     sys.exit(
---> 11         load_entry_point('pip-tools', 'console_scripts', 'pip-compile')()
     12     )

  /Users/albert/Projects/pip-tools/.venv/lib/python3.6/site-packages/click/core.py(764)__call__()
    763         """Alias for :meth:`main`."""
--> 764         return self.main(*args, **kwargs)
    765

  /Users/albert/Projects/pip-tools/.venv/lib/python3.6/site-packages/click/core.py(717)main()
    716                 with self.make_context(prog_name, args, **extra) as ctx:
--> 717                     rv = self.invoke(ctx)
    718                     if not standalone_mode:

  /Users/albert/Projects/pip-tools/.venv/lib/python3.6/site-packages/click/core.py(956)invoke()
    955         if self.callback is not None:
--> 956             return ctx.invoke(self.callback, **ctx.params)
    957

  /Users/albert/Projects/pip-tools/.venv/lib/python3.6/site-packages/click/core.py(555)invoke()
    554             with ctx:
--> 555                 return callback(*args, **kwargs)
    556

  /Users/albert/Projects/pip-tools/piptools/scripts/compile.py(197)cli()
    196                             clear_caches=rebuild, allow_unsafe=allow_unsafe)
--> 197         results = resolver.resolve(max_rounds=max_rounds)
    198         if generate_hashes:

  /Users/albert/Projects/pip-tools/piptools/resolver.py(101)resolve()
    100             log.debug(magenta('{:^60}'.format('ROUND {}'.format(current_round))))
--> 101             has_changed, best_matches = self._resolve_one_round()
    102             log.debug('-' * 60)

  /Users/albert/Projects/pip-tools/piptools/resolver.py(198)_resolve_one_round()
    197         for best_match in best_matches:
--> 198             for dep in self._iter_dependencies(best_match):
    199                 if self.allow_unsafe or dep.name not in UNSAFE_PACKAGES:

  /Users/albert/Projects/pip-tools/piptools/resolver.py(272)_iter_dependencies()
    271         if ireq.editable:
--> 272             for dependency in self.repository.get_dependencies(ireq):
    273                 yield dependency

  /Users/albert/Projects/pip-tools/piptools/repositories/local.py(65)get_dependencies()
     64     def get_dependencies(self, ireq):
---> 65         return self.repository.get_dependencies(ireq)
     66

  /Users/albert/Projects/pip-tools/piptools/repositories/pypi.py(211)get_dependencies()
    210                 download_dir = None
--> 211             elif ireq.link and not ireq.link.is_artifact:
    212                 # No download_dir for VCS sources.  This also works around pip

  /Users/albert/Projects/pip-tools/.venv/lib/python3.6/site-packages/pip/_internal/models/link.py(160)is_artifact()
    159
--> 160         if self.scheme in vcs.all_schemes: <----- !!!!!!!! here scheme is empty
    161             return False

> /Users/albert/Projects/pip-tools/.venv/lib/python3.6/site-packages/pip/_internal/models/link.py(74)scheme()
     73         # type: () -> str
---> 74         return urllib_parse.urlsplit(self.url)[0]
     75

ipdb> urllib_parse.urlsplit(self.url)
SplitResult(scheme='', netloc='', path='[email protected]:jazzband/pip-tools', query='', fragment='egg=pip-tools')

ipdb> urllib_parse.urlsplit(self.url)[0]
''

For an ireq.link.is_artifact is True (ireq.link.scheme is empty), this block

pip-tools/piptools/repositories/pypi.py

Lines 211 to 214 in 6d871f2

    
           elif ireq.link and not ireq.link.is_artifact: 
        
               # No download_dir for VCS sources.  This also works around pip 
        
               # using git-checkout-index, which gets rid of the .git dir. 
        
               download_dir = None

is not reachable. The form git+git@... causes the bug.

y3g0r · 2019-02-22T22:32:34Z

@atugushev exactly, the issue is on that line. In my fork I have a quick fix, but it's too dirty to use it for PR.

After some investigation I realized that the issue is in fact on pip side, because Link.is_artifact is supposed to return true only if the link is something like a link to a tarbal.

So we can raise/ping ticket on pip side and create a quick fix on pip-tools side or just wait until it's fixed in pip

atugushev · 2019-02-22T22:54:42Z

@y3g0r i've already reported a bug report to the pip project. Let's see where it goes.

RXminuS · 2019-04-09T14:30:47Z

Can you reference the PIP issue?

atugushev · 2019-04-09T14:32:35Z

Hello @RXminuS,

This is the issue pypa/pip#6293.

jedie · 2019-04-11T11:06:16Z

I ran into the same Problem, isn't it?

  File ".../lib/python3.6/site-packages/piptools/repositories/pypi.py", line 208, in resolve_reqs
    results = resolver._resolve_one(reqset, ireq)
  File ".../lib/python3.6/site-packages/pip/_internal/resolve.py", line 294, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File ".../lib/python3.6/site-packages/pip/_internal/resolve.py", line 242, in _get_abstract_dist_for
    self.require_hashes
  File ".../lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 349, in prepare_linked_requirement
    abstract_dist.prep_for_dist(finder, self.build_isolation)
  File ".../lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 158, in prep_for_dist
    self.req.prepare_metadata()
  File ".../lib/python3.6/site-packages/pip/_internal/req/req_install.py", line 530, in prepare_metadata
    self.run_egg_info()
  File ".../lib/python3.6/site-packages/pip/_internal/req/req_install.py", line 609, in run_egg_info
    command_desc='python setup.py egg_info')
  File ".../lib/python3.6/site-packages/pip/_internal/utils/misc.py", line 761, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pip._internal.exceptions.InstallationError: Command "python setup.py egg_info" failed with error code 1 in /tmp/tmpay192pcubuild/psycopg2/

Is there any work-a-round?

atugushev · 2019-04-11T11:09:19Z

@jedie how do you install psycopg2?

jedie · 2019-04-11T11:17:11Z

It's while i run pip-compile --upgrade foobar.in
Last output line from pip-compile is: psycopg2==2.8.1 not in cache, need to check index

I i change psycopg2<=2.8 then it works. So i think its a problem in psycopg2 not in pip-tools.

EDIT: Seems that a check was added in v2.8:

~/$ .virtualenv/bin/pip install -U "psycopg2<2.8"
Collecting psycopg2<2.8
  Using cached https://files.pythonhosted.org/packages/37/25/53e8398975aa3323de46a5cc2745aeb4c9db11352ca905d3a15c53b6a816/psycopg2-2.7.7-cp36-cp36m-manylinux1_x86_64.whl
Installing collected packages: psycopg2
Successfully installed psycopg2-2.7.7


~/$ .virtualenv/bin/pip install -U psycopg2
Collecting psycopg2
  Using cached https://files.pythonhosted.org/packages/52/be/f898e712f5f08131d651a62754fca82a1deb42e4e9889ad01932f770a2be/psycopg2-2.8.1.tar.gz
    Complete output from command python setup.py egg_info:
    running egg_info
    creating pip-egg-info/psycopg2.egg-info
    writing pip-egg-info/psycopg2.egg-info/PKG-INFO
    writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
    writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
    writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'
    
    Error: pg_config executable not found.
    
    pg_config is required to build psycopg2 from source.  Please add the directory
    containing pg_config to the $PATH or specify the full executable path with the
    option:
    
        python setup.py build_ext --pg-config /path/to/pg_config build ...
    
    or with the pg_config option in 'setup.cfg'.
    
    If you prefer to avoid building psycopg2 from source, please install the PyPI
    'psycopg2-binary' package instead.
    
    For further information please check the 'doc/src/install.rst' file (also at
    <http://initd.org/psycopg/docs/install.html>).
    
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-wko167ql/psycopg2/

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

InstallationError: Command "python setup.py egg_info" failed with error code 1 #738

InstallationError: Command "python setup.py egg_info" failed with error code 1 #738

y3g0r commented Feb 20, 2019

atugushev commented Feb 20, 2019

blueyed commented Feb 21, 2019

atugushev commented Feb 22, 2019 •

edited

Loading

y3g0r commented Feb 22, 2019

atugushev commented Feb 22, 2019

RXminuS commented Apr 9, 2019

atugushev commented Apr 9, 2019

jedie commented Apr 11, 2019

atugushev commented Apr 11, 2019

jedie commented Apr 11, 2019 •

edited

Loading

atugushev commented Apr 11, 2019

atugushev commented Jan 2, 2020

InstallationError: Command "python setup.py egg_info" failed with error code 1 #738

InstallationError: Command "python setup.py egg_info" failed with error code 1 #738

Comments

y3g0r commented Feb 20, 2019

Environment Versions

Steps to replicate

Expected result

Actual result

atugushev commented Feb 20, 2019

blueyed commented Feb 21, 2019

atugushev commented Feb 22, 2019 • edited Loading

y3g0r commented Feb 22, 2019

atugushev commented Feb 22, 2019

RXminuS commented Apr 9, 2019

atugushev commented Apr 9, 2019

jedie commented Apr 11, 2019

atugushev commented Apr 11, 2019

jedie commented Apr 11, 2019 • edited Loading

atugushev commented Apr 11, 2019

atugushev commented Jan 2, 2020

atugushev commented Feb 22, 2019 •

edited

Loading

jedie commented Apr 11, 2019 •

edited

Loading