Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installing a package from VCS fails with Invalid specifier: '==attr: ... if the package is already installed #5360

Closed
Jmennius opened this issue Sep 16, 2022 · 7 comments
Labels
Type: Possible Bug This issue describes a possible bug in pipenv.

Comments

@Jmennius
Copy link

Issue description

When trying install/lock a project from VCS that uses version = attr: <package.field> specifier in setup.cfg, if the package was (is) previously installed from a wheel, I get an error:
pipenv.patched.pip._vendor.packaging.specifiers.InvalidSpecifier: Invalid specifier: '==attr: PyInstaller.__version__'

if installing from VCS when the package is not installed from the wheel yet - the issue does not happen.

Expected result

No error occurs, package is installed and locked.

Actual result

$ pipenv --python ~/.pyenv/versions/3.8.13/bin/python3 --verbose install git+https://github.com/pyinstaller/pyinstaller.git@develop#egg=pyinstaller
Installing git+https://github.com/pyinstaller/pyinstaller.git@develop#egg=pyinstaller...
Installing package: git+https://github.com/pyinstaller/pyinstaller.git@develop#egg=pyinstaller
Writing supplied requirement line to temporary file: 'git+https://github.com/pyinstaller/pyinstaller.git@develop#egg=pyinstaller'
Installing 'pyinstaller'
$ /var/home/ievgenp/src/pipenv-test/.venv/bin/python /var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv/patched/pip/__pip-runner__.py install --no-input --verbose --upgrade --exists-action=i -r /tmp/pipenv-80p7itc2-requirements/pipenv-wvj33x41-requirement.txt -i https://pypi.org/simple
Using source directory: '/var/home/ievgenp/src/pipenv-test/.venv/src'
Adding pyinstaller to Pipfile's [packages]...
✔ Installation Succeeded 
Pipfile.lock (6f7040) out of date, updating to (2c5d04)...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
Reporter.starting()
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.starting()
Reporter.adding_requirement(SpecifierRequirement('altgraph'), None)
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.adding_requirement(SpecifierRequirement('altgraph'), None)
Reporter.adding_requirement(SpecifierRequirement('pyinstaller-hooks-contrib>=2021.4'), None)
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.adding_requirement(SpecifierRequirement('pyinstaller-hooks-contrib>=2021.4'), None)
Reporter.adding_requirement(SpecifierRequirement('setuptools'), None)
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.adding_requirement(SpecifierRequirement('setuptools'), None)
Reporter.starting_round(0)
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.starting_round(0)
Reporter.pinning(LinkCandidate('https://files.pythonhosted.org/packages/84/3f/1a5c9bef54cac9bf41edd6f4aaf61cd52ed578e10ccc607e0278012cb4a5/altgraph-0.17.2-py2.py3-none-any.whl (from https://pypi.org/simple/altgraph/)'))
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.pinning(LinkCandidate('https://files.pythonhosted.org/packages/84/3f/1a5c9bef54cac9bf41edd6f4aaf61cd52ed578e10ccc607e0278012cb4a5/altgraph-0.17.2-py2.py3-none-any.whl (from https://pypi.org/simple/altgraph/)'))
Reporter.ending_round(0, state)
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.ending_round(0, state)
Reporter.starting_round(1)
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.starting_round(1)
Reporter.pinning(LinkCandidate('https://files.pythonhosted.org/packages/33/39/803aed3f31826e308cd6391902b78d1f3375a4191d1a5d9c84a315e5f55e/pyinstaller_hooks_contrib-2022.10-py2.py3-none-any.whl (from https://pypi.org/simple/pyinstaller-hooks-contrib/) (requires-python:>=3.7)'))
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.pinning(LinkCandidate('https://files.pythonhosted.org/packages/33/39/803aed3f31826e308cd6391902b78d1f3375a4191d1a5d9c84a315e5f55e/pyinstaller_hooks_contrib-2022.10-py2.py3-none-any.whl (from https://pypi.org/simple/pyinstaller-hooks-contrib/) (requires-python:>=3.7)'))
Reporter.ending_round(1, state)
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.ending_round(1, state)
Reporter.starting_round(2)
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.starting_round(2)
Reporter.pinning(LinkCandidate('https://files.pythonhosted.org/packages/d9/5f/2daccd14278b6b780ae6799f85998377c06019354982391245f4b58a927d/setuptools-65.3.0-py3-none-any.whl (from https://pypi.org/simple/setuptools/) (requires-python:>=3.7)'))
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.pinning(LinkCandidate('https://files.pythonhosted.org/packages/d9/5f/2daccd14278b6b780ae6799f85998377c06019354982391245f4b58a927d/setuptools-65.3.0-py3-none-any.whl (from https://pypi.org/simple/setuptools/) (requires-python:>=3.7)'))
Reporter.ending_round(2, state)
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.ending_round(2, state)
Reporter.starting_round(3)
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.starting_round(3)
Reporter.ending(State(mapping=OrderedDict([('altgraph', LinkCandidate('https://files.pythonhosted.org/packages/84/3f/1a5c9bef54cac9bf41edd6f4aaf61cd52ed578e10ccc607e0278012cb4a5/altgraph-0.17.2-py2.py3-none-any.whl (from https://pypi.org/simple/altgraph/)')), ('pyinstaller-hooks-contrib', LinkCandidate('https://files.pythonhosted.org/packages/33/39/803aed3f31826e308cd6391902b78d1f3375a4191d1a5d9c84a315e5f55e/pyinstaller_hooks_contrib-2022.10-py2.py3-none-any.whl (from https://pypi.org/simple/pyinstaller-hooks-contrib/) (requires-python:>=3.7)')), ('setuptools', LinkCandidate('https://files.pythonhosted.org/packages/d9/5f/2daccd14278b6b780ae6799f85998377c06019354982391245f4b58a927d/setuptools-65.3.0-py3-none-any.whl (from https://pypi.org/simple/setuptools/) (requires-python:>=3.7)'))]), criteria={'altgraph': Criterion((SpecifierRequirement('altgraph'), via=None)), 'pyinstaller-hooks-contrib': Criterion((SpecifierRequirement('pyinstaller-hooks-contrib>=2021.4'), via=None)), 'setuptools': Criterion((SpecifierRequirement('setuptools'), via=None))}, backtrack_causes=[]))
INFO:pipenv.patched.pip._internal.resolution.resolvelib.reporter:Reporter.ending(State(mapping=OrderedDict([('altgraph', LinkCandidate('https://files.pythonhosted.org/packages/84/3f/1a5c9bef54cac9bf41edd6f4aaf61cd52ed578e10ccc607e0278012cb4a5/altgraph-0.17.2-py2.py3-none-any.whl (from https://pypi.org/simple/altgraph/)')), ('pyinstaller-hooks-contrib', LinkCandidate('https://files.pythonhosted.org/packages/33/39/803aed3f31826e308cd6391902b78d1f3375a4191d1a5d9c84a315e5f55e/pyinstaller_hooks_contrib-2022.10-py2.py3-none-any.whl (from https://pypi.org/simple/pyinstaller-hooks-contrib/) (requires-python:>=3.7)')), ('setuptools', LinkCandidate('https://files.pythonhosted.org/packages/d9/5f/2daccd14278b6b780ae6799f85998377c06019354982391245f4b58a927d/setuptools-65.3.0-py3-none-any.whl (from https://pypi.org/simple/setuptools/) (requires-python:>=3.7)'))]), criteria={'altgraph': Criterion((SpecifierRequirement('altgraph'), via=None)), 'pyinstaller-hooks-contrib': Criterion((SpecifierRequirement('pyinstaller-hooks-contrib>=2021.4'), via=None)), 'setuptools': Criterion((SpecifierRequirement('setuptools'), via=None))}, backtrack_causes=[]))
Traceback (most recent call last):
  File "/var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv/patched/pip/_vendor/packaging/specifiers.py", line 634, in __init__
    parsed.add(Specifier(specifier))
  File "/var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv/patched/pip/_vendor/packaging/specifiers.py", line 98, in __init__
    raise InvalidSpecifier(f"Invalid specifier: '{spec}'")
pipenv.patched.pip._vendor.packaging.specifiers.InvalidSpecifier: Invalid specifier: '==attr: PyInstaller.__version__'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv/resolver.py", line 867, in <module>
    main()
  File "/var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv/resolver.py", line 853, in main
    _main(
  File "/var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv/resolver.py", line 833, in _main
    resolve_packages(
  File "/var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv/resolver.py", line 794, in resolve_packages
    results = clean_results(results, resolver, project, dev)
  File "/var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv/resolver.py", line 659, in clean_results
    entry_dict = translate_markers(entry.get_cleaned_dict(keep_outdated=False))
  File "/var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv/resolver.py", line 279, in get_cleaned_dict
    self.validate_constraints()
  File "/var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv/resolver.py", line 575, in validate_constraints
    pinned_version = self.updated_version
  File "/var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv/resolver.py", line 641, in __getattribute__
    return super().__getattribute__(key)
  File "/var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv/resolver.py", line 423, in updated_version
    version = self.entry.specifiers
  File "/var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2591, in specifiers
    and self.line_instance.setup_info
  File "/var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 808, in setup_info
    self.setup_info = self.get_setup_info()
  File "/var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 821, in setup_info
    self.specifier = setup_info.version
  File "/var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 414, in specifier
    self.specifiers = SpecifierSet(spec)
  File "/var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv/patched/pip/_vendor/packaging/specifiers.py", line 636, in __init__
    parsed.add(LegacySpecifier(specifier))
  File "/var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv/patched/pip/_vendor/packaging/specifiers.py", line 253, in __init__
    super().__init__(spec, prereleases)
  File "/var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv/patched/pip/_vendor/packaging/specifiers.py", line 98, in __init__
    raise InvalidSpecifier(f"Invalid specifier: '{spec}'")
pipenv.patched.pip._vendor.packaging.specifiers.InvalidSpecifier: Invalid specifier: '==attr: PyInstaller.__version__'
✘ Locking Failed! 

Steps to replicate

pipenv install pyinstaller
pipenv --verbose install git+https://github.com/pyinstaller/pyinstaller.git@develop#egg=pyinstaller

or

pipenv lock  # create an empty venv with pipenv
pipenv run pip install pyinstaller  # install pyinstaller from a wheel into a venv
pipenv install git+https://github.com/pyinstaller/pyinstaller.git@develop#egg=pyinstaller  # try installing from a VCS

$ pipenv --support

Pipenv version: '2022.9.8'

Pipenv location: '/var/home/ievgenp/.pyenv/versions/3.8.13/lib/python3.8/site-packages/pipenv'

Python location: '/var/home/ievgenp/.pyenv/versions/3.8.13/bin/python3.8'

OS Name: 'posix'

User pip version: '22.2.2'

user Python installations found:

  • 3.10.4: /var/home/ievgenp/.pyenv/versions/3.10.4/bin/python3
  • 3.8.13: /var/home/ievgenp/.pyenv/versions/3.8.13/bin/python3.8
  • 3.8.13: /var/home/ievgenp/.pyenv/versions/3.8.13/bin/python3
  • 3.8.13: /var/home/ievgenp/.pyenv/versions/3.8.13/bin/python
  • 3.8.13: /var/home/ievgenp/.pyenv/versions/3.8.13/bin/python3.8
  • 3.8.6: /var/home/ievgenp/.pyenv/versions/3.8.6/bin/python3.8
  • 3.8.5: /var/home/ievgenp/.pyenv/versions/3.8.5/bin/python3.8
  • 3.7.13: /var/home/ievgenp/.pyenv/versions/3.7.13/bin/python3.7m
  • 3.7.9: /var/home/ievgenp/.pyenv/versions/3.7.9/bin/python3.7m
  • 3.7.1: /var/home/ievgenp/.pyenv/versions/3.7.1/bin/python
  • 3.7.1: /var/home/ievgenp/.pyenv/versions/test1/bin/python
  • 3.6.9: /usr/bin/python3.6
  • 3.6.9: /usr/bin/python3.6m
  • 3.6.9: /usr/bin/python3
  • 2.7.17: /usr/bin/python2
  • 2.7.17: /usr/bin/python2.7
  • 2.7.17: /usr/bin/python

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.8.13',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.19.8-200.fc36.x86_64',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP PREEMPT_DYNAMIC Thu Sep 8 19:02:21 UTC 2022',
 'python_full_version': '3.8.13',
 'python_version': '3.8',
 'sys_platform': 'linux'}

System environment variables:

  • PIPENV_VENV_IN_PROJECT
  • LD_LIBRARY_PATH
  • XDG_MENU_PREFIX
  • P9K_SSH
  • LANG
  • DISPLAY
  • HOSTNAME
  • PYENV_ROOT
  • OLDPWD
  • KEYTIMEOUT
  • PIPENV_SHELL_FANCY
  • DEBFULLNAME
  • COLORTERM
  • _P9K_TTY
  • PYTHON_CONFIGURE_OPTS
  • PYENV_HOOK_PATH
  • SSH_AUTH_SOCK
  • MINICOM
  • container
  • ZDOTDIR
  • USER
  • PYENV_DIR
  • DESKTOP_SESSION
  • VERSION
  • WAYLAND_DISPLAY
  • PWD
  • HOME
  • PYENV_VERSION
  • NAME
  • XDG_SESSION_TYPE
  • BUILDAH_LAYERS
  • XDG_DATA_DIRS
  • XDG_SESSION_DESKTOP
  • CMAKE_BUILD_PARALLEL_LEVEL
  • TERM
  • SHELL
  • VTE_VERSION
  • DOCKER_HOST
  • USER_CERTIFICATE
  • USER_PRIVATE_KEY
  • XDG_CURRENT_DESKTOP
  • PYENV_SHELL
  • SHLVL
  • TOOLBOX_PATH
  • DEBEMAIL
  • LOGNAME
  • DBUS_SESSION_BUS_ADDRESS
  • XDG_RUNTIME_DIR
  • XAUTHORITY
  • PATH
  • P9K_TTY
  • PIP_DISABLE_PIP_VERSION_CHECK
  • PIP_PYTHON_PATH
  • PYTHONDONTWRITEBYTECODE
  • PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

  • PIPENV_VENV_IN_PROJECT: 1
  • PIPENV_SHELL_FANCY: 1

Debug–specific environment variables:

  • PATH: /var/home/ievgenp/.pyenv/versions/3.8.13/bin:/var/home/ievgenp/.pyenv/libexec:/var/home/ievgenp/.pyenv/plugins/python-build/bin:/var/home/ievgenp/go/bin:/var/home/ievgenp/.pyenv/shims:/var/home/ievgenp/.pyenv/bin:/var/home/ievgenp/.local/bin:/var/home/ievgenp/.cargo/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/var/home/ievgenp/.local/share/flatpak/exports/bin:/var/lib/flatpak/exports/bin
  • SHELL: /usr/bin/zsh
  • LANG: en_US.UTF-8
  • PWD: /var/home/ievgenp/src/pipenv-test

Contents of Pipfile ('/var/home/ievgenp/src/pipenv-test/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
pyinstaller = {ref = "develop", git = "https://github.com/pyinstaller/pyinstaller.git"}

[dev-packages]

[requires]
python_version = "3.8"

Contents of Pipfile.lock ('/var/home/ievgenp/src/pipenv-test/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "7f7606f08e0544d8d012ef4d097dabdd6df6843a28793eb6551245d4b2db4242"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.8"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {},
    "develop": {}
}
@matteius
Copy link
Member

@Jmennius I was able to reproduce your issue -- I think it has to do with I don't see a version specifier in the setup.py for pyinstaller -- https://github.com/pyinstaller/pyinstaller/blob/develop/setup.py#L253-L272

This confuses the resolver because it needs to pin to a specific version I believe.

@matteius matteius added Type: Possible Bug This issue describes a possible bug in pipenv. Status: Awaiting Update ⏳ This issue requires more information before assistance can be provided. labels Sep 20, 2022
@Jmennius
Copy link
Author

@Jmennius I was able to reproduce your issue -- I think it has to do with I don't see a version specifier in the setup.py for pyinstaller -- https://github.com/pyinstaller/pyinstaller/blob/develop/setup.py#L253-L272

This confuses the resolver because it needs to pin to a specific version I believe.

But the version specifier is in setup.cfg - shouldn't it be enough?
Plus, it works in some cases - like if you install the package in a clean environment.

@matteius matteius added hacktoberfest Contributor Candidate The issue has been identified/triaged and contributions are welcomed/encouraged. Type: Possible Bug This issue describes a possible bug in pipenv. and removed Type: Possible Bug This issue describes a possible bug in pipenv. Status: Awaiting Update ⏳ This issue requires more information before assistance can be provided. labels Sep 20, 2022
@iuriguilherme
Copy link

iuriguilherme commented Oct 12, 2022

It looks like behaviour changed in setuptools: pypa/setuptools#1724

And per the documentation this should be working now: https://packaging.python.org/en/latest/guides/single-sourcing-package-version/

So this means pipenv is using an old setuptools version?

Anyway, I had to hardcode versions in every setup.cfg of my packages to prevent this:

[metadata]
name = package
version = v0.0.1

This is undesirable package mantaining workflow but it's what's allowing me to use pipenv install -e . without failing

@matteius
Copy link
Member

matteius commented Dec 9, 2022

@iuriguilherme Pipenv only calls out "setuptools>=36.2.1" in order to support python 3.7 I believe, or maybe this was for 3.6 and the pin could be increased. Either way, the end user can install whichever version of setuptools they want that is at least that new.

@matteius matteius added Status: Awaiting Update ⏳ This issue requires more information before assistance can be provided. and removed hacktoberfest Contributor Candidate The issue has been identified/triaged and contributions are welcomed/encouraged. labels Dec 9, 2022
@matteius matteius removed the Status: Awaiting Update ⏳ This issue requires more information before assistance can be provided. label Dec 19, 2022
@Jmennius
Copy link
Author

The issue I was having with PyInstaller was fixed via pyinstaller/pyinstaller#7444 for me.
I am not really sure if it's a bug in setuptools or if it's just improper usage of declarative/imperative setup...
Unless anybody comments in a few weeks I will close this from my side as fixed.

@matteius
Copy link
Member

@Jmennius We bumped the setuptools version requirement with the new pipenv release, so unless a new issue crops up, I think we can close this for now. Can defer to you for the next week or two.

@iuriguilherme
Copy link

src/
  package/
      __init__.py
      __main__.py
      _version.py
setup.cfg

__init__.py:

from ._version import __version__
__name__: str = "package"
__description__: str = "No one reads this"

_version.py:

__version__: str = 0.0.0.1

setup.cfg:

[metadata]
name = attr: package.__name__
version = attr: package._version.__version__
description = attr: package.__description__

Working for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Possible Bug This issue describes a possible bug in pipenv.
Projects
None yet
Development

No branches or pull requests

3 participants