Pytorch and torchvision CPU version install fails

[jekel]

Hello,
I wanted to install CPU version of torch 1.6.0 and torchvision 0.7.0
it's similar issue like #4422 and #4171
but in my case, pipenv unable to solve requirements even they are written in pipenv file

Issue description

Requirements check fails with expectation of pinned package version

Expected result

Torch and torchvision are installed

Actual result

Current constraints:
  torch==1.6.0+cpu from https://download.pytorch.org/whl/cpu/torch-1.6.0%2Bcpu-cp36-cp36m-linux_x86_64.whl#egg=torch (from -r /tmp/pipenv9nl1jcq7requirements/pipenv-cwkg_khz-constraints.txt (line 3))
  torchvision==0.7.0+cpu from https://download.pytorch.org/whl/cpu/torchvision-0.7.0%2Bcpu-cp36-cp36m-linux_x86_64.whl#egg=torchvision (from -r /tmp/pipenv9nl1jcq7requirements/pipenv-cwkg_khz-constraints.txt (line 2))

Finding the best candidates:
  found candidate https://download.pytorch.org/whl/cpu/torch-1.6.0%2Bcpu-cp36-cp36m-linux_x86_64.whl#egg=torch (constraint was ==1.6.0+cpu)
  found candidate https://download.pytorch.org/whl/cpu/torchvision-0.7.0%2Bcpu-cp36-cp36m-linux_x86_64.whl#egg=torchvision (constraint was ==0.7.0+cpu)

Finding secondary dependencies:
  https://download.pytorch.org/whl/cpu/torchvision-0.7.0%2Bcpu-cp36-cp36m-linux_x86_64.whl#egg=torchvision requires numpy, pillow>=4.1.1, torch==1.6.0
  https://download.pytorch.org/whl/cpu/torch-1.6.0%2Bcpu-cp36-cp36m-linux_x86_64.whl#egg=torch requires future, numpy

New dependencies found in this round:
  adding ['future', '', '[]']
  adding ['numpy', '', '[]']
  adding ['pillow', '>=4.1.1', '[]']
  adding ['torch', '==1.6.0', '[]']
Removed dependencies in this round:
------------------------------------------------------------
Result of round 1: not stable

                          ROUND 2                           
Current constraints:
  future (from torch==1.6.0+cpu->-r /tmp/pipenv9nl1jcq7requirements/pipenv-cwkg_khz-constraints.txt (line 3))
  numpy (from torch==1.6.0+cpu->-r /tmp/pipenv9nl1jcq7requirements/pipenv-cwkg_khz-constraints.txt (line 3))
  pillow>=4.1.1 (from torchvision==0.7.0+cpu->-r /tmp/pipenv9nl1jcq7requirements/pipenv-cwkg_khz-constraints.txt (line 2))
  torch==1.6.0,==1.6.0+cpu from https://download.pytorch.org/whl/cpu/torch-1.6.0%2Bcpu-cp36-cp36m-linux_x86_64.whl#egg=torch (from -r /tmp/pipenv9nl1jcq7requirements/pipenv-cwkg_khz-constraints.txt (line 3))
  torchvision==0.7.0+cpu from https://download.pytorch.org/whl/cpu/torchvision-0.7.0%2Bcpu-cp36-cp36m-linux_x86_64.whl#egg=torchvision (from -r /tmp/pipenv9nl1jcq7requirements/pipenv-cwkg_khz-constraints.txt (line 2))

Finding the best candidates:
  found candidate future==0.18.2 (constraint was <any>)
  found candidate numpy==1.19.2 (constraint was <any>)
  found candidate pillow==8.0.1 (constraint was >=4.1.1)
  found candidate https://download.pytorch.org/whl/cpu/torch-1.6.0%2Bcpu-cp36-cp36m-linux_x86_64.whl#egg=torch (constraint was ==1.6.0,==1.6.0+cpu)
  found candidate https://download.pytorch.org/whl/cpu/torchvision-0.7.0%2Bcpu-cp36-cp36m-linux_x86_64.whl#egg=torchvision (constraint was ==0.7.0+cpu)

Finding secondary dependencies:
  numpy==1.19.2             requires -
  pillow==8.0.1             requires -
  https://download.pytorch.org/whl/cpu/torchvision-0.7.0%2Bcpu-cp36-cp36m-linux_x86_64.whl#egg=torchvision requires numpy, pillow>=4.1.1, torch==1.6.0
  future==0.18.2            requires -
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/pipenv/resolver.py", line 807, in <module>
    main()
  File "/usr/lib/python3.6/site-packages/pipenv/resolver.py", line 803, in main
    parsed.requirements_dir, parsed.packages, parse_only=parsed.parse_only)
  File "/usr/lib/python3.6/site-packages/pipenv/resolver.py", line 785, in _main
    resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages)
  File "/usr/lib/python3.6/site-packages/pipenv/resolver.py", line 753, in resolve_packages
    requirements_dir=requirements_dir,
  File "/usr/lib/python3.6/site-packages/pipenv/resolver.py", line 736, in resolve
    req_dir=requirements_dir
  File "/usr/lib/python3.6/site-packages/pipenv/utils.py", line 1386, in resolve_deps
    req_dir=req_dir,
  File "/usr/lib/python3.6/site-packages/pipenv/utils.py", line 1093, in actually_resolve_deps
    resolver.resolve()
  File "/usr/lib/python3.6/site-packages/pipenv/utils.py", line 808, in resolve
    results = self.resolver.resolve(max_rounds=environments.PIPENV_MAX_ROUNDS)
  File "/usr/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 180, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/usr/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 268, in _resolve_one_round
    their_constraints.extend(self._iter_dependencies(best_match))
  File "/usr/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 369, in _iter_dependencies
    "Expected pinned or editable requirement, got {}".format(ireq)
TypeError: Expected pinned or editable requirement, got torch==1.6.0,==1.6.0+cpu from https://download.pytorch.org/whl/cpu/torch-1.6.0%2Bcpu-cp36-cp36m-linux_x86_64.whl#egg=torch (from -r /tmp/pipenv9nl1jcq7requirements/pipenv-cwkg_khz-constraints.txt (line 3))

Steps to replicate

command pipenv lock -v

Pipfile contents

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
torch = {file = "https://download.pytorch.org/whl/cpu/torch-1.6.0%252Bcpu-cp37-cp37m-linux_x86_64.whl"}
torchvision = {file = "https://download.pytorch.org/whl/cpu/torchvision-0.7.0%252Bcpu-cp37-cp37m-linux_x86_64.whl"}

Pipenv version: '2020.8.13'
Python version 3.6.10

I have tried also with latest python 3.8 version - but it does not change anything.
Specifying version field in package settings also does not help

[frostming]

torchvision requires torch==1.6.0 which is not considered a compatible version with torch==1.6.0+cpu. The issue lies in the upstream library piptools

[jekel]

So, i think that problem is near

pipenv/pipenv/patched/piptools/utils.py

Line 232 in 912164d

if ireq.req is None or len(ireq.specifier._specs) != 1:

when it checks package requirements, and there are specified both versions 1.6.0 and 1.6.0+cpu - it fails, bacause accepts only one.

btw, PIP correctly installs both packages

pip install torch==1.6.0+cpu torchvision==0.7.0+cpu -f https://download.pytorch.org/whl/torch_stable.html
Defaulting to user installation because normal site-packages is not writeable
Looking in links: https://download.pytorch.org/whl/torch_stable.html
Collecting torch==1.6.0+cpu
  Downloading https://download.pytorch.org/whl/cpu/torch-1.6.0%2Bcpu-cp36-cp36m-linux_x86_64.whl (154.6 MB)
     |████████████████████████████████| 154.6 MB 108 kB/s 
Collecting torchvision==0.7.0+cpu
  Downloading https://download.pytorch.org/whl/cpu/torchvision-0.7.0%2Bcpu-cp36-cp36m-linux_x86_64.whl (5.1 MB)
     |████████████████████████████████| 5.1 MB 3.4 MB/s 
Collecting numpy
  Downloading numpy-1.19.3-cp36-cp36m-manylinux2010_x86_64.whl (14.9 MB)
     |████████████████████████████████| 14.9 MB 13.6 MB/s 
Requirement already satisfied: future in /usr/lib/python3.6/site-packages (from torch==1.6.0+cpu) (0.16.0)
Requirement already satisfied: pillow>=4.1.1 in /usr/lib64/python3.6/site-packages (from torchvision==0.7.0+cpu) (5.0.0)
Installing collected packages: numpy, torch, torchvision
Successfully installed numpy-1.19.3 torch-1.6.0+cpu torchvision-0.7.0+cpu

[uranusjr]

The +cpu part is a local version identifier, and 1.6.0+cpu should be considered compatible with ==1.6.0. This is implemented in packaging:

>>> from packaging import version, specifiers
>>> version.parse('1.6.0+cpu') in specifiers.SpecifierSet('==1.6.0')
True

But indeed this is a pip-tools issue and should be reported upstream.

[uranusjr]

It seems like they are already aware of the issue, but do not have a fix yet. jazzband/pip-tools#1114

[fgriberi]

Does anyone have a workaround for this?

[[source]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cpu"
verify_ssl = false

[packages]
torchvision = {index = "pytorch", version = "==0.9.0"}
torch = {index = "pytorch", version = "==1.8.0"}

Using this works for torchvision, but torch still failling.

[MoRoBe-Work]

If you stumble upon this thread by googling "TypeError: Expected pinned or editable requirement, got torch" do try @fgriberi's solution, it worked fine for me although setting me back a pytorch version.

[matsutakk]

As @MoRoBe-Work pointed out, if I "pipenv install" by Pipfile as below,

[[source]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu111"
verify_ssl = false

[packages]
torchvision = {index = "pytorch", version = "==0.10.0"}
torch = {index = "pytorch", version = "==1.9.0"}

no error happens but print(torch.__version__) shows torch+cu102.

Would anyone know how to install torch+cu111 by pipenv ?
I need cu111 because NVIDIA A100 requires it.

Whole Pipfile

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[[source]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cu111"
verify_ssl = false

[packages]
mlflow = "==1.20.2"
hydra-core = "==1.1.1"
torchvision = {index = "pytorch", version = "==0.10.0"}
torch = {index = "pytorch", version = "==1.9.0"}

[dev-packages]

[requires]
python_version = "3.9"

[MoRoBe-Work]

I use the line
torch = {index = "pytorch", version = "==1.9.0+cu111"}
under packages and torch.__version__ shows 1.9.0+cu111

Does this work for you as well?

[matsutakk]

It works😀
I got a step closer to graduation thanks to you.

Thank you so much!