Pipenv and pip compatible HTML indices

[ravenscroftj]

I'm trying to install pytorch with a specific version of CUDA into my pipenv managed virtualenv.

I've found that running
pipenv run pip install torch==1.4.0+cu100 -f https://download.pytorch.org/whl/torch_stable.html
does work but obviously this is not perfect since the package goes into the virtualenv without updating the Pipfile.

I also found a stackoverflow post that seems to pose the same question - I tried reproducing their setup and ran into the same issue.

Is this functionality something that can be reproduced in pipenv or is it something that I'd need to open a feature request for?

Many thanks!

---

Pipenv version: '2018.11.26'

Pipenv location: '/usr/local/lib/python3.6/site-packages/pipenv'

Python location: '/usr/local/bin/python3.6'

Python installations found:

• 3.6.9: /home/james/.local/share/virtualenvs/james-S7OvSt0E/bin/python
• 3.6.9: /usr/local/bin/python3.6
• 3.6.9: /usr/local/bin/python3.6m
• 3.5.2: /usr/bin/python3.5m
• 3.5.2: /usr/bin/python3
• 2.7.12: /usr/bin/python2

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.6.9',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.15.0-74-generic',
 'platform_system': 'Linux',
 'platform_version': '#83~16.04.1-Ubuntu SMP Wed Dec 18 04:56:23 UTC 2019',
 'python_full_version': '3.6.9',
 'python_version': '3.6',
 'sys_platform': 'linux'}

System environment variables:

• XDG_SEAT_PATH
• XDG_CONFIG_DIRS
• LANG
• DISPLAY
• SHLVL
• LOGNAME
• XDG_VTNR
• MANDATORY_PATH
• LANGUAGE
• USER
• XAUTHORITY
• QT_LINUX_ACCESSIBILITY_ALWAYS_ON
• QT_QPA_PLATFORMTHEME
• XDG_GREETER_DATA_DIR
• XDG_SESSION_ID
• PWD
• DESKTOP_SESSION
• CINNAMON_VERSION
• XDG_SESSION_DESKTOP
• GDMSESSION
• GNOME_DESKTOP_SESSION_ID
• DEFAULTS_PATH
• DBUS_SESSION_BUS_ADDRESS
• VTE_VERSION
• GTK_OVERLAY_SCROLLING
• GJS_DEBUG_TOPICS
• XDG_DATA_DIRS
• QT_STYLE_OVERRIDE
• QT_ACCESSIBILITY
• GDM_LANG
• SHELL
• XDG_SESSION_TYPE
• GJS_DEBUG_OUTPUT
• TERM
• SESSION_MANAGER
• WINDOWID
• SSH_AUTH_SOCK
• GTK_MODULES
• XDG_CURRENT_DESKTOP
• PATH
• HOME
• XDG_SEAT
• XDG_RUNTIME_DIR
• XDG_SESSION_PATH
• GPG_AGENT_INFO
• OLDPWD
• PAGER
• LESS
• LC_CTYPE
• LSCOLORS
• LS_COLORS
• VIRTUAL_ENV_DISABLE_PROMPT
• LD_LIBRARY_PATH
• PERL5LIB
• PERL_LOCAL_LIB_ROOT
• PERL_MB_OPT
• PERL_MM_OPT
• SPARK_HOME
• NVM_DIR
• NVM_CD_FLAGS
• NVM_BIN
• _
• PIP_DISABLE_PIP_VERSION_CHECK
• PYTHONDONTWRITEBYTECODE
• PIP_PYTHON_PATH
• PIPENV_ACTIVE
• VIRTUAL_ENV
• CORENLP_HOME
• PIP_SHIMS_BASE_MODULE
• PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

• PIPENV_ACTIVE: 1

Debug–specific environment variables:

• PATH: /home/james/.local/share/virtualenvs/james-S7OvSt0E/bin:/home/james/.cargo/bin:/home/james/perl5/bin:/home/james/.nvm/versions/node/v8.0.0/bin:/home/james/.cargo/bin:/home/james/perl5/bin:/home/james/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/james/todo.txt_cli:/usr/local/go/bin:/home/james/go/bin:/home/james/todo.txt_cli:/usr/local/go/bin:/home/james/go/bin:/usr/local/cuda/bin/
• SHELL: /bin/zsh
• LANG: en_GB.UTF-8
• PWD: /home/james/workspace/banditsum-kl
• VIRTUAL_ENV: /home/james/.local/share/virtualenvs/james-S7OvSt0E

---

Contents of Pipfile ('/home/james/workspace/banditsum-kl/Pipfile'):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[[source]]
name = "pytorch"
url = "https://download.pytorch.org/whl/torch_stable.html"
verify_ssl = false

[dev-packages]

[packages]
pyrouge = "*"
tqdm = "*"
stanford-corenlp = "*"
numpy = "*"
torch = {index = "pytorch",version = "==1.4.0%2Bcu100"}

[requires]
python_version = "3.6"

[frostming]

@ravenscroftj What's the output did you see when running pipenv install?

[Mathnstein]

Repeated the above steps by running
pipenv run pip install torch==1.5.0 torchvision==0.6.0 -f https://download.pytorch.org/whl/torch)table.html
Output from install was normal (Installed future, pillow, torch and torchvision). Pipfile didn't update at all:

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
numpy = "*"
matplotlib = "*"

[requires]
python_version = "3.7"

Although entering the venv generated by pipenv and importing torch works fine.

[aehm03]

The problem seems to lie in the structure of the pytorch website. I propose a solution in the already linked stackoverflow thread: https://stackoverflow.com/questions/59752559/how-to-specify-pytorch-cuda-version-in-pipenv

[frostming]

Try this Pipfile

[[source]]
name = "pytorch"
url = "https://download.pytorch.org/whl/torch)table.html"
verify_ssl = false

[packages]
torch = {index = "pytorch",version = "==1.6.0"}

[janheinrichmerker]

@frostming With Pipenv version 2020.8.13 the above does not work.
(I suppose you meant https://download.pytorch.org/whl/torch_stable.html for the URL.)

Pipfile

[[source]]
name = "pytorch"
url = "https://download.pytorch.org/whl/torch_stable.html"
verify_ssl = false

[packages]
torch = {index = "pytorch",version = "==1.6.0"}

Also not working with torch version ==1.4.0+cu100 (like in the original question)

Output of pipenv lock --clear

Locking [dev-packages] dependencies…
Locking [packages] dependencies…
           Building requirements...
Resolving dependencies...
Locking Failed!
CRITICAL:pipenv.patched.notpip._internal.index.package_finder:Could not find a version that satisfies the requirement torch==1.6.0 (from -r C:\Users\Heinrich\AppData\Local\Temp\pipenvtf1wkpxzrequirements\pipenv-rt81g65u-constraints.txt (line 3)) (from versions: none)
[ResolutionFailure]:   File "c:/sdk/python/python38/lib/site-packages/pipenv/resolver.py", line 785, in _main
[ResolutionFailure]:       resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages)
[ResolutionFailure]:   File "c:/sdk/python/python38/lib/site-packages/pipenv/resolver.py", line 746, in resolve_packages
[ResolutionFailure]:       results, resolver = resolve(
[ResolutionFailure]:   File "c:/sdk/python/python38/lib/site-packages/pipenv/resolver.py", line 728, in resolve
[ResolutionFailure]:       return resolve_deps(
[ResolutionFailure]:   File "c:\sdk\python\python38\lib\site-packages\pipenv\utils.py", line 1378, in resolve_deps
[ResolutionFailure]:       results, hashes, markers_lookup, resolver, skipped = actually_resolve_deps(
[ResolutionFailure]:   File "c:\sdk\python\python38\lib\site-packages\pipenv\utils.py", line 1093, in actually_resolve_deps
[ResolutionFailure]:       resolver.resolve()
[ResolutionFailure]:   File "c:\sdk\python\python38\lib\site-packages\pipenv\utils.py", line 818, in resolve
[ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: No matching distribution found for torch==1.6.0 (from -r C:\Users\Heinrich\AppData\Local\Temp\pipenvtf1wkpxzrequirements\pipenv-rt81g65u-constraints.txt (line 3))

The issue doesn't seem to be fixed.