Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

python -m venv from within tox env fails with PyPy 7.3.16 on Windows #3284

Closed
jwodder opened this issue May 20, 2024 · 2 comments
Closed

python -m venv from within tox env fails with PyPy 7.3.16 on Windows #3284

jwodder opened this issue May 20, 2024 · 2 comments

Comments

@jwodder
Copy link

jwodder commented May 20, 2024 •
edited
Loading

Issue

(I don't know whether this is tox's fault or PyPy's fault, so I'm reporting it in both places. PyPy issue: pypy/pypy#4958)

Given the following tox.ini:

[testenv]
skip_install = True
commands =
    python script.py

and the following script.py:

import subprocess
import sys

subprocess.run([sys.executable, "-m", "venv", "venvdir"], check=True)

Running tox on Windows using PyPy 7.3.16 (whether as Python 3.9.19 or Python 3.10.14) fails with:

Error: problem finding exes in ['libffi-8.dll', 'libpypy3.10-c.dll', 'pypy.exe', 'pypy3.10.exe', 'pypy3.10w.exe', 'pypy3.exe', 'pypyw.exe', 'python.exe', 'python3.10.exe', 'python3.exe', 'pythonw.exe']
Traceback (most recent call last):
 File "D:\a\test\test\script.py", line 4, in <module>
   subprocess.run([sys.executable, "-m", "venv", "venvdir"], check=True)
 File "C:\hostedtoolcache\windows\PyPy\3.10.14\x86\Lib\subprocess.py", line 526, in run
   raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['D:\\a\\test\\test\\.tox\\py\\Scripts\\python.EXE', '-m', 'venv', 'venvdir']' returned non-zero exit status 1.

Running the script outside of tox works, as do running tox with PyPy 7.3.15 and running tox with PyPy 7.3.16 on Ubuntu or macOS.

Environment

Provide at least:

  • OS: Microsoft Windows Server 2022 (windows-latest in GitHub Actions)
Output of pip list of the host Python, where tox is installed 
Package       Version
------------- -----------
cachetools    5.3.3
cffi          1.17.0.dev0
chardet       5.2.0
colorama      0.4.6
distlib       0.3.8
filelock      3.14.0
greenlet      0.4.13
hpy           0.9.0
packaging     24.0
pip           24.0
platformdirs  4.2.2
pluggy        1.5.0
pyproject-api 1.6.1
readline      6.2.4.1
setuptools    65.5.0
tomli         2.0.1
tox           4.15.0
virtualenv    20.26.2

Output of running tox

Output of tox -rvv 
py: 3539 D created app data folder C:\Users\runneradmin\AppData\Local\pypa\virtualenv [virtualenv\app_data\__init__.py:40]
py: 3539 I find interpreter for spec PythonSpec(path=C:\hostedtoolcache\windows\PyPy\3.10.14\x86\python.exe) [virtualenv\discovery\builtin.py:72]
py: 3539 D filesystem is not case-sensitive [virtualenv\info.py:25]
py: 3539 I proposed PythonInfo(spec=PyPy3.10.14.final.0-64, exe=C:\hostedtoolcache\windows\PyPy\3.10.14\x86\python.exe, platform=win32, version='3.10.14 (75b3de9d9035, Apr 21 2024, 13:13:38)\n[PyPy 7.3.16 with MSC v.1929 64 bit (AMD64)]', encoding_fs_io=utf-8-utf-8) [virtualenv\discovery\builtin.py:79]
py: 3539 D accepted PythonInfo(spec=PyPy3.10.14.final.0-64, exe=C:\hostedtoolcache\windows\PyPy\3.10.14\x86\python.exe, platform=win32, version='3.10.14 (75b3de9d9035, Apr 21 2024, 13:13:38)\n[PyPy 7.3.16 with MSC v.1929 64 bit (AMD64)]', encoding_fs_io=utf-8-utf-8) [virtualenv\discovery\builtin.py:81]
py: 3555 D symlink on filesystem does work [virtualenv\info.py:44]
py: 3617 I create virtual environment via Pypy3Windows(dest=D:\a\test\test\.tox\py, clear=False, no_vcs_ignore=False, global=False) [virtualenv\run\session.py:50]
py: 3617 D create folder D:\a\test\test\.tox\py\Lib\site-packages [virtualenv\util\path\_sync.py:12]
py: 3617 D create folder D:\a\test\test\.tox\py\Scripts [virtualenv\util\path\_sync.py:12]
py: 3617 D write D:\a\test\test\.tox\py\pyvenv.cfg [virtualenv\create\pyenv_cfg.py:33]
py: 3617 D 	home = C:\hostedtoolcache\windows\PyPy\3.10.14\x86 [virtualenv\create\pyenv_cfg.py:38]
py: 3617 D 	implementation = PyPy [virtualenv\create\pyenv_cfg.py:38]
py: 3617 D 	version_info = 3.10.14.final.0 [virtualenv\create\pyenv_cfg.py:38]
py: 3617 D 	virtualenv = 20.26.2 [virtualenv\create\pyenv_cfg.py:38]
py: 3633 D 	include-system-site-packages = false [virtualenv\create\pyenv_cfg.py:38]
py: 3633 D 	base-prefix = C:\hostedtoolcache\windows\PyPy\3.10.14\x86 [virtualenv\create\pyenv_cfg.py:38]
py: 3633 D 	base-exec-prefix = C:\hostedtoolcache\windows\PyPy\3.10.14\x86 [virtualenv\create\pyenv_cfg.py:38]
py: 3633 D 	base-executable = C:\hostedtoolcache\windows\PyPy\3.10.14\x86\python.exe [virtualenv\create\pyenv_cfg.py:38]
py: 3633 D copy C:\hostedtoolcache\windows\PyPy\3.10.14\x86\python.exe to D:\a\test\test\.tox\py\Scripts\pypy.exe [virtualenv\util\path\_sync.py:40]
py: 3633 D copy C:\hostedtoolcache\windows\PyPy\3.10.14\x86\python.exe to D:\a\test\test\.tox\py\Scripts\pypy3.exe [virtualenv\util\path\_sync.py:40]
py: 3633 D copy C:\hostedtoolcache\windows\PyPy\3.10.14\x86\python.exe to D:\a\test\test\.tox\py\Scripts\python.exe [virtualenv\util\path\_sync.py:40]
py: 3633 D copy C:\hostedtoolcache\windows\PyPy\3.10.14\x86\python.exe to D:\a\test\test\.tox\py\Scripts\python3.10.exe [virtualenv\util\path\_sync.py:40]
py: 3633 D copy C:\hostedtoolcache\windows\PyPy\3.10.14\x86\python.exe to D:\a\test\test\.tox\py\Scripts\python3.exe [virtualenv\util\path\_sync.py:40]
py: 3633 D copy C:\hostedtoolcache\windows\PyPy\3.10.14\x86\libpypy3.10-c.dll to D:\a\test\test\.tox\py\Scripts\libpypy3.10-c.dll [virtualenv\util\path\_sync.py:40]
py: 4446 D copy C:\hostedtoolcache\windows\PyPy\3.10.14\x86\libffi-8.dll to D:\a\test\test\.tox\py\Scripts\libffi-8.dll [virtualenv\util\path\_sync.py:40]
py: 4446 D create virtualenv import hook file D:\a\test\test\.tox\py\Lib\site-packages\_virtualenv.pth [virtualenv\create\via_global_ref\api.py:91]
py: 4446 D create D:\a\test\test\.tox\py\Lib\site-packages\_virtualenv.py [virtualenv\create\via_global_ref\api.py:94]
py: 4446 D ============================== target debug ============================== [virtualenv\run\session.py:52]
py: 4446 D debug via 'D:\a\test\test\.tox\py\Scripts\pypy3.exe' 'C:\hostedtoolcache\windows\PyPy\3.10.14\x86\lib\site-packages\virtualenv\create\debug.py' [virtualenv\create\creator.py:200]
py: 4446 D {
  "sys": {
    "executable": "D:\\a\\test\\test\\.tox\\py\\Scripts\\pypy3.exe",
    "_base_executable": "C:\\hostedtoolcache\\windows\\PyPy\\3.10.14\\x86\\pypy3.exe",
    "prefix": "D:\\a\\test\\test\\.tox\\py",
    "base_prefix": "C:\\hostedtoolcache\\windows\\PyPy\\3.10.14\\x86",
    "real_prefix": null,
    "exec_prefix": "D:\\a\\test\\test\\.tox\\py",
    "base_exec_prefix": "C:\\hostedtoolcache\\windows\\PyPy\\3.10.14\\x86",
    "path": [
      "C:\\hostedtoolcache\\windows\\PyPy\\3.10.14\\x86\\Lib",
      "D:\\a\\test\\test\\.tox\\py",
      "D:\\a\\test\\test\\.tox\\py\\lib\\site-packages"
    ],
    "meta_path": [
      "<class '_virtualenv._Finder'>",
      "<class '_frozen_importlib.BuiltinImporter'>",
      "<class '_frozen_importlib.FrozenImporter'>",
      "<class '_frozen_importlib_external.PathFinder'>"
    ],
    "fs_encoding": "utf-8",
    "io_encoding": "utf-8"
  },
  "version": "3.10.14 (75b3de9d9035, Apr 21 2024, 13:13:38)\n[PyPy 7.3.16 with MSC v.1929 64 bit (AMD64)]",
  "makefile_filename": "C:\\hostedtoolcache\\windows\\PyPy\\3.10.14\\x86\\Lib\\config-3.10\\Makefile",
  "os": "<module 'os' from 'C:\\\\hostedtoolcache\\\\windows\\\\PyPy\\\\3.10.14\\\\x86\\\\Lib\\\\os.py'>",
  "site": "<module 'site' from 'C:\\\\hostedtoolcache\\\\windows\\\\PyPy\\\\3.10.14\\\\x86\\\\Lib\\\\site.py'>",
  "datetime": "<module 'datetime' from 'C:\\\\hostedtoolcache\\\\windows\\\\PyPy\\\\3.10.14\\\\x86\\\\Lib\\\\datetime.py'>",
  "math": "<module 'math' (built-in)>",
  "json": "<module 'json' from 'C:\\\\hostedtoolcache\\\\windows\\\\PyPy\\\\3.10.14\\\\x86\\\\Lib\\\\json\\\\__init__.py'>"
} [virtualenv\run\session.py:53]
py: 4607 I add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=C:\Users\runneradmin\AppData\Local\pypa\virtualenv) [virtualenv\run\session.py:57]
py: 4669 D install pip from wheel C:\hostedtoolcache\windows\PyPy\3.10.14\x86\lib\site-packages\virtualenv\seed\wheels\embed\pip-24.0-py3-none-any.whl via CopyPipInstall [virtualenv\seed\embed\via_app_data\via_app_data.py:49]
py: 4669 D build install image for pip-24.0-py3-none-any.whl to C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.10\image\1\CopyPipInstall\pip-24.0-py3-none-any [virtualenv\seed\embed\via_app_data\pip_install\base.py:47]
py: 4669 D install setuptools from wheel C:\hostedtoolcache\windows\PyPy\3.10.14\x86\lib\site-packages\virtualenv\seed\wheels\embed\setuptools-69.5.1-py3-none-any.whl via CopyPipInstall [virtualenv\seed\embed\via_app_data\via_app_data.py:49]
py: 4669 D build install image for setuptools-69.5.1-py3-none-any.whl to C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.10\image\1\CopyPipInstall\setuptools-69.5.1-py3-none-any [virtualenv\seed\embed\via_app_data\pip_install\base.py:47]
py: 4700 D install wheel from wheel C:\hostedtoolcache\windows\PyPy\3.10.14\x86\lib\site-packages\virtualenv\seed\wheels\embed\wheel-0.43.0-py3-none-any.whl via CopyPipInstall [virtualenv\seed\embed\via_app_data\via_app_data.py:49]
py: 4700 D build install image for wheel-0.43.0-py3-none-any.whl to C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.10\image\1\CopyPipInstall\wheel-0.43.0-py3-none-any [virtualenv\seed\embed\via_app_data\pip_install\base.py:47]
py: 4825 D copy directory C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.10\image\1\CopyPipInstall\wheel-0.43.0-py3-none-any\wheel to D:\a\test\test\.tox\py\Lib\site-packages\wheel [virtualenv\util\path\_sync.py:40]
py: 4919 D copy directory C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.10\image\1\CopyPipInstall\wheel-0.43.0-py3-none-any\wheel-0.43.0.dist-info to D:\a\test\test\.tox\py\Lib\site-packages\wheel-0.43.0.dist-info [virtualenv\util\path\_sync.py:40]
py: 4950 D copy C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.10\image\1\CopyPipInstall\wheel-0.43.0-py3-none-any\wheel-0.43.0.virtualenv to D:\a\test\test\.tox\py\Lib\site-packages\wheel-0.43.0.virtualenv [virtualenv\util\path\_sync.py:40]
py: 4966 D generated console scripts wheel.exe wheel3.exe wheel-3.10.exe wheel3.10.exe [virtualenv\seed\embed\via_app_data\pip_install\base.py:43]
py: 6362 D copy C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.10\image\1\CopyPipInstall\setuptools-69.5.1-py3-none-any\distutils-precedence.pth to D:\a\test\test\.tox\py\Lib\site-packages\distutils-precedence.pth [virtualenv\util\path\_sync.py:40]
py: 6362 D copy directory C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.10\image\1\CopyPipInstall\setuptools-69.5.1-py3-none-any\pkg_resources to D:\a\test\test\.tox\py\Lib\site-packages\pkg_resources [virtualenv\util\path\_sync.py:40]
py: 6479 D copy directory C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.10\image\1\CopyPipInstall\setuptools-69.5.1-py3-none-any\setuptools to D:\a\test\test\.tox\py\Lib\site-packages\setuptools [virtualenv\util\path\_sync.py:40]
py: 6753 D copy directory C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.10\image\1\CopyPipInstall\setuptools-69.5.1-py3-none-any\setuptools-69.5.1.dist-info to D:\a\test\test\.tox\py\Lib\site-packages\setuptools-69.5.1.dist-info [virtualenv\util\path\_sync.py:40]
py: 6769 D copy C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.10\image\1\CopyPipInstall\setuptools-69.5.1-py3-none-any\setuptools-69.5.1.virtualenv to D:\a\test\test\.tox\py\Lib\site-packages\setuptools-69.5.1.virtualenv [virtualenv\util\path\_sync.py:40]
py: 6785 D copy directory C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.10\image\1\CopyPipInstall\setuptools-69.5.1-py3-none-any\_distutils_hack to D:\a\test\test\.tox\py\Lib\site-packages\_distutils_hack [virtualenv\util\path\_sync.py:40]
py: 6785 D generated console scripts  [virtualenv\seed\embed\via_app_data\pip_install\base.py:43]
py: 8426 D copy directory C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.10\image\1\CopyPipInstall\pip-24.0-py3-none-any\pip to D:\a\test\test\.tox\py\Lib\site-packages\pip [virtualenv\util\path\_sync.py:40]
py: 8785 D copy directory C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.10\image\1\CopyPipInstall\pip-24.0-py3-none-any\pip-24.0.dist-info to D:\a\test\test\.tox\py\Lib\site-packages\pip-24.0.dist-info [virtualenv\util\path\_sync.py:40]
py: 8785 D copy C:\Users\runneradmin\AppData\Local\pypa\virtualenv\wheel\3.10\image\1\CopyPipInstall\pip-24.0-py3-none-any\pip-24.0.virtualenv to D:\a\test\test\.tox\py\Lib\site-packages\pip-24.0.virtualenv [virtualenv\util\path\_sync.py:40]
py: 8785 D generated console scripts pip.exe pip3.exe pip-3.10.exe pip3.10.exe [virtualenv\seed\embed\via_app_data\pip_install\base.py:43]
py: 8801 I add activators for Bash, Batch, Fish, Nushell, PowerShell, Python [virtualenv\run\session.py:63]
py: 8918 D write D:\a\test\test\.tox\py\pyvenv.cfg [virtualenv\create\pyenv_cfg.py:33]
py: 8919 D 	home = C:\hostedtoolcache\windows\PyPy\3.10.14\x86 [virtualenv\create\pyenv_cfg.py:38]
py: 8919 D 	implementation = PyPy [virtualenv\create\pyenv_cfg.py:38]
py: 8919 D 	version_info = 3.10.14.final.0 [virtualenv\create\pyenv_cfg.py:38]
py: 8919 D 	virtualenv = 20.26.2 [virtualenv\create\pyenv_cfg.py:38]
py: 8921 D 	include-system-site-packages = false [virtualenv\create\pyenv_cfg.py:38]
py: 8921 D 	base-prefix = C:\hostedtoolcache\windows\PyPy\3.10.14\x86 [virtualenv\create\pyenv_cfg.py:38]
py: 8921 D 	base-exec-prefix = C:\hostedtoolcache\windows\PyPy\3.10.14\x86 [virtualenv\create\pyenv_cfg.py:38]
py: 8921 D 	base-executable = C:\hostedtoolcache\windows\PyPy\3.10.14\x86\python.exe [virtualenv\create\pyenv_cfg.py:38]
py: 8924 W freeze> python -m pip freeze --all [tox\tox_env\api.py:425]
py: 10957 I exit 0 (2.03 seconds) D:\a\test\test> python -m pip freeze --all pid=3588 [tox\execute\api.py:286]
py: 10957 W cffi==1.17.0.dev0,greenlet==0.4.13,hpy==0.9.0,pip==24.0,readline==6.2.4.1,setuptools==69.5.1,wheel==0.43.0 [tox\tox_env\python\api.py:238]
py: 10957 W commands[0]> python script.py [tox\tox_env\api.py:425]
Error: problem finding exes in ['libffi-8.dll', 'libpypy3.10-c.dll', 'pypy.exe', 'pypy3.10.exe', 'pypy3.10w.exe', 'pypy3.exe', 'pypyw.exe', 'python.exe', 'python3.10.exe', 'python3.exe', 'pythonw.exe']
Traceback (most recent call last):
  File "D:\a\test\test\script.py", line 4, in <module>
    subprocess.run([sys.executable, "-m", "venv", "venvdir"], check=True)
  File "C:\hostedtoolcache\windows\PyPy\3.10.14\x86\Lib\subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['D:\\a\\test\\test\\.tox\\py\\Scripts\\python.EXE', '-m', 'venv', 'venvdir']' returned non-zero exit status 1.
py: 12974 C exit 1 (2.02 seconds) D:\a\test\test> python script.py pid=4592 [tox\execute\api.py:286]
  py: FAIL code 1 (9.47=setup[7.45]+cmd[2.02] seconds)
  evaluation failed :( (12.61 seconds)
@jwodder jwodder changed the title python -m venv from within tox env fails with PyPy 7.3.16 python -m venv from within tox env fails with PyPy 7.3.16 on Windows May 20, 2024
@jwodder jwodder mentioned this issue May 20, 2024
Closed
@mattip
Copy link

mattip commented May 23, 2024

Should be fixed in latest nightlies.

@webknjaz webknjaz mentioned this issue Dec 16, 2024
Merged
4 tasks
webknjaz added a commit to webknjaz/pip-tools that referenced this issue Dec 16, 2024
@webknjaz
🧪 Bump PyPy to v3.10
01606eb 
This is necessary because:
* `pypy-3.8` is EOL
* `pypy-3.8` has flaky SEGFAULTs on import [[1]] [[2]] [[3]] due to a
  bug in their GC that is fixed in `pypy-3.9`
* `pypy-3.9` has a bug that does not have a backport with the fix [[4]].
  Furthermore, PyPy maintainers recommend using 3.10 [[5]].

[1]: https://github.com/jazzband/pip-tools/actions/runs/12162197242/job/33918558133?pr=2106#step:8:59
[2]: pytest-dev/pytest#11771 (comment)
[3]: https://pypy.org/posts/2024/03/fixing-bug-incremental-gc.html
[4]: tox-dev/tox#3284
[5]: pypy/pypy#4958 (comment)
@webknjaz webknjaz mentioned this issue Dec 16, 2024
Merged
4 tasks
@webknjaz
Copy link
Contributor

@jwodder it seems like there's not going to be any new releases of PyPy-3.9: pypy/pypy#4958 (comment). And PyPy-3.10 already has the fix released.

cc @gaborbernat this issue is a close candidate ^

@gaborbernat gaborbernat closed this as not planned Won't fix, can't repro, duplicate, stale Dec 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@jwodder @webknjaz @gaborbernat @mattip