tests/unit/discovery/py_info/test_py_info.py::test_custom_venv_install_scheme_is_prefered failure with Python 3.11.0b1

[mgorny]

Issue

The following test is failing when running the test suite with Python 3.11.0b1:

FAILED tests/unit/discovery/py_info/test_py_info.py::test_custom_venv_install_scheme_is_prefered - AssertionError: assert 'local/bin...

Environment

Provide at least:

• OS: Gentoo Linux
• pip list of the host python where virtualenv is installed: pip-list.txt

Output of tox

$ tox -e py311
py311 create: /tmp/virtualenv-20.14.1/.tox/py311
py311 inst: /tmp/virtualenv-20.14.1/.tox/.tmp/package/1/virtualenv-20.14.1.tar.gz
py311 installed: attrs==21.4.0,coverage==6.3.2,coverage-enable-subprocess==1.0,distlib==0.3.4,filelock==3.6.0,flaky==3.7.0,freezegun==1.2.1,iniconfig==1.1.1,packaging==21.3,platformdirs==2.5.2,pluggy==1.0.0,py==1.11.0,pyparsing==3.0.9,pytest==7.1.2,pytest-env==0.6.2,pytest-freezegun==0.4.2,pytest-mock==3.7.0,pytest-randomly==3.12.0,pytest-timeout==2.1.0,python-dateutil==2.8.2,six==1.16.0,tomli==2.0.1,virtualenv @ file:///tmp/virtualenv-20.14.1/.tox/.tmp/package/1/virtualenv-20.14.1.tar.gz
py311 run-test-pre: PYTHONHASHSEED='2515633405'
py311 run-test: commands[0] | python -m coverage erase
py311 run-test: commands[1] | python -m coverage run -m pytest --color=yes --junitxml /tmp/virtualenv-20.14.1/.tox/junit.py311.xml tests --int --timeout 600
========================================================= test session starts =========================================================
platform linux -- Python 3.11.0b1, pytest-7.1.2, pluggy-1.0.0
cachedir: .tox/py311/.pytest_cache
Using --randomly-seed=2173749093
rootdir: /tmp/virtualenv-20.14.1, configfile: setup.cfg
plugins: env-0.6.2, freezegun-0.4.2, mock-3.7.0, randomly-3.12.0, timeout-2.1.0, flaky-3.7.0
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 315 items                                                                                                                   

tests/unit/activation/test_batch.py ss                                                                                          [  0%]
tests/unit/create/via_global_ref/test_build_c_ext.py ..                                                                         [  1%]
tests/unit/config/test_env_var.py ........                                                                                      [  3%]
tests/unit/test_run.py ....                                                                                                     [  5%]
tests/unit/create/via_global_ref/test_api.py .                                                                                  [  5%]
tests/unit/seed/embed/test_base_embed.py ....                                                                                   [  6%]
tests/unit/config/test___main__.py ......                                                                                       [  8%]
tests/unit/seed/embed/test_pip_invoke.py ....                                                                                   [  9%]
tests/unit/activation/test_bash.py ..                                                                                           [ 10%]
tests/unit/seed/wheels/test_acquire.py ........                                                                                 [ 13%]
tests/unit/discovery/windows/test_windows_pep514.py ss                                                                          [ 13%]
tests/unit/create/via_global_ref/builtin/pypy/test_pypy3.py ...                                                                 [ 14%]
tests/unit/discovery/test_discovery.py .......                                                                                  [ 16%]
tests/unit/activation/test_powershell.py ss                                                                                     [ 17%]
tests/unit/activation/test_nushell.py ss                                                                                        [ 18%]
tests/unit/seed/wheels/test_wheels_util.py ....                                                                                 [ 19%]
tests/unit/activation/test_csh.py ss                                                                                            [ 20%]
tests/unit/discovery/test_py_spec.py ...............................................                                            [ 34%]
tests/unit/seed/wheels/test_acquire_find_wheel.py ....                                                                          [ 36%]
tests/unit/activation/test_fish.py ss                                                                                           [ 36%]
tests/unit/activation/test_activation_support.py ............                                                                   [ 40%]
tests/unit/activation/test_python_activator.py ..                                                                               [ 41%]
tests/unit/config/test_ini.py .                                                                                                 [ 41%]
tests/unit/test_util.py s.                                                                                                      [ 42%]
tests/unit/seed/embed/test_bootstrap_link_via_app_data.py ..........                                                            [ 45%]
tests/unit/create/test_creator.py .......s.s.....s.s...ss..s.s.s..s...ss..s...s.......s...                                      [ 63%]
tests/integration/test_run_int.py .                                                                                             [ 63%]
tests/integration/test_zipapp.py ...                                                                                            [ 64%]
tests/unit/activation/test_activator.py .                                                                                       [ 64%]
tests/unit/create/test_interpreters.py ..                                                                                       [ 65%]
tests/unit/seed/wheels/test_bundle.py .....                                                                                     [ 66%]
tests/unit/config/cli/test_parser.py ...                                                                                        [ 67%]
tests/unit/activation/test_activate_this.py .                                                                                   [ 68%]
tests/unit/discovery/py_info/test_py_info_exe_based_of.py .........................                                             [ 76%]
tests/unit/seed/wheels/test_periodic_update.py ...........................                                                      [ 84%]
tests/unit/discovery/py_info/test_py_info.py .....................F...s......................                                   [100%]

============================================================== FAILURES ===============================================================
_____________________________________________ test_custom_venv_install_scheme_is_prefered _____________________________________________

mocker = <pytest_mock.plugin.MockerFixture object at 0x7f053f404e90>

    def test_custom_venv_install_scheme_is_prefered(mocker):
        # The paths in this test are Fedora paths, but we set them for nt as well, so the test also works on Windows,
        # despite the actual values are nonsense there.
        # Values were simplified to be compatible with all the supported Python versions.
        default_scheme = {
            "stdlib": "{base}/lib/python{py_version_short}",
            "platstdlib": "{platbase}/lib/python{py_version_short}",
            "purelib": "{base}/local/lib/python{py_version_short}/site-packages",
            "platlib": "{platbase}/local/lib/python{py_version_short}/site-packages",
            "include": "{base}/include/python{py_version_short}",
            "platinclude": "{platbase}/include/python{py_version_short}",
            "scripts": "{base}/local/bin",
            "data": "{base}/local",
        }
        venv_scheme = {key: path.replace("local", "") for key, path in default_scheme.items()}
        sysconfig_install_schemes = {
            "posix_prefix": default_scheme,
            "nt": default_scheme,
            "pypy": default_scheme,
            "pypy_nt": default_scheme,
            "venv": venv_scheme,
        }
        if getattr(sysconfig, "get_preferred_scheme", None):
            sysconfig_install_schemes[sysconfig.get_preferred_scheme("prefix")] = default_scheme
    
        if sys.version_info[0] == 2:
            sysconfig_install_schemes = _stringify_schemes_dict(sysconfig_install_schemes)
    
        # On Python < 3.10, the distutils schemes are not derived from sysconfig schemes
        # So we mock them as well to assert the custom "venv" install scheme has priority
        distutils_scheme = {
            "purelib": "$base/local/lib/python$py_version_short/site-packages",
            "platlib": "$platbase/local/lib/python$py_version_short/site-packages",
            "headers": "$base/include/python$py_version_short/$dist_name",
            "scripts": "$base/local/bin",
            "data": "$base/local",
        }
        distutils_schemes = {
            "unix_prefix": distutils_scheme,
            "nt": distutils_scheme,
        }
    
        if sys.version_info[0] == 2:
            distutils_schemes = _stringify_schemes_dict(distutils_schemes)
    
        # We need to mock distutils first, so they don't see the mocked sysconfig,
        # if imported for the first time.
        # That can happen if the actual interpreter has the "venv" INSTALL_SCHEME
        # and hence this is the first time we are touching distutils in this process.
        # If distutils saw our mocked sysconfig INSTALL_SCHEMES, we would need
        # to define all install schemes.
        mocker.patch("distutils.command.install.INSTALL_SCHEMES", distutils_schemes)
        mocker.patch("sysconfig._INSTALL_SCHEMES", sysconfig_install_schemes)
    
        pyinfo = PythonInfo()
        pyver = "{}.{}".format(pyinfo.version_info.major, pyinfo.version_info.minor)
>       assert pyinfo.install_path("scripts") == "bin"
E       AssertionError: assert 'local/bin' == 'bin'
E         - bin
E         + local/bin

default_scheme = {'data': '{base}/local', 'include': '{base}/include/python{py_version_short}', 'platinclude': '{platbase}/include/python{py_version_short}', 'platlib': '{platbase}/local/lib/python{py_version_short}/site-packages', ...}
distutils_scheme = {'data': '$base/local', 'headers': '$base/include/python$py_version_short/$dist_name', 'platlib': '$platbase/local/lib/python$py_version_short/site-packages', 'purelib': '$base/local/lib/python$py_version_short/site-packages', ...}
distutils_schemes = {'nt': {'data': '$base/local', 'headers': '$base/include/python$py_version_short/$dist_name', 'platlib': '$platbase/lo...l/lib/python$py_version_short/site-packages', 'purelib': '$base/local/lib/python$py_version_short/site-packages', ...}}
mocker     = <pytest_mock.plugin.MockerFixture object at 0x7f053f404e90>
pyinfo     = PythonInfo({'platform': 'linux', 'implementation': 'CPython', 'version_info': VersionInfo(major=3, minor=11, micro=0, ...stem_stdlib': '/usr/lib/python3.11', 'system_stdlib_platform': '/usr/lib/python3.11', 'max_size': 9223372036854775807})
pyver      = '3.11'
sysconfig_install_schemes = {'nt': {'data': '{base}/local', 'include': '{base}/include/python{py_version_short}', 'platinclude': '{platbase}/inclu...include/python{py_version_short}', 'platlib': '{platbase}/local/lib/python{py_version_short}/site-packages', ...}, ...}
venv_scheme = {'data': '{base}/', 'include': '{base}/include/python{py_version_short}', 'platinclude': '{platbase}/include/python{py_version_short}', 'platlib': '{platbase}//lib/python{py_version_short}/site-packages', ...}

tests/unit/discovery/py_info/test_py_info.py:382: AssertionError
--------------------------------------------------------- Captured log setup ----------------------------------------------------------
DEBUG    filelock:_api.py:169 Attempting to acquire lock 139660512750032 on /tmp/pytest-of-mgorny/pytest-0/session-app-data0/py_info/1/.lock
DEBUG    filelock:_api.py:173 Lock 139660512750032 acquired on /tmp/pytest-of-mgorny/pytest-0/session-app-data0/py_info/1/.lock
DEBUG    filelock:_api.py:203 Attempting to release lock 139660512750032 on /tmp/pytest-of-mgorny/pytest-0/session-app-data0/py_info/1/.lock
DEBUG    filelock:_api.py:206 Lock 139660512750032 released on /tmp/pytest-of-mgorny/pytest-0/session-app-data0/py_info/1/.lock
-------------------------------------------------------- Captured log teardown --------------------------------------------------------
DEBUG    filelock:_api.py:169 Attempting to acquire lock 139660513560720 on /tmp/pytest-of-mgorny/pytest-0/session-app-data0/py_info/1/.lock
DEBUG    filelock:_api.py:173 Lock 139660513560720 acquired on /tmp/pytest-of-mgorny/pytest-0/session-app-data0/py_info/1/.lock
DEBUG    filelock:_api.py:203 Attempting to release lock 139660513560720 on /tmp/pytest-of-mgorny/pytest-0/session-app-data0/py_info/1/.lock
DEBUG    filelock:_api.py:206 Lock 139660513560720 released on /tmp/pytest-of-mgorny/pytest-0/session-app-data0/py_info/1/.lock
========================================================== warnings summary ===========================================================
.tox/py311/lib/python3.11/site-packages/virtualenv/discovery/cached_py_info.py:11
  /tmp/virtualenv-20.14.1/.tox/py311/lib/python3.11/site-packages/virtualenv/discovery/cached_py_info.py:11: DeprecationWarning: 'pipes' is deprecated and slated for removal in Python 3.13
    import pipes

.tox/py311/lib/python3.11/site-packages/pytest_freezegun.py:17: 630 warnings
tests/unit/seed/wheels/test_acquire.py: 2 warnings
tests/unit/seed/wheels/test_periodic_update.py: 30 warnings
  /tmp/virtualenv-20.14.1/.tox/py311/lib/python3.11/site-packages/pytest_freezegun.py:17: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    if LooseVersion(pytest.__version__) < LooseVersion('3.6.0'):

tests/unit/activation/test_batch.py::test_batch[with_prompt]
  /tmp/virtualenv-20.14.1/.tox/py311/lib/python3.11/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils.
    warnings.warn("Setuptools is replacing distutils.")

tests/unit/activation/test_batch.py: 14 warnings
tests/unit/create/via_global_ref/test_build_c_ext.py: 14 warnings
tests/unit/config/test___main__.py: 7 warnings
tests/unit/seed/embed/test_pip_invoke.py: 28 warnings
tests/unit/activation/test_bash.py: 14 warnings
tests/unit/activation/test_powershell.py: 7 warnings
tests/unit/activation/test_nushell.py: 7 warnings
tests/unit/activation/test_csh.py: 14 warnings
tests/unit/activation/test_fish.py: 14 warnings
tests/unit/activation/test_python_activator.py: 14 warnings
tests/unit/seed/embed/test_bootstrap_link_via_app_data.py: 98 warnings
tests/unit/create/test_creator.py: 112 warnings
tests/unit/activation/test_activate_this.py: 1 warning
  /tmp/virtualenv-20.14.1/.tox/py311/lib/python3.11/site-packages/virtualenv/activation/via_template.py:57: DeprecationWarning: read_binary is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
    binary = read_binary(self.__module__, str(template))

tests/unit/activation/test_powershell.py::test_powershell[no_prompt]
tests/unit/activation/test_powershell.py::test_powershell[with_prompt]
tests/unit/activation/test_nushell.py::test_nushell[with_prompt]
tests/unit/activation/test_nushell.py::test_nushell[no_prompt]
tests/unit/activation/test_csh.py::test_csh[with_prompt]
tests/unit/activation/test_csh.py::test_csh[no_prompt]
tests/unit/activation/test_fish.py::test_fish[with_prompt]
tests/unit/activation/test_fish.py::test_fish[no_prompt]
  /tmp/virtualenv-20.14.1/tests/unit/activation/conftest.py:258: PytestRemovedIn8Warning: pytest.skip(msg=...) is now deprecated, use pytest.skip(reason=...) instead
    pytest.skip(msg=six.text_type(version))

tests/unit/discovery/py_info/test_py_info.py::test_custom_venv_install_scheme_is_prefered
  /usr/lib/python3.11/importlib/__init__.py:126: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
    return _bootstrap._gcd_import(name[level:], package, level)

tests/unit/discovery/py_info/test_py_info.py::test_custom_venv_install_scheme_is_prefered
  /usr/lib/python3.11/distutils/command/install.py:13: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
    from distutils.sysconfig import get_config_vars

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
---------------------------------- generated xml file: /tmp/virtualenv-20.14.1/.tox/junit.py311.xml -----------------------------------
======================================================= short test summary info =======================================================
SKIPPED [2] tests/unit/activation/conftest.py:255: BatchActivator not supported
SKIPPED [1] tests/unit/discovery/windows/test_windows_pep514.py:33: no Windows registry
SKIPPED [1] tests/unit/discovery/windows/test_windows_pep514.py:14: no Windows registry
SKIPPED [1] tests/unit/activation/conftest.py:258: PowerShell(
version=FileNotFoundError(2, 'No such file or directory'),
creator=CPython3Posix(dest=/tmp/pytest-of-mgorny/pytest-0/activation-tester-env3/e-$ èрт🚒♞中片-j, clear=False, no_vcs_ignore=False, global=False),
interpreter=PythonInfo(spec=CPython3.11.0.beta.1-64, system=/usr/bin/python3.11, exe=/tmp/virtualenv-20.14.1/.tox/py311/bin/python, platform=linux, version='3.11.0b1 (main, May 11 2022, 10:19:29) [GCC 11.3.0]', encoding_fs_io=utf-8-utf-8)) is not available due [Errno 2] No such file or directory: 'pwsh'
SKIPPED [1] tests/unit/activation/conftest.py:258: PowerShell(
version=FileNotFoundError(2, 'No such file or directory'),
creator=CPython3Posix(dest=/tmp/pytest-of-mgorny/pytest-0/activation-tester-env4/e-$ èрт🚒♞中片-j, clear=False, no_vcs_ignore=False, global=False),
interpreter=PythonInfo(spec=CPython3.11.0.beta.1-64, system=/usr/bin/python3.11, exe=/tmp/virtualenv-20.14.1/.tox/py311/bin/python, platform=linux, version='3.11.0b1 (main, May 11 2022, 10:19:29) [GCC 11.3.0]', encoding_fs_io=utf-8-utf-8)) is not available due [Errno 2] No such file or directory: 'pwsh'
SKIPPED [1] tests/unit/activation/conftest.py:258: Nushell(
version=TypeError('expected str, bytes or os.PathLike object, not NoneType'),
creator=CPython3Posix(dest=/tmp/pytest-of-mgorny/pytest-0/activation-tester-env4/e-$ èрт🚒♞中片-j, clear=False, no_vcs_ignore=False, global=False),
interpreter=PythonInfo(spec=CPython3.11.0.beta.1-64, system=/usr/bin/python3.11, exe=/tmp/virtualenv-20.14.1/.tox/py311/bin/python, platform=linux, version='3.11.0b1 (main, May 11 2022, 10:19:29) [GCC 11.3.0]', encoding_fs_io=utf-8-utf-8)) is not available due expected str, bytes or os.PathLike object, not NoneType
SKIPPED [1] tests/unit/activation/conftest.py:258: Nushell(
version=TypeError('expected str, bytes or os.PathLike object, not NoneType'),
creator=CPython3Posix(dest=/tmp/pytest-of-mgorny/pytest-0/activation-tester-env5/e-$ èрт🚒♞中片-j, clear=False, no_vcs_ignore=False, global=False),
interpreter=PythonInfo(spec=CPython3.11.0.beta.1-64, system=/usr/bin/python3.11, exe=/tmp/virtualenv-20.14.1/.tox/py311/bin/python, platform=linux, version='3.11.0b1 (main, May 11 2022, 10:19:29) [GCC 11.3.0]', encoding_fs_io=utf-8-utf-8)) is not available due expected str, bytes or os.PathLike object, not NoneType
SKIPPED [1] tests/unit/activation/conftest.py:258: Csh(
version=FileNotFoundError(2, 'No such file or directory'),
creator=CPython3Posix(dest=/tmp/pytest-of-mgorny/pytest-0/activation-tester-env6/e-$ èрт🚒♞中片-j, clear=False, no_vcs_ignore=False, global=False),
interpreter=PythonInfo(spec=CPython3.11.0.beta.1-64, system=/usr/bin/python3.11, exe=/tmp/virtualenv-20.14.1/.tox/py311/bin/python, platform=linux, version='3.11.0b1 (main, May 11 2022, 10:19:29) [GCC 11.3.0]', encoding_fs_io=utf-8-utf-8)) is not available due [Errno 2] No such file or directory: 'csh'
SKIPPED [1] tests/unit/activation/conftest.py:258: Csh(
version=FileNotFoundError(2, 'No such file or directory'),
creator=CPython3Posix(dest=/tmp/pytest-of-mgorny/pytest-0/activation-tester-env7/e-$ èрт🚒♞中片-j, clear=False, no_vcs_ignore=False, global=False),
interpreter=PythonInfo(spec=CPython3.11.0.beta.1-64, system=/usr/bin/python3.11, exe=/tmp/virtualenv-20.14.1/.tox/py311/bin/python, platform=linux, version='3.11.0b1 (main, May 11 2022, 10:19:29) [GCC 11.3.0]', encoding_fs_io=utf-8-utf-8)) is not available due [Errno 2] No such file or directory: 'csh'
SKIPPED [1] tests/unit/activation/conftest.py:258: Fish(
version=FileNotFoundError(2, 'No such file or directory'),
creator=CPython3Posix(dest=/tmp/pytest-of-mgorny/pytest-0/activation-tester-env8/e-$ èрт🚒♞中片-j, clear=False, no_vcs_ignore=False, global=False),
interpreter=PythonInfo(spec=CPython3.11.0.beta.1-64, system=/usr/bin/python3.11, exe=/tmp/virtualenv-20.14.1/.tox/py311/bin/python, platform=linux, version='3.11.0b1 (main, May 11 2022, 10:19:29) [GCC 11.3.0]', encoding_fs_io=utf-8-utf-8)) is not available due [Errno 2] No such file or directory: 'fish'
SKIPPED [1] tests/unit/activation/conftest.py:258: Fish(
version=FileNotFoundError(2, 'No such file or directory'),
creator=CPython3Posix(dest=/tmp/pytest-of-mgorny/pytest-0/activation-tester-env9/e-$ èрт🚒♞中片-j, clear=False, no_vcs_ignore=False, global=False),
interpreter=PythonInfo(spec=CPython3.11.0.beta.1-64, system=/usr/bin/python3.11, exe=/tmp/virtualenv-20.14.1/.tox/py311/bin/python, platform=linux, version='3.11.0b1 (main, May 11 2022, 10:19:29) [GCC 11.3.0]', encoding_fs_io=utf-8-utf-8)) is not available due [Errno 2] No such file or directory: 'fish'
SKIPPED [1] tests/unit/test_util.py:19: subprocess patch only applied on Windows python2
SKIPPED [1] tests/unit/create/test_creator.py:315: venv without clear might fail
SKIPPED [2] tests/unit/create/test_creator.py:523: stdlib components without py files only possible on CPython2
SKIPPED [2] tests/unit/create/test_creator.py:467: we rewrite sys.path only on PY2
SKIPPED [8] tests/unit/create/conftest.py:101: could not resolve interpreter based on old_virtualenv because failed to create old virtualenv assert not 100
 +  where 100 = <Popen: returncode: 100 args: ['/tmp/pytest-of-mgorny/pytest-0/env-for-old-v...>.returncode
SKIPPED [1] tests/unit/create/test_creator.py:389: OS does not have an altsep
SKIPPED [1] tests/unit/create/test_creator.py:294: python 2 only tests
SKIPPED [1] tests/conftest.py:390: test not valid if run under system
FAILED tests/unit/discovery/py_info/test_py_info.py::test_custom_venv_install_scheme_is_prefered - AssertionError: assert 'local/bin...
================================ 1 failed, 285 passed, 29 skipped, 1018 warnings in 823.62s (0:13:43) =================================
ERROR: InvocationError for command /tmp/virtualenv-20.14.1/.tox/py311/bin/python -m coverage run -m pytest --color=yes --junitxml /tmp/virtualenv-20.14.1/.tox/junit.py311.xml tests --int --timeout 600 (exited with code 1)
_______________________________________________________________ summary _______________________________________________________________
ERROR:   py311: commands failed

[gaborbernat]

A PR fixing this would be welcome 👍