Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-99370: calculate zip path from prefix when in a venv #99371

Merged
merged 1 commit into from
Nov 14, 2022

Conversation

kkpattern
Copy link
Contributor

@kkpattern kkpattern commented Nov 11, 2022

Before python3.11, when in a venv the zip path is calculated from prefix. In python3.11 the behavior is accidentally changed to calculating from the default prefix. This change will break venv created from a non-installed python with a stdlib zip file. This commit restores the behavior back to before python3.11.

@cpython-cla-bot
Copy link

cpython-cla-bot bot commented Nov 11, 2022

All commit authors signed the Contributor License Agreement.
CLA signed

@kkpattern
Copy link
Contributor Author

@zooba PR is created. 🥳

@kkpattern
Copy link
Contributor Author

I looked at the failed Windows test case. It's because it's more complicated to create a "fake" non-installed python in tests on the Windows platform. On the other hand, the related code change only affects POSIX platform. So I think is reasonable to skip test_zippath_from_non_installed on Windows platform?

@zooba
Copy link
Member

zooba commented Nov 11, 2022

I don't think using venv is going to be a good way to create the non-installed environment. It's certainly off-label use, so might change without warning in the future (other than this test, which clearly isn't there to test that functionality).

I'd suggest grabbing a temp dir and copying sys.executable into it, along with whatever other files are needed. You can launch it with PYTHONHOME in the environment to create a venv from it without copying the entire stdlib, and then set up just enough to check sys.path from the venv (running that bit with -S (no site module) should help)

@zooba
Copy link
Member

zooba commented Nov 11, 2022

But also, thinking about this, maybe the venv test isn't proving as much as I first thought. If it doesn't feel like it's falling into place, we probably can live without it.

@kkpattern
Copy link
Contributor Author

Cool. I will first try to update the test to manually create the non-installed environment. If then we still feel it's too tricky, we can remove it. At least it has helped us verify that the new behavior is consistent with python3.10.

@kkpattern kkpattern force-pushed the fix_zippath_venv_non_installed branch from 9b0a2eb to 3110166 Compare November 11, 2022 17:33
@bedevere-bot
Copy link

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

Before python3.11, when in a venv the zip path is calculated
from prefix on POSIX platforms. In python3.11 the behavior is
accidentally changed to calculating from default prefix. This
change will break venv created from a non-installed python
with a stdlib zip file. This commit restore the behavior back
to before python3.11.
@kkpattern kkpattern force-pushed the fix_zippath_venv_non_installed branch from 3110166 to d463c31 Compare November 11, 2022 17:59
@kkpattern
Copy link
Contributor Author

I have updated the test_venv case to manually create the non-installed python and skip on the Windows platform. Turns out sys.executable is the only file that needs to be copied since we already copied the stdlib file.

@zooba
Copy link
Member

zooba commented Nov 14, 2022

Nice work! Thanks!

@zooba zooba added the needs backport to 3.11 only security fixes label Nov 14, 2022
@zooba zooba merged commit e3d4fed into python:main Nov 14, 2022
@miss-islington
Copy link
Contributor

Thanks @kkpattern for the PR, and @zooba for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

@bedevere-bot
Copy link

GH-99477 is a backport of this pull request to the 3.11 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label Nov 14, 2022
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 14, 2022
…GH-99371)

Before python3.11, when in a venv the zip path is calculated
from prefix on POSIX platforms. In python3.11 the behavior is
accidentally changed to calculating from default prefix. This
change will break venv created from a non-installed python
with a stdlib zip file. This commit restores the behavior back
to before python3.11.
(cherry picked from commit e3d4fed)

Co-authored-by: Kai Zhang <[email protected]>
@miss-islington
Copy link
Contributor

Thanks @kkpattern for the PR, and @zooba for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 14, 2022
…GH-99371)

Before python3.11, when in a venv the zip path is calculated
from prefix on POSIX platforms. In python3.11 the behavior is
accidentally changed to calculating from default prefix. This
change will break venv created from a non-installed python
with a stdlib zip file. This commit restores the behavior back
to before python3.11.
(cherry picked from commit e3d4fed)

Co-authored-by: Kai Zhang <[email protected]>
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL7 3.x has failed when building commit e3d4fed.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/15/builds/3415) and take a look at the build logs.
  4. Check if the failure is related to this commit (e3d4fed) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/15/builds/3415

Failed tests:

  • test_venv

Failed subtests:

  • test_zippath_from_non_installed_posix - test.test_venv.BasicTest.test_zippath_from_non_installed_posix

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

409 tests OK.

10 slowest tests:

  • test_concurrent_futures: 2 min 28 sec
  • test_multiprocessing_spawn: 2 min 5 sec
  • test_capi: 1 min 39 sec
  • test_multiprocessing_forkserver: 1 min 20 sec
  • test_asyncio: 1 min 10 sec
  • test_multiprocessing_fork: 1 min 9 sec
  • test_signal: 57.8 sec
  • test_tokenize: 51.2 sec
  • test_socket: 38.8 sec
  • test_io: 38.4 sec

1 test failed:
test_venv

23 tests skipped:
test_check_c_globals test_devpoll test_gdb test_idle test_ioctl
test_kqueue test_launcher test_msilib test_peg_generator
test_smtpnet test_ssl test_startfile test_tcl test_tix
test_tkinter test_ttk test_ttk_textonly test_turtle
test_winconsoleio test_winreg test_winsound test_wmi
test_zipfile64

1 re-run test:
test_venv

Total duration: 3 min 41 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/test_python_zxgkmsoz/tmpiicg4gmk/bin/python', '-m', 'venv', '--without-pip', '/tmp/test_python_zxgkmsoz/tmp475rjj2k']' returned non-zero exit status 127.


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmp1xw18gip/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmpv5h25s15']' returned non-zero exit status 127.

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL8 3.x has failed when building commit e3d4fed.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/185/builds/3001) and take a look at the build logs.
  4. Check if the failure is related to this commit (e3d4fed) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/185/builds/3001

Failed tests:

  • test_venv

Failed subtests:

  • test_zippath_from_non_installed_posix - test.test_venv.BasicTest.test_zippath_from_non_installed_posix

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

416 tests OK.

10 slowest tests:

  • test_gdb: 3 min 32 sec
  • test_concurrent_futures: 2 min 28 sec
  • test_multiprocessing_spawn: 2 min 10 sec
  • test_capi: 1 min 59 sec
  • test_asyncio: 1 min 34 sec
  • test_multiprocessing_forkserver: 1 min 27 sec
  • test_multiprocessing_fork: 1 min 15 sec
  • test_signal: 59.6 sec
  • test_tokenize: 50.7 sec
  • test_socket: 37.1 sec

1 test failed:
test_venv

16 tests skipped:
test_check_c_globals test_devpoll test_ioctl test_kqueue
test_launcher test_msilib test_peg_generator test_startfile
test_tix test_tkinter test_ttk test_winconsoleio test_winreg
test_winsound test_wmi test_zipfile64

1 re-run test:
test_venv

Total duration: 4 min 53 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/test_python_i7na38ea/tmpyijmj39b/bin/python', '-m', 'venv', '--without-pip', '/tmp/test_python_i7na38ea/tmpgh4uugp0']' returned non-zero exit status 127.


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmpigyy2tpg/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmpct5mhiks']' returned non-zero exit status 127.

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE RHEL8 3.x has failed when building commit e3d4fed.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/559/builds/2661) and take a look at the build logs.
  4. Check if the failure is related to this commit (e3d4fed) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/559/builds/2661

Failed tests:

  • test_venv

Failed subtests:

  • test_zippath_from_non_installed_posix - test.test_venv.BasicTest.test_zippath_from_non_installed_posix

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

415 tests OK.

10 slowest tests:

  • test_gdb: 5 min 45 sec
  • test_multiprocessing_spawn: 3 min 41 sec
  • test_concurrent_futures: 3 min 11 sec
  • test_tokenize: 2 min 24 sec
  • test_capi: 2 min 14 sec
  • test_unparse: 2 min 4 sec
  • test_lib2to3: 2 min 1 sec
  • test_asyncio: 1 min 59 sec
  • test_multiprocessing_forkserver: 1 min 58 sec
  • test_multiprocessing_fork: 1 min 19 sec

1 test failed:
test_venv

17 tests skipped:
test_check_c_globals test_devpoll test_ioctl test_kqueue
test_launcher test_msilib test_peg_generator test_perf_profiler
test_startfile test_tix test_tkinter test_ttk test_winconsoleio
test_winreg test_winsound test_wmi test_zipfile64

1 re-run test:
test_venv

Total duration: 10 min 39 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmpliz7n9jt/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmp_mdgppxs']' returned non-zero exit status 127.


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/test_python_e73_pi2s/tmp1in85p3z/bin/python', '-m', 'venv', '--without-pip', '/tmp/test_python_e73_pi2s/tmpvdptg_v6']' returned non-zero exit status 127.

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL8 FIPS Only Blake2 Builtin Hash 3.x has failed when building commit e3d4fed.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/469/builds/3699) and take a look at the build logs.
  4. Check if the failure is related to this commit (e3d4fed) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/469/builds/3699

Failed tests:

  • test_venv

Failed subtests:

  • test_zippath_from_non_installed_posix - test.test_venv.BasicTest.test_zippath_from_non_installed_posix

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

416 tests OK.

10 slowest tests:

  • test_gdb: 4 min 50 sec
  • test_concurrent_futures: 2 min 48 sec
  • test_capi: 2 min 4 sec
  • test_multiprocessing_spawn: 1 min 56 sec
  • test_asyncio: 1 min 43 sec
  • test_multiprocessing_forkserver: 1 min 18 sec
  • test_multiprocessing_fork: 1 min 4 sec
  • test_tokenize: 1 min
  • test_site: 1 min
  • test_signal: 50.4 sec

1 test failed:
test_venv

16 tests skipped:
test_check_c_globals test_devpoll test_ioctl test_kqueue
test_launcher test_msilib test_peg_generator test_startfile
test_tix test_tkinter test_ttk test_winconsoleio test_winreg
test_winsound test_wmi test_zipfile64

1 re-run test:
test_venv

Total duration: 7 min 50 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/test_python_4z4243qe/tmpmdnm_iyj/bin/python', '-m', 'venv', '--without-pip', '/tmp/test_python_4z4243qe/tmpylg_59oj']' returned non-zero exit status 127.


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmpv817i14e/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmpwh51swck']' returned non-zero exit status 127.

miss-islington added a commit that referenced this pull request Nov 14, 2022
Before python3.11, when in a venv the zip path is calculated
from prefix on POSIX platforms. In python3.11 the behavior is
accidentally changed to calculating from default prefix. This
change will break venv created from a non-installed python
with a stdlib zip file. This commit restores the behavior back
to before python3.11.
(cherry picked from commit e3d4fed)

Co-authored-by: Kai Zhang <[email protected]>
@kkpattern
Copy link
Contributor Author

Looks like something is wrong with the PYTHONPATH and the non-installed python cannot find the libpython.so. Is there a way I can reproduce this locally?

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL7 3.11 has failed when building commit 5971a65.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/972/builds/595) and take a look at the build logs.
  4. Check if the failure is related to this commit (5971a65) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/972/builds/595

Failed tests:

  • test_venv

Failed subtests:

  • test_zippath_from_non_installed_posix - test.test_venv.BasicTest.test_zippath_from_non_installed_posix

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

413 tests OK.

10 slowest tests:

  • test_concurrent_futures: 2 min 29 sec
  • test_multiprocessing_spawn: 2 min 1 sec
  • test_multiprocessing_forkserver: 1 min 22 sec
  • test_asyncio: 1 min 12 sec
  • test_multiprocessing_fork: 1 min 12 sec
  • test_capi: 1 min 4 sec
  • test_tokenize: 54.8 sec
  • test_signal: 48.0 sec
  • test_unparse: 45.7 sec
  • test_io: 38.7 sec

1 test failed:
test_venv

20 tests skipped:
test_devpoll test_gdb test_idle test_ioctl test_kqueue
test_launcher test_msilib test_smtpnet test_ssl test_startfile
test_tcl test_tix test_tk test_ttk_guionly test_ttk_textonly
test_turtle test_winconsoleio test_winreg test_winsound
test_zipfile64

1 re-run test:
test_venv

Total duration: 3 min 37 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL7-x86_64/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
KeyError: '/psm_84356f31'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL7-x86_64/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
KeyError: '/psm_7cc210ff'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL7-x86_64/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL7-x86_64/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmpwm6k8n_7/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmpqkjbpuqp']' returned non-zero exit status 127.


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL7-x86_64/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
KeyError: '/psm_162cd031'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL7-x86_64/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL7-x86_64/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmp54j_3rsi/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmp10mc_dko']' returned non-zero exit status 127.

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Ubuntu Shared 3.x has failed when building commit e3d4fed.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/506/builds/3598) and take a look at the build logs.
  4. Check if the failure is related to this commit (e3d4fed) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/506/builds/3598

Failed tests:

  • test_venv

Failed subtests:

  • test_zippath_from_non_installed_posix - test.test_venv.BasicTest.test_zippath_from_non_installed_posix

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

412 tests OK.

10 slowest tests:

  • test_tools: 8 min 37 sec
  • test_gdb: 6 min 48 sec
  • test_multiprocessing_spawn: 4 min 13 sec
  • test_concurrent_futures: 3 min 35 sec
  • test_asyncio: 2 min 38 sec
  • test_capi: 2 min 30 sec
  • test_tokenize: 1 min 55 sec
  • test_multiprocessing_forkserver: 1 min 49 sec
  • test_unparse: 1 min 39 sec
  • test_lib2to3: 1 min 27 sec

1 test failed:
test_venv

20 tests skipped:
test_check_c_globals test_devpoll test_idle test_ioctl test_kqueue
test_launcher test_msilib test_peg_generator test_startfile
test_tcl test_tix test_tkinter test_ttk test_ttk_textonly
test_turtle test_winconsoleio test_winreg test_winsound test_wmi
test_zipfile64

1 re-run test:
test_venv

Total duration: 33 min 28 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/test_python_65orywdj/tmps594v1c8/bin/python', '-m', 'venv', '--without-pip', '/tmp/test_python_65orywdj/tmpujbij78m']' returned non-zero exit status 127.


Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmpw0s7plkr/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmpsddfu_js']' returned non-zero exit status 127.

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE RHEL7 3.11 has failed when building commit 5971a65.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/937/builds/455) and take a look at the build logs.
  4. Check if the failure is related to this commit (5971a65) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/937/builds/455

Failed tests:

  • test_venv

Failed subtests:

  • test_zippath_from_non_installed_posix - test.test_venv.BasicTest.test_zippath_from_non_installed_posix

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

413 tests OK.

10 slowest tests:

  • test_multiprocessing_spawn: 3 min 25 sec
  • test_concurrent_futures: 2 min 59 sec
  • test_tokenize: 2 min 55 sec
  • test_unparse: 2 min 6 sec
  • test_multiprocessing_forkserver: 1 min 54 sec
  • test_capi: 1 min 52 sec
  • test_unicodedata: 1 min 49 sec
  • test_asyncio: 1 min 47 sec
  • test_lib2to3: 1 min 42 sec
  • test_venv: 1 min 35 sec

1 test failed:
test_venv

20 tests skipped:
test_devpoll test_gdb test_idle test_ioctl test_kqueue
test_launcher test_msilib test_smtpnet test_ssl test_startfile
test_tcl test_tix test_tk test_ttk_guionly test_ttk_textonly
test_turtle test_winconsoleio test_winreg test_winsound
test_zipfile64

1 re-run test:
test_venv

Total duration: 8 min 7 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL7-ppc64le/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL7-ppc64le/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmprro6vuy1/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmpaa8nwo9b']' returned non-zero exit status 127.


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL7-ppc64le/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL7-ppc64le/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmpthsvn5_9/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmpvp_g4opd']' returned non-zero exit status 127.

@kkpattern
Copy link
Contributor Author

OK. I figured out what happened. So if we enable shared python build. Besides PYTHONPATH, we also needs to set LD_LIBRARY_PATH(on linux) or DYLD_LIBRARY_PATH(on macos) so what python binary can find libpython.so. I confirmed the fix locally. Is there a way I can manually trigger a buildbot build before we merge the fix?

@kkpattern
Copy link
Contributor Author

I will try to follow the instruction on custom builders to test my fix.

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Fedora Stable 3.11 has failed when building commit 5971a65.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/942/builds/406) and take a look at the build logs.
  4. Check if the failure is related to this commit (5971a65) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/942/builds/406

Failed tests:

  • test_venv

Failed subtests:

  • test_zippath_from_non_installed_posix - test.test_venv.BasicTest.test_zippath_from_non_installed_posix

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

420 tests OK.

10 slowest tests:

  • test_gdb: 3 min 3 sec
  • test_concurrent_futures: 2 min 37 sec
  • test_multiprocessing_spawn: 2 min 23 sec
  • test_asyncio: 1 min 54 sec
  • test_multiprocessing_forkserver: 1 min 27 sec
  • test_capi: 1 min 21 sec
  • test_multiprocessing_fork: 1 min 17 sec
  • test_nntplib: 1 min 8 sec
  • test_tokenize: 1 min 5 sec
  • test_unparse: 52.7 sec

1 test failed:
test_venv

13 tests skipped:
test_devpoll test_ioctl test_kqueue test_launcher test_msilib
test_startfile test_tix test_tk test_ttk_guionly test_winconsoleio
test_winreg test_winsound test_zipfile64

1 re-run test:
test_venv

Total duration: 5 min 14 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-x86_64/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-x86_64/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmpxf8n7116/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmpbd26gncr']' returned non-zero exit status 127.


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-x86_64/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
KeyError: '/psm_ed0dcc6c'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-x86_64/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
KeyError: '/psm_386ca08c'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-x86_64/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
KeyError: '/psm_1f418420'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-x86_64/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-x86_64/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmp1b4hagmi/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmp_l291812']' returned non-zero exit status 127.

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL8 3.11 has failed when building commit 5971a65.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/929/builds/548) and take a look at the build logs.
  4. Check if the failure is related to this commit (5971a65) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/929/builds/548

Failed tests:

  • test_venv

Failed subtests:

  • test_zippath_from_non_installed_posix - test.test_venv.BasicTest.test_zippath_from_non_installed_posix

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

420 tests OK.

10 slowest tests:

  • test_gdb: 4 min 40 sec
  • test_concurrent_futures: 2 min 32 sec
  • test_multiprocessing_spawn: 2 min 7 sec
  • test_asyncio: 1 min 30 sec
  • test_multiprocessing_forkserver: 1 min 25 sec
  • test_multiprocessing_fork: 1 min 15 sec
  • test_capi: 1 min 13 sec
  • test_nntplib: 1 min 7 sec
  • test_tokenize: 54.3 sec
  • test_signal: 47.9 sec

1 test failed:
test_venv

13 tests skipped:
test_devpoll test_ioctl test_kqueue test_launcher test_msilib
test_startfile test_tix test_tk test_ttk_guionly test_winconsoleio
test_winreg test_winsound test_zipfile64

1 re-run test:
test_venv

Total duration: 6 min 10 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-x86_64/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
KeyError: '/psm_922b28e7'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-x86_64/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-x86_64/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmpe6o9e6qc/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmppwfqrlgg']' returned non-zero exit status 127.


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-x86_64/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
KeyError: '/psm_aef5ddaf'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-x86_64/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-x86_64/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmpi4877qzh/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmpc80g6nyg']' returned non-zero exit status 127.


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-x86_64/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
KeyError: '/psm_bf656d9d'

@kkpattern
Copy link
Contributor Author

Hi @zooba looks like I don't have write access to python/cpython so I can't test the code with buildbot-custom. I pushed my fix here. I have tested it on linux and macOS with/without shared enabled. Could you help me test it with a custom buildbot builder? Thanks.

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL8 FIPS Only Blake2 Builtin Hash 3.11 has failed when building commit 5971a65.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/962/builds/549) and take a look at the build logs.
  4. Check if the failure is related to this commit (5971a65) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/962/builds/549

Failed tests:

  • test_venv

Failed subtests:

  • test_zippath_from_non_installed_posix - test.test_venv.BasicTest.test_zippath_from_non_installed_posix

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

420 tests OK.

10 slowest tests:

  • test_gdb: 6 min 8 sec
  • test_concurrent_futures: 2 min 56 sec
  • test_asyncio: 2 min 18 sec
  • test_multiprocessing_spawn: 1 min 57 sec
  • test_capi: 1 min 45 sec
  • test_multiprocessing_forkserver: 1 min 19 sec
  • test_multiprocessing_fork: 1 min 8 sec
  • test_tokenize: 1 min 2 sec
  • test_unparse: 54.6 sec
  • test_signal: 52.8 sec

1 test failed:
test_venv

13 tests skipped:
test_devpoll test_ioctl test_kqueue test_launcher test_msilib
test_startfile test_tix test_tk test_ttk_guionly test_winconsoleio
test_winreg test_winsound test_zipfile64

1 re-run test:
test_venv

Total duration: 10 min 6 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmpxhyybmqj/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmpcv0z3q1h']' returned non-zero exit status 127.


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmpl_z96cfs/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmpnbful785']' returned non-zero exit status 127.

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64 Fedora 3.11 has failed when building commit 5971a65.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/940/builds/607) and take a look at the build logs.
  4. Check if the failure is related to this commit (5971a65) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/940/builds/607

Failed tests:

  • test_venv

Failed subtests:

  • test_zippath_from_non_installed_posix - test.test_venv.BasicTest.test_zippath_from_non_installed_posix

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

413 tests OK.

10 slowest tests:

  • test_tokenize: 7 min 42 sec
  • test_multiprocessing_spawn: 6 min 24 sec
  • test_unicodedata: 5 min 13 sec
  • test_unparse: 4 min 41 sec
  • test_lib2to3: 4 min 28 sec
  • test_concurrent_futures: 3 min 49 sec
  • test_tools: 3 min 40 sec
  • test_multiprocessing_forkserver: 3 min 24 sec
  • test_capi: 3 min 15 sec
  • test_asyncio: 3 min

1 test failed:
test_venv

20 tests skipped:
test_devpoll test_gdb test_idle test_ioctl test_kqueue
test_launcher test_msilib test_smtpnet test_ssl test_startfile
test_tcl test_tix test_tk test_ttk_guionly test_ttk_textonly
test_turtle test_winconsoleio test_winreg test_winsound
test_zipfile64

1 re-run test:
test_venv

Total duration: 55 min 37 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.11.edelsohn-fedora-ppc64/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/shager/cpython-buildarea/3.11.edelsohn-fedora-ppc64/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmp25pvmqbl/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmpqo07m1go']' returned non-zero exit status 127.


Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.11.edelsohn-fedora-ppc64/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/shager/cpython-buildarea/3.11.edelsohn-fedora-ppc64/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmpbzfuffcg/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmpbuappl9n']' returned non-zero exit status 127.

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE Fedora Stable 3.11 has failed when building commit 5971a65.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/933/builds/439) and take a look at the build logs.
  4. Check if the failure is related to this commit (5971a65) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/933/builds/439

Failed tests:

  • test_venv

Failed subtests:

  • test_zippath_from_non_installed_posix - test.test_venv.BasicTest.test_zippath_from_non_installed_posix

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

420 tests OK.

10 slowest tests:

  • test_gdb: 5 min 11 sec
  • test_multiprocessing_spawn: 3 min 30 sec
  • test_concurrent_futures: 3 min 12 sec
  • test_tokenize: 3 min 7 sec
  • test_unparse: 2 min 42 sec
  • test_asyncio: 2 min 13 sec
  • test_unicodedata: 2 min 6 sec
  • test_lib2to3: 1 min 51 sec
  • test_faulthandler: 1 min 42 sec
  • test_multiprocessing_forkserver: 1 min 41 sec

1 test failed:
test_venv

13 tests skipped:
test_devpoll test_ioctl test_kqueue test_launcher test_msilib
test_startfile test_tix test_tk test_ttk_guionly test_winconsoleio
test_winreg test_winsound test_zipfile64

1 re-run test:
test_venv

Total duration: 9 min 4 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-ppc64le/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-ppc64le/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmp_nn1mk_m/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmpv9exb4ss']' returned non-zero exit status 127.


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-ppc64le/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-ppc64le/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmpvze2c7s8/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmpe9xiif3n']' returned non-zero exit status 127.

@zware
Copy link
Member

zware commented Nov 14, 2022

Hi @zooba looks like I don't have write access to python/cpython so I can't test the code with buildbot-custom. I pushed my fix here. I have tested it on linux and macOS with/without shared enabled. Could you help me test it with a custom buildbot builder? Thanks.

Go ahead and create a pull request for it, and we can apply the test-with-buildbots label to test it.

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Ubuntu Shared 3.11 has failed when building commit 5971a65.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/981/builds/570) and take a look at the build logs.
  4. Check if the failure is related to this commit (5971a65) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/981/builds/570

Failed tests:

  • test_venv

Failed subtests:

  • test_zippath_from_non_installed_posix - test.test_venv.BasicTest.test_zippath_from_non_installed_posix

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

416 tests OK.

10 slowest tests:

  • test_tools: 8 min 46 sec
  • test_concurrent_futures: 5 min 25 sec
  • test_multiprocessing_spawn: 4 min 22 sec
  • test_gdb: 3 min 11 sec
  • test_tokenize: 2 min 21 sec
  • test_capi: 2 min 13 sec
  • test_asyncio: 1 min 54 sec
  • test_multiprocessing_forkserver: 1 min 52 sec
  • test_unparse: 1 min 51 sec
  • test_lib2to3: 1 min 38 sec

1 test failed:
test_venv

17 tests skipped:
test_devpoll test_idle test_ioctl test_kqueue test_launcher
test_msilib test_startfile test_tcl test_tix test_tk
test_ttk_guionly test_ttk_textonly test_turtle test_winconsoleio
test_winreg test_winsound test_zipfile64

1 re-run test:
test_venv

Total duration: 33 min 10 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.11.bolen-ubuntu/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
KeyError: '/psm_a9261fc9'


Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.11.bolen-ubuntu/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/srv/buildbot/buildarea/3.11.bolen-ubuntu/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmpan3wpz2e/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmplu4xu1gn']' returned non-zero exit status 127.


Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.11.bolen-ubuntu/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
KeyError: '/psm_3f0e56f4'


Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.11.bolen-ubuntu/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/srv/buildbot/buildarea/3.11.bolen-ubuntu/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmpxy9s25qd/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmpcf51smg9']' returned non-zero exit status 127.

@kkpattern
Copy link
Contributor Author

Go ahead and create a pull request for it, and we can apply the test-with-buildbots label to test it.

Thanks! I have created the PR: #99483

# First try to create a non-installed python. It's not a real full
# functional non-installed python, but enough for this test.
non_installed_dir = os.path.realpath(tempfile.mkdtemp())
try:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than an enormous try block, you can use self.addCleanup(rmtree, non_installed_dir)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Do I need to wait for the buildbots test to finish before pushing the change?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, let's get the failures fixed and clean up afterwards :)

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE RHEL8 3.11 has failed when building commit 5971a65.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/997/builds/440) and take a look at the build logs.
  4. Check if the failure is related to this commit (5971a65) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/997/builds/440

Failed tests:

  • test_venv

Failed subtests:

  • test_zippath_from_non_installed_posix - test.test_venv.BasicTest.test_zippath_from_non_installed_posix

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

420 tests OK.

10 slowest tests:

  • test_gdb: 10 min 21 sec
  • test_multiprocessing_spawn: 4 min 2 sec
  • test_tokenize: 3 min 44 sec
  • test_concurrent_futures: 3 min 24 sec
  • test_mailbox: 2 min 58 sec
  • test_unparse: 2 min 14 sec
  • test_asyncio: 2 min 11 sec
  • test_capi: 2 min 9 sec
  • test_lib2to3: 2 min 6 sec
  • test_unicodedata: 2 min 5 sec

1 test failed:
test_venv

13 tests skipped:
test_devpoll test_ioctl test_kqueue test_launcher test_msilib
test_startfile test_tix test_tk test_ttk_guionly test_winconsoleio
test_winreg test_winsound test_zipfile64

1 re-run test:
test_venv

Total duration: 15 min 46 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-ppc64le/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-ppc64le/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmpub2vbdz6/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmpbto7kawp']' returned non-zero exit status 127.


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-ppc64le/build/Lib/test/test_venv.py", line 593, in test_zippath_from_non_installed_posix
    subprocess.check_call(cmd, env={"PYTHONPATH": pythonpath})
  File "/home/buildbot/buildarea/3.11.cstratak-RHEL8-ppc64le/build/Lib/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/tmpdfrtg95k/bin/python', '-m', 'venv', '--without-pip', '/tmp/tmp675iwfqi']' returned non-zero exit status 127.

CuriousLearner added a commit to CuriousLearner/cpython that referenced this pull request Nov 16, 2022
* main: (8272 commits)
  Update Windows readme.txt to clarify Visual Studio required versions (pythonGH-99522)
  pythongh-99460 Emscripten trampolines on optimized METH_O and METH_NOARGS code paths (python#99461)
  pythongh-92647: [Enum] use final status to determine lookup or create (pythonGH-99500)
  pythongh-81057: Move Globals in Core Code to _PyRuntimeState (pythongh-99496)
  Post 3.12.0a2
  pythongh-99300: Use Py_NewRef() in Python/Python-ast.c (python#99499)
  pythongh-93649: Split pytime and datetime tests from _testcapimodule.c (python#99494)
  pythongh-99370: fix test_zippath_from_non_installed_posix (pythonGH-99483)
  pythonGH-99205: remove `_static` field from `PyThreadState` and `PyInterpreterState` (pythonGH-99385)
  pythongh-81057: Move the Remaining Import State Globals to _PyRuntimeState (pythongh-99488)
  pythongh-87604: Avoid publishing list of active per-interpreter audit hooks via the gc module (pythonGH-99373)
  pythongh-93649: Split getargs tests from _testcapimodule.c (python#99346)
  pythongh-81057: Move Global Variables Holding Objects to _PyRuntimeState. (pythongh-99487)
  pythonGH-98219: reduce sleep time in `asyncio` subprocess test (python#99464)
  pythonGH-99388: add `loop_factory` parameter to `asyncio.run` (python#99462)
  pythongh-99300: Use Py_NewRef() in PC/ directory (python#99479)
  pythongh-99300: Use Py_NewRef() in Doc/ directory  (python#99480)
  pythongh-99300: Use Py_NewRef() in Modules/ directory (python#99473)
  pythongh-99300: Use Py_NewRef() in Modules/ directory (python#99469)
  pythongh-99370: Calculate zip path from prefix when in a venv (pythonGH-99371)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants