Potential fix for pylauncher with python version #1186

Gitznik · 2024-01-02T18:06:22Z

I have added an entry to docs/changelog.md

Summary of changes

When pylauncher is detected, check the provided python version to see if it was provided including python (as per our examples). If so, log a warning and remove python from the version, leaving only the semver, as expected by pylauncher.

This might fix #1150

Test plan

Not tested yet, as I'm not on windows and I don't have pylauncher. I've added an automated test that might verify the new behavior.
Maybe someone using pylauncher can assist me here with a manual validation.

# command(s) to exercise these changes

maxwell-k · 2024-01-05T09:20:27Z

TLDR; in my opinion, this change is good ~~and doesn't fix #1150~~

Testing on Linux, this change causes expected behaviour more often and prints a helpful warning message. The change also means that if a version of Python is not installed, all three ways of specifying that version (e.g. (1) 3.11, (2) python3.11, (3) /usr/bin/python3.11) produce the same traceback.

This testing also illustrates another part of #1150 ; without the python launcher installed and using either the main branch or this one the results include five different trace backs. It can be difficult to see how these relate to the command that triggered them. I think my expectation was that these would print a message from which I could understand I need to install the launcher. I appreciate that would be a separate PR to this one.

Testing the main branch for a baseline understanding:

with Python 3.12 installed
without Python 3.11 installed
without python launcher installed

Summary

pipx run pycowsay moo  # expected output
pipx run --python=/usr/bin/python3.12 pycowsay moo  # expected output
pipx run --python=python3.12 pycowsay moo  # trace back 1
pipx run --python=3.12 pycowsay moo  # trace back 2
pipx run --python=/usr/bin/python3.11 pycowsay moo  # trace back 3
pipx run --python=python3.11 pycowsay moo  # trace back 4
pipx run --python=3.11 pycowsay moo  # trace back 5

Testing the pylauncher_fix_version branch:

with Python 3.12 installed
without Python 3.11 installed
without python launcher installed

The summary is as above.

Testing the pylauncher_fix_version branch:

with Python 3.12 installed
without Python 3.11 installed
with python launcher installed

Summary:

pipx run pycowsay moo  # expected output
pipx run --python=/usr/bin/python3.12 pycowsay moo  # expected output
pipx run --python=python3.12 pycowsay moo # warning then expected output
pipx run --python=3.12 pycowsay moo  # expected output
pipx run --python=/usr/bin/python3.11 pycowsay moo  # trace back 6
pipx run --python=python3.11 pycowsay moo  # trace back 6
pipx run --python=3.11 pycowsay moo  # trace back 6

Expected output

  ---
< moo >
  ---
   \   ^__^
    \  (oo)\_______
       (__)\       )\/\
           ||----w |
           ||     ||

Trace back 1

Traceback (most recent call last):
  File "/home/maxwell-k/.local/bin/pipx", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/main.py", line 914, in cli
    return run_pipx_command(parsed_pipx_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/main.py", line 197, in run_pipx_command
    commands.run(
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/commands/run.py", line 200, in run
    run_package(
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/commands/run.py", line 157, in run_package
    _download_and_run(
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/commands/run.py", line 234, in _download_and_run
    venv.create_venv(venv_args, pip_args, override_shared)
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/venv.py", line 160, in create_venv
    venv_process = run_subprocess(cmd + venv_args + [str(self.root)])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/util.py", line 179, in run_subprocess
    completed_process = subprocess.run(
                        ^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib64/python3.12/subprocess.py", line 1950, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/maxwell-k/python3.12'

Trace back 2

Traceback (most recent call last):
  File "/home/maxwell-k/.local/bin/pipx", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/main.py", line 914, in cli
    return run_pipx_command(parsed_pipx_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/main.py", line 197, in run_pipx_command
    commands.run(
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/commands/run.py", line 200, in run
    run_package(
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/commands/run.py", line 157, in run_package
    _download_and_run(
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/commands/run.py", line 234, in _download_and_run
    venv.create_venv(venv_args, pip_args, override_shared)
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/venv.py", line 160, in create_venv
    venv_process = run_subprocess(cmd + venv_args + [str(self.root)])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/util.py", line 179, in run_subprocess
    completed_process = subprocess.run(
                        ^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib64/python3.12/subprocess.py", line 1950, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/maxwell-k/3.12'

Trace back 3

Traceback (most recent call last):
  File "/home/maxwell-k/.local/bin/pipx", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/main.py", line 914, in cli
    return run_pipx_command(parsed_pipx_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/main.py", line 197, in run_pipx_command
    commands.run(
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/commands/run.py", line 200, in run
    run_package(
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/commands/run.py", line 157, in run_package
    _download_and_run(
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/commands/run.py", line 234, in _download_and_run
    venv.create_venv(venv_args, pip_args, override_shared)
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/venv.py", line 160, in create_venv
    venv_process = run_subprocess(cmd + venv_args + [str(self.root)])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/util.py", line 179, in run_subprocess
    completed_process = subprocess.run(
                        ^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib64/python3.12/subprocess.py", line 1950, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/bin/python3.11'

Trace back 4

Traceback (most recent call last):
  File "/home/maxwell-k/.local/bin/pipx", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/main.py", line 914, in cli
    return run_pipx_command(parsed_pipx_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/main.py", line 197, in run_pipx_command
    commands.run(
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/commands/run.py", line 200, in run
    run_package(
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/commands/run.py", line 157, in run_package
    _download_and_run(
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/commands/run.py", line 234, in _download_and_run
    venv.create_venv(venv_args, pip_args, override_shared)
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/venv.py", line 160, in create_venv
    venv_process = run_subprocess(cmd + venv_args + [str(self.root)])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/util.py", line 179, in run_subprocess
    completed_process = subprocess.run(
                        ^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib64/python3.12/subprocess.py", line 1950, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/maxwell-k/python3.11'

Trace back 5

Traceback (most recent call last):
  File "/home/maxwell-k/.local/bin/pipx", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/main.py", line 914, in cli
    return run_pipx_command(parsed_pipx_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/main.py", line 197, in run_pipx_command
    commands.run(
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/commands/run.py", line 200, in run
    run_package(
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/commands/run.py", line 157, in run_package
    _download_and_run(
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/commands/run.py", line 234, in _download_and_run
    venv.create_venv(venv_args, pip_args, override_shared)
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/venv.py", line 160, in create_venv
    venv_process = run_subprocess(cmd + venv_args + [str(self.root)])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/util.py", line 179, in run_subprocess
    completed_process = subprocess.run(
                        ^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib64/python3.12/subprocess.py", line 1950, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/maxwell-k/3.11'

Trace back 6

Traceback (most recent call last):
  File "/home/maxwell-k/.local/bin/pipx", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/main.py", line 911, in cli
    return run_pipx_command(parsed_pipx_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/main.py", line 192, in run_pipx_command
    py_launcher_python = find_py_launcher_python(args.python)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maxwell-k/.local/lib/python3.12/site-packages/pipx/interpreter.py", line 33, in find_py_launcher_python
    py = subprocess.run(
         ^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/py', '-3.11', '-c', 'import sys; print(sys.executable)']' returned non-zero exit status 64.

Gitznik · 2024-01-05T16:08:34Z

I'll convert this to draft for now as it only fixes a subset of the mentioned problems.

Gitznik · 2024-01-06T17:13:12Z

Testing on Linux, this change causes expected behaviour more often and prints a helpful warning message. The change also means that if a version of Python is not installed, all three ways of specifying that version (e.g. (1) 3.11, (2) python3.11, (3) /usr/bin/python3.11) produce the same traceback.

This testing also illustrates another part of #1150 ; without the python launcher installed and using either the main branch or this one the results include five different trace backs. It can be difficult to see how these relate to the command that triggered them. I think my expectation was that these would print a message from which I could understand I need to install the launcher. I appreciate that would be a separate PR to this one.

I've added some changes to unify how the python interpreter is evaluated based on the --python flag. Still need to clean up the now redundant checks and add some tests, but looks promising IMO.

In my testing I am unable to run into any tracebacks, except for the one mentioned in #1195.

Testing the pylauncher_fix_version branch:

No Pylauncher, 3.11 installed, 3.10 not installed

pipx run pycowsay moo # expected output
pipx run --python=/home/robert/.pyenv/shims/python3.11 pycowsay moo # expected output
pipx run --python=python3.11 pycowsay moo # expected output
pipx run --python=3.11 pycowsay moo # warning1
pipx run --python=/home/robert/.pyenv/shims/python3.10 pycowsay moo # warning4
pipx run --python=python3.10 pycowsay moo # warning2
pipx run --python=3.10 pycowsay moo # warning1

Pylauncher, 3.11 installed, 3.10 not installed

pipx run pycowsay moo # expected output
pipx run --python=/usr/bin/python3.11 pycowsay moo # expected output
pipx run --python=python3.11 pycowsay moo # expected output
pipx run --python=3.11 pycowsay moo # expected output
pipx run --python=/usr/bin/python3.10 pycowsay moo # warning4
pipx run --python=python3.10 pycowsay moo # warning3
pipx run --python=3.10 pycowsay moo # warning3

warning1:
⚠️ No executable for the provided Python version '3.XX' found in PATH.
Please make sure the provided version is on your PATH or the file
path is valid. The provided version looks like a version for Python
Launcher, but py was not found on PATH.

warning2:
⚠️ No executable for the provided Python version
'<python version/path>' found in PATH. Please make
sure the provided version is on your PATH or the file path is valid.

warning3:
⚠️ No executable for the provided Python version '<python version/path>'
found in py launcher. Please make sure the provided version is listed
when running py --list.

warning4:
⚠️ No executable for the provided Python version
'<python version/path>' found in PATH. Please make
sure the provided version is on your PATH or the file path is valid.
The provided version looks like a path, but no executable was found there.

src/pipx/commands/install.py

src/pipx/interpreter.py

gaborbernat

Let's rebase and add changelog.

Potential fix for pylauncher with python version

efeb9cf

maxwell-k mentioned this pull request Jan 5, 2024

--python python3.11 results in an error #1150

Closed

Gitznik marked this pull request as draft January 5, 2024 16:08

Gitznik added 2 commits January 6, 2024 11:43

Merge remote-tracking branch 'origin/main' into pylauncher_fix_version

7d82745

Validate provided python version at startup

35014b6

Gitznik added 7 commits January 8, 2024 18:23

Add hint when version looks like path but is not found

09b2e12

add tests

ddba548

fix interpreter resolution test on windows

f6029dc

remove redundant error handling from install command

6211351

Reduce complexity for checking path

5844036

Merge branch 'main' into pylauncher_fix_version

37c350d

Fix changelog

e74cd71

Gitznik marked this pull request as ready for review January 11, 2024 18:09

Gitznik commented Jan 11, 2024

View reviewed changes

src/pipx/commands/install.py Show resolved Hide resolved

guahki reviewed Jan 12, 2024

View reviewed changes

src/pipx/interpreter.py Show resolved Hide resolved

Merge branch 'main' into pylauncher_fix_version

c418297

This was referenced Jan 12, 2024

Pipx managing itself is broken on windows #1203

Closed

--python flag is broken on Windows #1205

Closed

gaborbernat requested changes Jan 14, 2024

View reviewed changes

Gitznik added 2 commits January 14, 2024 21:06

Merge main

af74893

Add news fragment

8e750b3

Gitznik requested a review from gaborbernat January 14, 2024 20:07

Split doc and fix part into seperate fragments

e84a269

gaborbernat approved these changes Jan 14, 2024

View reviewed changes

gaborbernat merged commit 6918c4d into pypa:main Jan 14, 2024
14 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Potential fix for pylauncher with python version #1186

Potential fix for pylauncher with python version #1186

Gitznik commented Jan 2, 2024

maxwell-k commented Jan 5, 2024 •

edited

Loading

Gitznik commented Jan 5, 2024

Gitznik commented Jan 6, 2024 •

edited

Loading

gaborbernat left a comment

Potential fix for pylauncher with python version #1186

Potential fix for pylauncher with python version #1186

Conversation

Gitznik commented Jan 2, 2024

Summary of changes

Test plan

maxwell-k commented Jan 5, 2024 • edited Loading

Gitznik commented Jan 5, 2024

Gitznik commented Jan 6, 2024 • edited Loading

gaborbernat left a comment

Choose a reason for hiding this comment

maxwell-k commented Jan 5, 2024 •

edited

Loading

Gitznik commented Jan 6, 2024 •

edited

Loading