--python flag is broken on Windows

[guahki]

Describe the bug

The --python flag used to work with a executable name in the PATH. E.g. pipx install --python python3.12.exe ... worked, if python3.12.exe was found on the path.

Since #1168, however, this does no longer work. See analysis below.

$ where python3.12.exe
C:\Users\Dominik\AppData\Local\Microsoft\WindowsApps\python3.12.exe
$ pipx install --python python3.12.exe flit
creating virtual environment...

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\not_of_interest\Scripts\pipx.exe\__main__.py", line 7, in <module>
  File "C:\not_of_interest\Lib\site-packages\pipx\main.py", line 911, in cli
    return run_pipx_command(parsed_pipx_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\not_of_interest\Lib\site-packages\pipx\main.py", line 212, in run_pipx_command
    return commands.install(
           ^^^^^^^^^^^^^^^^^
  File "C:\not_of_interest\Lib\site-packages\pipx\commands\install.py", line 83, in install
    venv.create_venv(venv_args, pip_args, override_shared)
  File "C:\not_of_interest\Lib\site-packages\pipx\venv.py", line 160, in create_venv
    venv_process = run_subprocess(cmd + venv_args + [str(self.root)])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\not_of_interest\Lib\site-packages\pipx\util.py", line 179, in run_subprocess
    completed_process = subprocess.run(
                        ^^^^^^^^^^^^^^^
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.496.0_x64__qbz5n2kfra8p0\Lib\subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.496.0_x64__qbz5n2kfra8p0\Lib\subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.496.0_x64__qbz5n2kfra8p0\Lib\subprocess.py", line 1538, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [WinError 2] Das System kann die angegebene Datei nicht finden

How to reproduce

On a windows system, pass any python executable available in the PATH.

Expected behavior

It should be possible to pass executables to be found on the PATH to the --python flag.

Analysis

The changes introduced by #1168 call os.path.realpath on the argument before giving it to subprocess.run. This causes the current working directory to be prepended and subprocess.run can not lookup the executable, as it gets a resolved/absolute (non-existing) path. Note, this is even shadowed in the verbose log, as this prints a version of the cmd, which was not passed through os.path.realpath.

See my initial direct suggestion in #1186 (comment) to fix (or at least workaround) the issue. I tried to be lazy and not open an issue, but @Gitznik rightfully told me to do so.

[Gitznik]

Thank you for opening a separate issue :)

Would be great if someone running windows could pick this up. Changes need verifying, that

1. Invocations with py launcher versions still work
2. invocation as described here with a python version on the PATH work
3. Invocations providing the full path to the executable work
4. Invocations with versions that are subject to path redirection still work (see pipx unable to create venv when using Python from Microsoft Store #1164)

I might miss some possible invocations methods.

[chrysle]

This actually sounds like a duplicate of #1189.

[guahki]

@Gitznik: thanks for your support. If I can help with anything, let me know.

@chrysle: no, #1189 is about the console script getting appended with .exe twice. Nothing to do with the Python interpreter in use. It might be, that the source is similar, but I doubt it. Especially, since the author says, it is broken since 1.2.x (#1189 (comment)), and #1168 (which broke the behavior descibed here) was introduced in 1.4.0.

[Gitznik]

@guahki if you're interested you're more than welcome to contribute a fix. Another option I see is to set strict=True on os.path.realpath, see here, and only use the value realpath creates if it does not error out. If it errors -> use the original value.

[guahki]

I opened a PR implementing a quite easy solution: just don't do the realpath, if the path was not abolute.

I played around with it and as far as I can tell, "requirements" 1-3 mentioned above by @Gitznik hold true. I can't test the 4., as I can not reproduce the original issue (I'm using Windows Store Python since multiple years and never had this problem).

[Gitznik]

@dechamps any chance you can help us verify the original issue solved in #1168 is still working with @guahki's PR?

[dechamps]

@dechamps any chance you can help us verify the original issue solved in #1168 is still working with @guahki's PR?

Commented on #1232 directly.