build[uv] uses wrong Python when testing on Windows

[njzjz]

Description

It worked yesterday but failed today.

See below. The cp311-win_amd64 build uses Python 3.7.9 to create the environment.

Building cp311-win_amd64 wheel
CPython 3.11 Windows 64bit
...
Testing wheel...
  
  + uv venv 'C:\Users\runneradmin\AppData\Local\Temp\cibw-run-90ozl8o3\cp311-win_amd64\venv-test' --python=python
  Using Python 3.7.9 interpreter at: C:\hostedtoolcache\windows\Python\3.7.9\x64\python3.exe
  Creating virtualenv at: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-90ozl8o3\cp311-win_amd64\venv-test
  Activate with: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-90ozl8o3\cp311-win_amd64\venv-test\Scripts\activate
  + where python
  C:\Users\runneradmin\AppData\Local\Temp\cibw-run-90ozl8o3\cp311-win_amd64\venv-test\Scripts\python.exe
  C:\Users\runneradmin\AppData\Local\Temp\cibw-run-90ozl8o3\cp311-win_amd64\build\venv\Scripts\python.exe
  C:\hostedtoolcache\windows\Python\3.7.9\x64\python.exe
  + uv pip install 'C:\Users\runneradmin\AppData\Local\Temp\cibw-run-90ozl8o3\cp311-win_amd64\repaired_wheel\deepmd_kit-3.0.0a1.dev211+g4a27dec7-py37-none-win_amd64.whl[cpu]'
    × No solution found when resolving dependencies:
    ╰─▶ Because only deepmd-kit[cpu]==3.0.0a1.dev211+g4a27dec7 is available and
        the current Python version (3.7.9) does not satisfy Python>=3.8, we can
        conclude that all versions of deepmd-kit[cpu] are incompatible.
        And because you require deepmd-kit[cpu], we can conclude that the
        requirements are unsatisfiable.
                                                              ✕ 6.49s
Error: Command ['uv', 'pip', 'install', 'C:\\Users\\runneradmin\\AppData\\Local\\Temp\\cibw-run-90ozl8o3\\cp311-win_amd64\\repaired_wheel\\deepmd_kit-3.0.0a1.dev211+g4a27dec7-py37-none-win_amd64.whl[cpu]'] failed with code 1. 

Build log

https://github.com/deepmodeling/deepmd-kit/actions/runs/9483099069/job/26133743134

CI config

https://github.com/deepmodeling/deepmd-kit/blob/devel/.github/workflows/build_wheel.yml

[henryiii]

Could you check the previous version of uv? https://github.com/deepmodeling/deepmd-kit/blob/73dab63fa847acb1acd0aa76700130e10a8e8516/.github/workflows/build_wheel.yml#L78 can be modified to install a specific version. Like https://astral.sh/uv/0.2.6/install.sh.

We specify the Python version at

cibuildwheel/cibuildwheel/windows.py

Line 521 in 78da7ba

virtualenv_env["VIRTUAL_ENV"] = str(venv_dir)

[njzjz]

It works with uv 0.2.6: https://github.com/deepmodeling/deepmd-kit/actions/runs/9488527936/job/26147696043

Testing wheel...
  
  + uv venv 'C:\Users\runneradmin\AppData\Local\Temp\cibw-run-l0v678gi\cp311-win_amd64\venv-test' --python=python
  Using Python 3.11.9 interpreter at: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-l0v678gi\cp311-win_amd64\build\venv\Scripts\python.exe
  Creating virtualenv at: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-l0v678gi\cp311-win_amd64\venv-test
  Activate with: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-l0v678gi\cp311-win_amd64\venv-test\Scripts\activate
  + where python
  C:\Users\runneradmin\AppData\Local\Temp\cibw-run-l0v678gi\cp311-win_amd64\venv-test\Scripts\python.exe
  C:\Users\runneradmin\AppData\Local\Temp\cibw-run-l0v678gi\cp311-win_amd64\build\venv\Scripts\python.exe
  C:\hostedtoolcache\windows\Python\3.7.9\x64\python.exe

[henryiii]

Can you try with the latest one and RUST_LOG set to uv=trace? Our CI is using the latest one and isn't broken.

[njzjz]

Testing wheel...
  + uv venv 'C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\venv-test' --python=python
  DEBUG Searching for Python  in search path or `py` launcher output
  TRACE Searching PATH for executables: python3.exe, python.exe
  TRACE Checking `PATH` directory for interpreters: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\build\venv
  TRACE Checking `PATH` directory for interpreters: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\build\venv\Scripts
  TRACE Found possible Python executable: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\build\venv\Scripts\python.exe
  TRACE Cached interpreter info for Python 3.11.9, skipping probing: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\build\venv\Scripts\python.exe
  DEBUG Found CPython 3.11.9 at `C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\build\venv\Scripts\python.exe` (search path)
  DEBUG Ignoring Python interpreter at `C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\build\venv\Scripts\python.exe`: system interpreter required
  TRACE Checking `PATH` directory for interpreters: C:\Program Files\PowerShell\7
  TRACE Checking `PATH` directory for interpreters: C:\Program Files\MongoDB\Server\5.0\bin
  TRACE Checking `PATH` directory for interpreters: C:\aliyun-cli
  TRACE Checking `PATH` directory for interpreters: C:\vcpkg
  TRACE Checking `PATH` directory for interpreters: C:\cf-cli
  TRACE Checking `PATH` directory for interpreters: C:\Program Files (x86)\NSIS\
  TRACE Checking `PATH` directory for interpreters: C:\tools\zstd
  TRACE Checking `PATH` directory for interpreters: C:\Program Files\Mercurial\
  TRACE Checking `PATH` directory for interpreters: C:\hostedtoolcache\windows\stack\2.15.7\x64
  TRACE Checking `PATH` directory for interpreters: C:\\ghcup\bin
  TRACE Checking `PATH` directory for interpreters: C:\mingw64\bin
  TRACE Checking `PATH` directory for interpreters: C:\Program Files\dotnet
  TRACE Checking `PATH` directory for interpreters: C:\Program Files\MySQL\MySQL Server 5.7\bin
  TRACE Checking `PATH` directory for interpreters: C:\Program Files\R\R-4.4.0\bin\x64
  TRACE Checking `PATH` directory for interpreters: C:\SeleniumWebDrivers\GeckoDriver
  TRACE Checking `PATH` directory for interpreters: C:\SeleniumWebDrivers\EdgeDriver\
  TRACE Checking `PATH` directory for interpreters: C:\SeleniumWebDrivers\ChromeDriver
  TRACE Checking `PATH` directory for interpreters: C:\Program Files (x86)\sbt\bin
  TRACE Checking `PATH` directory for interpreters: C:\Program Files\Git\bin
  TRACE Checking `PATH` directory for interpreters: C:\Program Files (x86)\pipx_bin
  TRACE Checking `PATH` directory for interpreters: C:\npm\prefix
  TRACE Checking `PATH` directory for interpreters: C:\hostedtoolcache\windows\go\1.21.11\x64\bin
  TRACE Checking `PATH` directory for interpreters: C:\hostedtoolcache\windows\Python\3.7.9\x64\Scripts
  TRACE Checking `PATH` directory for interpreters: C:\hostedtoolcache\windows\Python\3.7.9\x64
  TRACE Found possible Python executable: C:\hostedtoolcache\windows\Python\3.7.9\x64\python3.exe
  TRACE Querying interpreter executable at C:\hostedtoolcache\windows\Python\3.7.9\x64\python3.exe
  DEBUG Found CPython 3.7.9 at `C:\hostedtoolcache\windows\Python\3.7.9\x64\python3.exe` (search path)
  Using Python 3.7.9 interpreter at: C:\hostedtoolcache\windows\Python\3.7.9\x64\python3.exe
  Creating virtualenv at: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\venv-test
  Activate with: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\venv-test\Scripts\activate
  + where python
  C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\venv-test\Scripts\python.exe
  C:\Users\runneradmin\AppData\Local\Temp\cibw-run-x8apk4mi\cp311-win_amd64\build\venv\Scripts\python.exe
  C:\hostedtoolcache\windows\Python\3.7.9\x64\python.exe

[zanieb]

Hi! What's happening here is the Python interpreter at the top of the PATH is inside a virtual environment so we skip it. We don't use interpreters found inside virtual environments to create virtual environments.

[zanieb]

And now I see why you were using the --python=python flag, we changed this behavior to be consistent in astral-sh/uv#3728 because it seemed confusing otherwise.

edit: Henry pointed out this pull request isn't the reason, investigating what pull request changed it.

[mayeut]

Our CI is using the latest one and isn't broken.

I'm not sure there are tests for tests (except on macOS).

[zanieb]

Thanks looks like you found an unexpected change in the latest release astral-sh/uv#4289

I would still strongly recommend not using --python=python — its a weird way to request the interpreter you want and I will probably change the behavior here to be more consistent.

[henryiii]

Passing through the proper path in #1881. Could you test the branch, @njzjz?

[njzjz]

Passing through the proper path in #1881. Could you test the branch, @njzjz?

Thanks. It works:

Testing wheel...
  + uv venv 'C:\Users\runneradmin\AppData\Local\Temp\cibw-run-d2jic__1\cp311-win_amd64\venv-test' '--python=C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\python.3.11.9\tools\python.exe'
  DEBUG Checking for Python interpreter at path `C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\python.3.11.9\tools\python.exe`
  TRACE Cached interpreter info for Python 3.11.9, skipping probing: C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\python.3.11.9\tools\python.exe
  Using Python 3.11.9 interpreter at: C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\python.3.11.9\tools\python.exe
  Creating virtualenv at: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-d2jic__1\cp311-win_amd64\venv-test
  Activate with: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-d2jic__1\cp311-win_amd64\venv-test\Scripts\activate