Include a free-threaded PEP703 build in the releases page

[tonybaloney]

Feature or enhancement

Proposal:

Will the next CPython 3.13 releases (alpha 3) include a set of builds that have --disable-gil set?

I'm looking at how free-threaded builds can be tested in CI workflows and the solutions all involve compiling from source with the flags then caching the binaries. It would be a lot easier if the CPython release builds included a free-threaded variant that folks can download. Especially for Windows (because it requires a lot of extra software) and macOS because of the signing requirements.

Related: actions/setup-python#771

CC @hugovk

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

[tonybaloney]

Question mainly for the Python 3.13 release manager (@Yhg1s )

[hugovk]

cc @colesbury, and also @zooba for Windows and @ned-deily and macOS.

---

If you want to test Ubuntu via GitHub Actions, as of yesterday you can use https://github.com/deadsnakes/action with nogil: true:

    - name: Set up Python ${{ matrix.python-version }}
      if: endsWith(matrix.python-version, '-dev')
      uses: deadsnakes/[email protected]
      with:
        python-version: ${{ matrix.python-version }}
        nogil: true

This uses the deadsnakes PPA which already had --disable-gil builds available. For example, see python3.13-nogil under "python3.13 - 3.13.0~a1-1+jammy2" at https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa/+packages

(It's still a bit early to test, and we're also pending https://github.com/pypa/packaging and wheel releases for the cp313t wheel tag.)

[itamaro]

I think it's great that the ecosystem is putting the pieces for a free-threaded build in place, but I wonder if it's too early to include it in the official release, considering that the --disable-gil build still isn't what it says on the tin (as in, the GIL is not yet disabled).

[corona10]

I also agree with @itamaro. It's too early stage to discuss this issue.
If we fail to implement -–disable-gilable CPython in 3.13, no need to publish.
With the current status, even if you enable -–disable-gil right now, the GIL still exists.

[hugovk]

It may be too early to provide the installers, but don't think it's too early to discuss this.

What do we need to do to to prepare, so we're ready when the time comes?

[itamaro]

Reopening because the discussion is relevant

[ronaldoussoren]

FWIW: It is easy to side-by-side install two framework installs of Python using the --with-framework-name option. I use this to build a "PythonDebug.framework" that configured --with-pydebug while keeping "Python.framework" for regular use.

I'm not sure how hard it will be to create a second installer for macOS. Adjusting the "build-installer.py" script is easy enough, but that's just the first step in generating the actual installer.

[hugovk]

We will need to make sure the installers are very clearly labelled experimental on python.org, and make sure people don't install by mistake.

For example, at Pillow, we recently dropped 32-bit wheels for Windows, but will likely re-introduce them partly because a lot of people install 32-bit Python on 64-bit Windows, probably just because it's the first in the list (see python/pythondotorg#2194).

I've opened python/pythondotorg#2336 for this.

[zooba]

I'm inclined to only publish a zip file of binaries for Windows, at least for 3.13.

Trying to make a parallel installer is going to be a bug farm that I simply don't have time for, and unless someone else steps up to take it on (without mentorship! I'm not offering to train someone right now, so you'd better already be a WiX/MSI expert) I think it'll hurt more than it helps. Similarly for Store installs.

Getting reliable CI is more important, though, particularly with PGO enabled. I don't want regular releases to be prevented because something in the nogil build is a problem, and I do want to stick to a single automated release script.

[ned-deily]

FWIW: It is easy to side-by-side install two framework installs of Python using the --with-framework-name option. I use this to build a "PythonDebug.framework" that configured --with-pydebug while keeping "Python.framework" for regular use.

I'm not sure how hard it will be to create a second installer for macOS. Adjusting the "build-installer.py" script is easy enough, but that's just the first step in generating the actual installer.

For 3.13 on macOS, I think the most user-friendly option is to include a free-threaded variant as an optional install in the standard installer download, rather than adding a second installer file. The additional size to the download should have no significant impact as current installer files are currently only 43MB to start with. Ideally, we'd want to allow both variants to be able to to be installed side-by-side; while that should mostly work today, there are some loose ends that would need to be taken care of involving path names and the like. If it turns out to be too disruptive to support co-existing installs, the installer package could simply inhibit the installation of the normal variant if the user selects the optional free-threaded variant.

[zooba]

I think the most user-friendly option is to include a free-threaded variant as an optional install in the standard installer download, rather than adding a second installer file.

This would probably be an easy option on Windows, too, once we get filenames sorted out. All the stdlib extension modules would need two copies, but that's going to be a requirement for everyone else too.

It looks like t is the ABI flag, so would we have python3.13t as the executable?

[hugovk]

It looks like t is the ABI flag, so would we have python3.13t as the executable?

Unknown, hopefully will be answered in python/steering-council#221.

[zooba]

Until that issue is resolved, we can't add any new releases. The SC is about to be elected, so hopefully the new one will make this one of their first decisions.

[hugovk]

Good timing, the old SC has decided on python3.13t:

python/steering-council#221 (comment)

[zooba]

I can't speak for Mac (@ned-deily?), but the Windows installer will have a new option on the advanced page to also install free-threaded binaries. See this post for the overview.

[ned-deily]

Yes, we will provide a very similar install-time customize option for the macOS python.org installer package for 3.13 (in an upcoming alpha) so no changes should be needed to the releases page.

[zooba]

We'll want to get some text added to the release page mentioning that it's in there, though.

Just remembered that I was good and did the docs update, so https://docs.python.org/3.13/using/windows.html#installing-free-threaded-binaries also has the instructions.

[ned-deily]

Update: 3.13.0b2 is now released (https://discuss.python.org/t/3-13-0b2-now-available/55056) and the macOS installer provided for it on python.org now (finally!) includes an optional and experimental python3.13t free-threaded build along with the traditional build. We are using #120098 to track macOS installer-specific issues.