-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Build Python packages using the limited API #42
Comments
It is worth noting that the Python Buffer Protocol C API landed in Python 3.11 (additional ref). So think that is a minimum for us Also find this listing of functions in the Limited and Stable API quite helpful |
Yes. I have been able to build most of RAPIDS using Cython's limited API supported (along with some additional changes I have locally) in Python 3.11. Python 3.11 is definitely a must. But as I said above in the "intermediate vs long-term" bullet, we could still benefit before dropping Python<3.11 support by building one wheel for each older Python version and then build an abi3 wheel to be used for Python 3.11+. |
I've made PRs ro rmm, raft, and cuml that address the issues in those repos. I've also taken steps to remove ucxx's usage of the the numpy C API (#41), which in turn removes one of its primary incompatibilities. The last major issue in RAPIDS code that I see is the usage of the array module in the Array class that is vendored by both kvikio and ucxx (and ucx-py). If that can be removed, then I think we'll be in good shape on the RAPIDS end, and we'll just be waiting on support for this feature in Cython itself. @jakirkham expressed interest in helping out with that in the process of making that Array class more broadly usable. |
A small warning here: There's definitely places where Cython is substituting private C API for private Python API, so future compatibility definitely isn't guaranteed (it'll just be a runtime failure rather than a compile-time failure). We'll see how that evolves - I hope to be able to make some of these warnings rather than failures (since it's largely just non-essential introspection support). We're also having to build a few more runtime version-checks into our code. Which is obviously a little risky because although you're compiling the same thing, you're taking different paths on different Python versions. So the upshot is that your testing matrix probably doesn't reduce to a single version. (From Cython's point of view the testing matrix probably expands, because we really should be testing combinations like |
Thanks for chiming in here @da-woods! I appreciate your comments. I agree that there is more complexity around testing here than simply a set and forget single version. At present, RAPIDS typically supports 2 or 3 Python versions at a time. We tend to lag a bit behind NEP 29/SPEC 0 timelines, so we support older versions a bit longer at the expense of not supporting new ones until they've been out for a bit. A significant part of the resource constraint equation for us is certainly on the testing side since running our full test suites on multiple Python versions adds up quickly. The way that I had envisioned this working, if we did move forward, would be that we built on the oldest supported Python (e.g. |
This PR removes usage of the only method in raft's Cython that is not part of the Python limited API. Contributes to rapidsai/build-planning#42 Authors: - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - Dante Gama Dessavre (https://github.com/dantegd) URL: #5871
This PR removes usage of the only method in rmm's Cython that is not part of the Python limited API. Contributes to rapidsai/build-planning#42 Authors: - Vyas Ramasubramani (https://github.com/vyasr) - https://github.com/jakirkham Approvers: - https://github.com/jakirkham URL: #1545
This PR removes usage of the only method in raft's Cython that is not part of the Python limited API. Contributes to rapidsai/build-planning#42 Authors: - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - Dante Gama Dessavre (https://github.com/dantegd) URL: #2282
This PR removes usage of the only method in raft's Cython that is not part of the Python limited API. Contributes to rapidsai/build-planning#42 Authors: - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#2282
With the latest versions of branch-24.10, which contain a number of changes I made over the past few months for limited API compatibility along with the removal of pyarrow and numpy as build requirements in cudf and cuspatial, most of RAPIDS now builds with the limited API flag on. I have run some smoke tests and things generally work OK, but I haven't done anything extensive. ucxx and kvikio remain outstanding since we need to rewrite the Array class to not use the Python array's C API since that does not support the limited API. The latest tests can be seen in rapidsai/devcontainers#278. |
I don't know if it's any help, but the quickest non-
It isn't as good, but it's surprisingly close given how much it actually does. "Only" 70% slower. You probably have to replace
with |
Thanks for the tip David! That could be helpful, but I'll have to look at the Array class more closely to be sure. I suspect that there are larger refactorings of our code base that could be done to make this unnecessary. |
In `Array`, `Py_ssize_t[::1]` objects are currently backed by [CPython `array`'s]( https://cython.readthedocs.io/en/latest/src/userguide/memoryviews.html#cpython-array-module ) with some internal bits expressed in Cython. However these are not compatible with [Python's Limited API and Stable ABI]( https://docs.python.org/3/c-api/stable.html#c-api-stability ). To address that, switch to [Cython's own `array` type]( https://cython.readthedocs.io/en/latest/src/userguide/memoryviews.html#cython-arrays ). As this is baked into Cython and doesn't use anything special, it is compatible with Python's Limited API and Stable ABI. xref: rapidsai/build-planning#42 Authors: - https://github.com/jakirkham Approvers: - Peter Andreas Entschev (https://github.com/pentschev) URL: #1087
In `Array`, `Py_ssize_t[::1]` objects are currently backed by [CPython `array`'s]( https://cython.readthedocs.io/en/latest/src/userguide/memoryviews.html#cpython-array-module ) with some internal bits expressed in Cython. However these are not compatible with [Python's Limited API and Stable ABI]( https://docs.python.org/3/c-api/stable.html#c-api-stability ). To address that, switch to [Cython's own `array` type]( https://cython.readthedocs.io/en/latest/src/userguide/memoryviews.html#cython-arrays ). As this is baked into Cython and doesn't use anything special, it is compatible with Python's Limited API and Stable ABI. xref: rapidsai/build-planning#42 Authors: - https://github.com/jakirkham Approvers: - Mads R. B. Kristensen (https://github.com/madsbk) - Peter Andreas Entschev (https://github.com/pentschev) URL: #307
In `Array`, `Py_ssize_t[::1]` objects are currently backed by [CPython `array`'s]( https://cython.readthedocs.io/en/latest/src/userguide/memoryviews.html#cpython-array-module ) with some internal bits expressed in Cython. However these are not compatible with [Python's Limited API and Stable ABI]( https://docs.python.org/3/c-api/stable.html#c-api-stability ). To address that, switch to [Cython's own `array` type]( https://cython.readthedocs.io/en/latest/src/userguide/memoryviews.html#cython-arrays ). As this is baked into Cython and doesn't use anything special, it is compatible with Python's Limited API and Stable ABI. xref: rapidsai/build-planning#42 Authors: - https://github.com/jakirkham Approvers: - Mads R. B. Kristensen (https://github.com/madsbk) URL: #504
Had played with a few different approaches to rewrite That should address this piece of this issue. Though am sure there are other things we may still need to do |
Wow that's awesome. Thanks for trying that out John! I'll refresh rapidsai/devcontainers#278 for ucxx and kvikio and see what breaks next. |
Looks like ucxx is using a few functions that aren't part of the limited API ( |
Linking some background on that that might be helpful: rapidsai/ucxx#276 (comment) |
Looking at the CPython API docs for That said the PR James linked to uses The |
It is worth noting that Python 3.13's free-threading build (GIL disabled) is not compatible with the Limited C API or Stable ABI Of course the non-free-threading (GIL enabled) Python 3.13 can use the Limited C API and Stable ABI |
Yeah I'm not thinking too much about the free-threading builds yet. Given the phrasing:
(emphasis mine). It seems to me like they can't make free threaded builds the default without something akin to a major version bump unless they get the limited/stable ABI working, because without that all of a sudden it would be valid to install a bunch of incompatible packages into an environment. Either that, or they try and get installers pip/uv/etc to handle this, but even then you'll see tons of errors from people with older versions of those installers in environments that don't get updated. For the moment I'm OK waiting to see what direction this goes, but I'd personally be pretty surprised if they decided to break the limited API altogether to get free threading in as default. |
The reason I mention it is most of the value of the Stable ABI and Limited API is building a package for one Python version and reusing that package for multiple versions. However with this change in Python 3.13 and the fact that the Python Buffer Protocol was only added in Python 3.11, it means we can only confidently build something for Python 3.11 and allow it to be installed for Python 3.12. Before Python 3.11 we lack the Python Buffer Protocol, so have to build per Python version there. Starting with Python 3.13, we lack an ability to constrain a package to only non-free-threading builds. IOW we have to assume a package using the Stable ABI and Limited API could wind up being installed on a free-threading build of Python unexpectedly. So we would need to build for Python 3.13 separately (and if we want to build for free-threading we would need to add that as an additional build). IOW we are stuck with an island of compatibility at the moment with Python 3.11 and 3.12 on the Stable ABI and Limited API. |
This can't be true... The page you linked above says:
Installers have to respect this. If they don't, free-threaded builds would start breaking all over the places when incompatible extensions are installed. My assumption is that for as long as free-threaded builds are not the default build, if we want to support them we will have to build separate wheels for them, and if that is the case we could use limited API builds for the default builds and then separate builds for each Python version under free threading. I don't think anyone knows what will happen at the point when free-threading becomes the default yet, though. |
What I'm trying to say though is there is not a way to separate them AFAIK. Though if you would like to outline a proposal, would be interested to read it |
I don't understand what you mean by "there is not a way to separate them".
The package files are delineated, so as long as you appropriately tag wheels that are built with the limited API and installers are made aware of this too, what more do we need? |
Could you please outline a proposal of how you see this being used? How will we build for/support/package RAPIDS for each Python version in the range 3.10-3.13 (including free-threading)? Think that will make it easier to identify and discuss edge cases |
Let's use 3.11-3.14 so that we have two free-threading releases. Let's just pick rmm to simplify the discussion. I would build one abi3 wheel for the non free-threaded builds that works on all Python>=3.11, and then I would build two free-threaded wheels, one for 3.13 and 3.14. When 3.15 is released, I would add a new 3.15 free-threaded build, with no change to the not free-threaded build. Is that what you were looking for in a proposal? Of course, there are plenty of open questions to address:
|
Thanks Vyas! 🙏 That's a good start Now for the package using the limited API/stable ABI, how will the pack metadata capture its intended compatibility? IOW what would go in |
I don't think anything changes in pyproject.toml. It's still the same package. When you build the wheel you have to specify the appropriate flags and it changes the output filename. I don't know which build backends are currently supporting the wheel name appropriately (since we have to add the |
Just want to mention PyPA has a tool called The "Motivation" section of the docs explains this well: https://github.com/pypa/abi3audit?tab=readme-ov-file#motivation Context for creation of that tool: pypa/auditwheel#395 |
You can see an example of using abi3audit in this pynvjitlink PR. |
Python has a limited API that is guaranteed to be stable across minor releases. Any code using the Python C API that limits itself to using code in the limited API is guaranteed to also compile on future minor versions of Python within the same major family. More importantly, all symbols in the current (and some historical) version of the limited API are part of Python's stable ABI, which also does not change between Python minor versions and allows extensions compiled against one Python version to continue working on future versions of Python.
Currently RAPIDS builds a single wheel per Python version. If we were to compile using the Python stable ABI, we would be able to instead build a single wheel that works for all Python versions that we support. There would be a number of benefits here:
Here are the tasks (some ours, some external) that need to be accomplished to make this possible:
At this stage, it is not yet clear whether the tradeoffs required will be worthwhile, or at what point the ecosystem's support for the limited API will be reliable enough for us to use in production. However, it shouldn't be too much work to get us to the point of at least being able to experiment with limited API builds, so we can start answering questions around performance and complexity fairly soon. I expect that we can pretty easily remove explicit reliance on any APIs that are not part of the stable ABI, at which point this really becomes a question of the level of support our binding tools provide and if/when we're comfortable with those.
The text was updated successfully, but these errors were encountered: