Add Rust beta + Python beta to the test matrix

[IvanIsCoding]

After #1064 we no longer depend on new PyO3 releases to support newer Python versions. Hence this is an attempt to use the latest versions of Rust and Python to catch issues early

[IvanIsCoding]

I need to handle SciPy and possibly NumPy not being available. It seemed to easy to be true, but oh well

[jakelishman]

Iirc, Numpy and SciPy started publishing test wheels for 3.12 for their releases after 3.12rc1 came out in early August 2023 (that's the point at which CPython guarantees ABI stability). 3.13rc1 is due on the 30th of July this year.

[IvanIsCoding]

Iirc, Numpy and SciPy started publishing test wheels for 3.12 for their releases after 3.12rc1 came out in early August 2023 (that's the point at which CPython guarantees ABI stability). 3.13rc1 is due on the 30th of July this year.

I will try to make a weekly build with an image that has the toolchain to compile NumPy. I don’t think we need to wait for NumPy pre-built wheels, ideally we’d be able to test 3.14 alpha the day it is available for GitHub actions

[jakelishman]

That's fair, though if you want to go that way, it might be faster to pull the pre-built nightlies from the conda nightlies: https://anaconda.org/scientific-python-nightly-wheels/numpy (etc for scipy).

[jakelishman]

ofc that means also dealing with numpy.next in addition to python.next and rust.next, but that might actually be a good thing, given your intent here?

[IvanIsCoding]

That's fair, though if you want to go that way, it might be faster to pull the pre-built nightlies from the conda nightlies: https://anaconda.org/scientific-python-nightly-wheels/numpy (etc for scipy).

That is a clever idea, I don’t mind using Conda as it is a non-blocking test. We no longer depend on SciPy so it’s just NumPy.

Overall my goal is to test how true our ABI compatibility is. We released 0.15 tested with 3.8-3.12 today but it might run with 3.13 or even with 3.14.

[jakelishman]

I don't think you need to use conda as your package manager - there's a PyPI-compatible interface at https://pypi.anaconda.org/scientific-python-nightly-wheels/simple, so you can point pip at it with pip install -i <url> numpy scipy. I can't remember what the rules on repo fallback are, and there was some sort of supply-chain attack based on that premise within the last year, so it might have changed - if so, you might need to ensure your other dependencies are manually satisfied from regular PyPI first.

[IvanIsCoding]

I think this is working now, I had to remove Pillow from the 3.13-dev install because it is not released. But I am glad our code works with 3.13! Adopting the ABI was a good choice.

[coveralls]

Pull Request Test Coverage Report for Build 10912348902

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage remained the same at 95.816%

---

Totals
Change from base Build 10888128918:	0.0%
Covered Lines:	17999
Relevant Lines:	18785

---

💛 - Coveralls

[mtreinish]

LGTM, lets give this a try. I'm a bit worried about the long term stability of using rust beta in CI. It should be fine, but having run nightly for a long time in the early project (out of necessity because pyo3 was only compatible with nightly) I know there are weird compile time issues that can come up. But lets give this a try and if we have stability issues we can revisit this. Getting ahead of the game to validate compatibility before the release will hopefully save us a mad rush after each next release.

[IvanIsCoding]

I will keep an eye on it but Rust beta should be more mature than nightly on Head. If it is not we will use stable for Python beta