CI, BLD, TST: Re-enable Emscripten/Pyodide CI job for NumPy #25894
Conversation
`ninja` is unavailable for WASM for now, I have added a new requirements file for this and used a `sys_platform != 'emscripten'` platform marker to ignore the `ninja` dependency on WASM-based platforms.
This commit performs the following: 1. Skip `RuntimeWarnings` on exotic `np.where()` tests on WASM because of the lack of floating point exception support 2. Skip NumPy config tests that use subprocess module on WASM 3. Ignore threaded tests for PRNGs on WASM 4. Remove numpygh-5411 Cython `AttributeError` check. See cython/cython#5411, which is now resolved for Cython>3, and we are at Cython>=3.0.6. 4. For f2py, check compilers only if not on WASM 5. Skip pickle serialisation tests for `stringdtype` on WASM runtimes
agriyakhetarpal
force-pushed
the
enable-emscripten-ci
branch
from
891c3cc
to
f5f20df
Compare
|
Pyodide build is passing here as well: https://github.com/numpy/numpy/actions/runs/8066720365 |
agriyakhetarpal
changed the title
|
It looks like the build with nightly OpenBLAS wheels is failing – is there something I can do to help here? (cc: @mattip) |
|
RE:Blas Is this based on the current tip of NumPy main? |
Yes, this has been rebased on top of the |
|
The BLAS failures are happening on every PR and are unrelated. |
I just checked; I have been mostly conforming to the |
There was a problem hiding this comment.
Looks good to me for pyodide-build side. Thanks for working on this @agriyakhetarpal!
There was a problem hiding this comment.
This looks good, nice. A few more small comments/requests.
the floating-point exception support could be marked with
@pytest.mark.xfail(strict=True)
I'd prefer not to do that. There are already lots of other identical skips for both floating-point errors and subprocesses, so keeping these the same is both nicer and less churn.
|
I have resolved all the review comments that were posted; this should now be polished enough for another review, or for merging as and when needed. |
This commit performs the following actions: 1. Adds WASM builds to the CPU family for Meson configurations, but without SSE or SIMD instructions. 2. Enables `IEEE_QUAD_LE` longdouble format for the wasm32 target (cross-builds). 3. Enables run for Emscripten/Pyodide wheels by setting the `if:` condition to `true`. 4. Uses recursive submodules to ensure that vendored-meson is received. 5. Moves the Meson cross file to `tools/ci/emscripten/` (i.e., creates a separate Emscripten folder to store relevant files) 6. Adds a patch for vendored-meson detection for Pyodide and applies this Pyodide-meson patch in the Emscripten CI jobs 7. Builds wasm32 wheels without BLAS and LAPACK support (see numpy#24750 (comment)) 8. Forces coloured and prettified outputs for test runs Some of these changes have been copied with updates and suggestions received from numpy#24603 on 23/02/2024 and authorship is preserved with this commit. [skip cirrus] [skip circle] [skip azp] Co-Authored-By: Ralf Gommers <[email protected]> Co-authored-by: Hood Chatham <[email protected]>
rgommers
force-pushed
the
enable-emscripten-ci
branch
from
facec72
to
d2d2c25
Compare
There was a problem hiding this comment.
All looks good now. I did a slight cleanup of the commits that reverted something. In it goes. Nice work @agriyakhetarpal, and thanks for the reviews @hoodmane, @ryanking13 and @ngoldbaum.
Based on numpy/numpy#25894. This comment updates versions for Pyodide, Emscripten, and improves some names for job steps. Multiple reusable actions have also been updated in accordance with the latest versions available. Co-Authored-By: Lars Grüter <[email protected]>
Based on numpy/numpy#25894. This comment updates versions for Pyodide, Emscripten, and improves some names for job steps. Multiple reusable actions have also been updated in accordance with the latest versions available. Co-Authored-By: Lars Grüter <[email protected]>
Based on numpy/numpy#25894. This comment updates versions for Pyodide, Emscripten, and improves some names for job steps. Multiple reusable actions have also been updated in accordance with the latest versions available. Co-Authored-By: Lars Grüter <[email protected]>
Based on numpy/numpy#25894. This comment updates versions for Pyodide, Emscripten, and improves some names for job steps. Multiple reusable actions have also been updated in accordance with the latest versions available. Co-Authored-By: Lars Grüter <[email protected]>
Based on numpy/numpy#25894. This comment updates versions for Pyodide, Emscripten, and improves some names for job steps. Multiple reusable actions have also been updated in accordance with the latest versions available. Co-Authored-By: Lars Grüter <[email protected]>
Based on numpy/numpy#25894. This comment updates versions for Pyodide, Emscripten, and improves some names for job steps. Multiple reusable actions have also been updated in accordance with the latest versions available. Co-Authored-By: Lars Grüter <[email protected]>
Description
This PR supersedes gh-24603. It adds a CI job to test NumPy v2.0.0.dev0 against a Pyodide (wasm32) runtime. Some of the key changes here are:
ci/tools/emscripten/directory based on an upstream change to Pyodide at Update numpy to 1.26.4 and don't set MESON env variable pyodide/pyodide#4502. This patch ensures that the correct Meson build system (i.e.,vendored-meson) is found during the build process.f2pytests, since Fortran cannot run in WASMnp.where()that require floating point exception supportCythonCImportsshadowing functionality can confuse pickle cython/cython#5411, has now been resolved. Theimport cythonline was breaking the test discovery because Cython is not supported in Pyodide in-tree yet.Thanks to @rgommers and the notes provided on gh-24603, all tests pass! Here's a workflow run from my fork where they can be observed: https://github.com/agriyakhetarpal/numpy/actions/runs/8065629246