Add wheel support for aarch64

[odidev]

Added wheel support for aarch64.
Related to #73. @hynek Could you please review the changes.

[hynek]

hey @asottile you think this is the best/most viable way forward for now?

[asottile]

as far as I understand, the arm manylinux discussion stalled: pypa/manylinux#84

so even if there are images for it, there's no spec for it outside x86 / x64

so while this'll produce arm wheels, and they may be uploadable to pypi, they may be unusable on actual arm platforms due to the lack of an actual specced set of supported symbols / processor features

(then again, my reading of the linked issue is cursory and admittedly I didn't take time to read into the finer points, and I don't know the first thing about alternative arm architectures beyond what aws provides)

[odidev]

as far as I understand, the arm manylinux discussion stalled: pypa/manylinux#84

@asottile this PR adds only aarch64[ARMv8] support and pip version>=19.3 started supporting aarch64 and it is able to download ARMv8 architecture specific wheels. Please have a look at below logs-

root@5b2602200e8b:/# pip3 --version
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)
root@5b2602200e8b:/# pip3 install scipy
Collecting scipy
Downloading scipy-1.5.4-cp38-cp38-manylinux2014_aarch64.whl (24.5 MB)
|████████████████████████████████| 24.5 MB 1.7 MB/s
Collecting numpy>=1.14.5
Downloading numpy-1.19.4-cp38-cp38-manylinux2014_aarch64.whl (12.2 MB)
|████████████████████████████████| 12.2 MB 1.7 MB/s
Installing collected packages: numpy, scipy
Successfully installed numpy-1.19.4 scipy-1.5.4
root@5b2602200e8b:/#

[asottile]

@odidev please read the lines directly after that in my message

[odidev]

Hi @asottile, I went through your comment and I understood that you are concerned that argon2-cffi might not be supported for aarch64. Please correct me if I am wrong. If I am correct, please suggest if there is a method by which we can test if this package is supported on aarch64. Thanks in advance.

[asottile]

ok well I don't think you're understanding the nuances of another architecture

even if a package builds correctly on one processor of aarch64, it does not mean that it will successfully run on other aarch64 processors or even a majority of aarch64 processors

for x86 / x64, a lot of work has gone into standardizing what symbols and processor features are allowed to be used in a binary artifact -- no such standardization has been done for other architectures in manylinux. as such, while a wheel can be built that carries an installable tag, there is no guarantee that it will actually function on all or a majority of processors carrying that tag. for example, the extension may take advantage of processor extensions that are only available in a particular concrete processor that's being targetted that won't be available elsewhere -- some of these may be compiled in via the compiler itself

so while it may build and may run on a particular aarch64, uploading a manylinux wheel for it may not be the best idea because manylinux for non x86/x64 is currently the wild wild west in terms of certified compatibility

[rhenwood-arm]

Hi @asottile (I work with @odidev)

I think I understand your concern: If we build for one version of the AArch64 specification, how can we be confident it will run on the others?

Manylinux2014 specifies the environment that must be present to build a native wheel. It includes the OS, and the version of GCC and Glibc: CentOS7, GCC 4.8.(0,5), GlibC (2.17). There is a more detailed discussion of these requirements on this thread:
dockcross/dockcross#443

By specifying CentOS7, Manylinux2014 effectively ensures that the wheel is built against the AArch64 v8.0 specification. All AArch64 platforms in the v8 series (and Glibc) maintain backward compatibility with v8.0 so a binary built for v8.0 can run on any v8 platform.

Does that help?

[asottile]

I understand all of that, and if you read the thread linked above there has been no standardization of the allowable symbols or instructions on architectures outside of the x86/x64 architectures. Additionally if you read, there's additional concerns about using an operating system distribution, libc, and compiler which were released before aarch64 processors were practically available. And lastly, no one has stepped up (in fact there's been people who have stepped down!) to try and standardize and validate aarch64 as a manylinux-viable target architecture. A lot of work went into that and vetting particular compiler features, available symbols, instructions, and target processors etc. for x86/x64 and none of that work has even been scoped or started for alternate architectures in the python community.

I feel like a broken record here, please review the previous messages I've sent and please read the manylinux thread linked above in its entirety. I'm going to unsubscribe from this thread because I feel like you're not listening to my concerns or doing the research necessary. If you need me please @-mention me.

[hynek]

OK to be clear, we won't ship wheels, until pypa/manylinux#84 is fixed.

Shipping something that is not fully specced/standardized would just mean more bug reports that are undebugable and inscrutable.

I will leave this open because I absolutely hope that we can ship them eventually. But until there's no standard, I cannot justify shipping something on PyPI. Please add new comments only if you have something new to contribute.

[webknjaz]

OK to be clear, we won't ship wheels, until pypa/manylinux#84 is fixed.

FYI PyPA now ships manylinux wheels for 2014 and 2_24 tags that have AARCH64. pyca/cryptography even uses those. I don't think other problems raised in that issue should be waited for.

[hynek]

Are there any best practices/docs somewhere to be found?

[rhenwood-arm]

@hynek : Depending on what you're looking for: I have accumulated methods to build aarch64 wheels here: https://github.com/ARM-software/developer/blob/master/projects/python-wheels/multi-platform.md

Circle-CI now have a preview AArch64 builder: https://github.com/CircleCI-Public/arm-preview-docs/blob/main/README.md - this may be the path of least resistance?

[webknjaz]

Are there any best practices/docs somewhere to be found?

I'm not aware of any but I really like how PyCA does this. I followed their example and created a bunch of base containers with the build deps that I then use for building wheels (example: https://github.com/ansible/pylibssh/actions/runs/625397435/workflow). Those containers are published to GitHub Container Registry. For non-x86 envs, it's quite easy to set up QEMU with something like sudo podman run --rm --privileged multiarch/qemu-user-static --reset -p yes. And then you just need to point to images that interest you. You may need to adjust any OS-specific commands you have because manylinux2014 is based on CentOS but manylinux_2_24 is built on Debian 9.

[webknjaz]

I strongly recommend building the base images separately because it takes an hour under QEMU, this will crucially improve the wheel build time. If you'll only keep python -m build . && auditwheel repair ... bits in the builders, the builds will probably fit in 10 min.

[hynek]

AARCH64 support for Linux will be provided by https://github.com/hynek/argon2-cffi-bindings that this project will depend on from the next release on.

Please check https://pypi.org/project/argon2-cffi-bindings/#files if those wheels are installable for you. If you feel generous, running the current main branch on your target arch would work too.

[seifertm]

@hynek aarch64 support sounds great, thanks for your continuous effort!

Out of curiosity: Could you explain the reasoning behind splitting the package up into argon2-cffi and argon2-cffi-bindings?

[hynek]

The bindings rarely change, but are a PITA to package and release. I hope it also encourages more contributions to argon2-Cffi and removes some resistance for me to publish new releases.

finally it might be useful to others trying to play with argon2. There’s a been at least two packages but they’ve stalled and I suspect it’s because of compiler shenanigans.

---

Unrelatedly, universal2 wheels are almost done!

[webknjaz]

@hynek why did you keep the leading underscore? One would expect it to be public API since it's a top-level package. It seems counterintuitive to have it in a separate package for people to consume while the package name says the opposite.

[hynek]

As I write in https://github.com/hynek/argon2-cffi-bindings#python-api, I consider it an implementation detail both for me and potential other users. I don't want user accidentally poke around in a module argon2_cffi_bindings because tab completion took them there or whatever.

[webknjaz]

Ah, I missed that. Thanks for the explanation. I thought maybe it was a mistake :)

[hynek]

Last call for everyone running pip install -U pip; pip install argon2-cffi-bindings and reporting back for platforms other than macOS/amd64 whether or not y'all are getting binary wheels!

[seifertm]

Last call for everyone running pip install -U pip; pip install argon2-cffi-bindings and reporting back for platforms other than macOS/amd64 whether or not y'all are getting binary wheels!

This command installs argon2_cffi_bindings-21.2.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl for Python 3.9 on Linux (x86_64) and the installed module is importable:

$ pip install -U pip; pip install argon2-cffi-bindings
…
Successfully installed pip-21.3.1
…
Collecting argon2-cffi-bindings
  Downloading argon2_cffi_bindings-21.2.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (86 kB)
…
Successfully installed argon2-cffi-bindings-21.2.0 cffi-1.15.0 pycparser-2.21
$ python
>>> from _argon2_cffi_bindings import ffi, lib
>>>

[mgmarino]

Installing argon2-cffi from master here in a docker container (python:3.7.9-slim), being built with target platform of linux/arm64 grabs the binary wheels of argon2-cffi-bindings, i.e.

Here are the excerpts, let me know if more info is needed.

#37 56.65 Collecting argon2-cffi
#37 56.65   Cloning https://github.com/hynek/argon2-cffi.git (to revision 024e35889f875a739bf36883ebcf5b283eb83950) to /tmp/pip-install-jnsesn_x/argon2-cffi_cac067ea93b24e86934020704fff986c
#37 56.71   Running command git clone --filter=blob:none -q https://github.com/hynek/argon2-cffi.git /tmp/pip-install-jnsesn_x/argon2-cffi_cac067ea93b24e86934020704fff986c
#37 59.16   Running command git rev-parse -q --verify 'sha^024e35889f875a739bf36883ebcf5b283eb83950'
#37 59.23   Running command git fetch -q https://github.com/hynek/argon2-cffi.git 024e35889f875a739bf36883ebcf5b283eb83950
#37 60.56   Resolved https://github.com/hynek/argon2-cffi.git to commit 024e35889f875a739bf36883ebcf5b283eb83950
#37 60.56   Running command git submodule update --init --recursive -q
#37 62.97   Installing build dependencies: started
#37 76.70   Installing build dependencies: finished with status 'done'
#37 76.70   Getting requirements to build wheel: started
#37 77.94   Getting requirements to build wheel: finished with status 'done'
#37 77.95   Preparing metadata (pyproject.toml): started
#37 79.00   Preparing metadata (pyproject.toml): finished with status 'done'
...
#37 367.2 Collecting argon2-cffi-bindings
#37 367.3   Downloading argon2_cffi_bindings-21.1.0-cp36-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (79 kB)
...
#37 379.1   Building wheel for argon2-cffi (pyproject.toml): started
#37 380.1   Building wheel for argon2-cffi (pyproject.toml): finished with status 'done'
#37 380.1   Created wheel for argon2-cffi: filename=argon2_cffi-21.2.0.dev0-py3-none-any.whl size=12902 sha256=45e3767eff1e03fa8b61e4ebdba3f5d4bd384356e7379ccd66ae5ba088c5a113
#37 380.1   Stored in directory: /root/.cache/pip/wheels/71/b5/6d/f343d5a7b1091a1c5ecce331173db2223031a2a2476e7e5c5b
...
#37 608.6 Successfully installed ... argon2-cffi-21.2.0.dev0 argon2-cffi-bindings-21.1.0 ...

[hynek]

Alright, hang on, argon2-cffi 21.2.0 will hit PyPI sometime next week.