it's almost always a mistake to build arm64/x86_64 wheels on macOS; warn about it?

[glyph]

Description

As shown in hhas/appscript#5, sometimes maintainers who are trying to update to the Apple Silicon transition will want to build wheels for arm64. The obvious way to do that is to add 'arm64' to their CIBW_ARCHS_MACOS and start testing in an arm64 environment, as one would on (e.g.) any Linux distro.

This is almost always a configuration error, and building universal2 wheels is the right thing to do. Of course it's possible that in some expert cases one might literally want separate wheels, but distributing architecture-specific wheels makes it a lot tougher for users who want to just use a library in an app built for most macs.

Build log

No response

CI config

No response

[Jackenmen]

Some past discussion here: #1204 (including a related discussion on numpy repository)

[henryiii]

This is not clear-cut. The best/simplest thing really is to build separate wheels, then combine them later. Except there's not a good tool for combining them right now and it's pretty manual - multi build had scripts to do it. And you might ideally want to build the ARM on ARM and Intel on Intel, so you'd not even be in a cibuildwheel run when you want to combine them.

Universal are harder to build, since you have to have all universal dependencies and libraries. You can't just brew install libs for a universal build. They also ideally need to be re-tagged, as only pip from the last year or so can install them unless you add the x86 / arm tags too. For simple builds, this is probably the way to go (with the retagging), but it's by no means the "only" way to go.

I've been trying to get a way to add tags into wheel, but wheel has been in flux and not ready for it.

[joerick]

Yes, I agree. If you have a specific need for universal2 wheels and a project doesn't make them available, please look into delocate-fuse.

[glyph]

I was definitely mistaken about the level of subtlety here, but I do think that in a lot of cases, universal2 "just works" and in those cases it would be best to recommend it strongly, with a clear fallback for the cases where it's more complex.

In the cases where it is possible to build a universal2 wheel, there are fewer failure modes, more likelihood that the versions of installed libraries are matched, and therefore fewer moving parts and ways in which things can go wrong and perform inconsistently.

You can't just brew install libs for a universal build.

Yes, but you also really shouldn't be brew installing stuff for distributing in wheels. It's an attractive nuisance unless you really know what you're doing. From one of the homebrew maintainers on this topic:

This use case (bundling bottles built by Homebrew maintainers and redistributing them in a downstream app) is not supported by Homebrew. Doing things such as cross-compiling for M1 Macs or creating universal binaries is also not supported.

Homebrew binaries are for distributing within Homebrew itself, not for redistributing as apps or as other binary artifacts such as wheels. I know how convenient it can be to use them this way (and I've certainly availed myself of this somewhat risky convenience in the past) but very basic features for redistribution are missing, like e.g. pinning versions of the lib within homebrew so as to ensure a consistent build process; homebrew wants to update itself very aggressively, and generally can't be downgraded again if something breaks. There's something to be said for that as a philosophy for tooling (and I'm generally in favor) but not as a build environment for redistributables.

only pip from the last year or so can install them unless you add the x86 / arm tags too

Is there really any reason to be using a pip that old on macOS? It's not like a linux distro where you end up nailed to the floorboards because of system-level dependencies on a particular tool version; you can pip install --upgrade pip in a virtualenv and modulo issues with the new dependency resolver, there's not that much that can go wrong with a pip upgrade, is there?

I don't want to make people's lives difficult but in the absence of any specific difficulty I think it's good not to normalize living on years-out-of-date software, particularly for security-critical portions of the stack like what pip represents.

[EwoutH]

There are a lot of wheels downloaded each day. universal2 wheels are often (almost) double the size, making them unnecessarily large. This increases bandwidth demands on both sides of the link. This is also a temporarily problem, since in a few years most users will be on Arm64 anyways.

So I would be against changing the default or throwing a warning, but improved documentation stating the advantages and disadvantages of both options could be useful.

[joerick]

Thanks for Homebrew maintainer quote, it's good to have something definitive on that topic.

Is there really any reason to be using a pip that old on macOS?

I don't know, perhaps there are a couple causes (e.g. the bundled pip when doing python -m venv comes to mind), but the end result is frustration for users (the error messages for this aren't helpful) and raised issues for packaging maintainers, many of whom also don't know why the universal2 wheel isn't found.

(From the stats spreadsheet)

But, I do agree that because the solution pip install -U pip is so simple, it's not insurmountable.

The packager burden is the main reason I think native wheels are the default. For some libraries, that compilation is made harder by the added cross-compilation step. When it works, it's easy, but when it doesn't it's a lot harder.

Universal2 requires each dependency of libraries to also be universal2, adding more work to packagers. In particular, basically no scientific users of Python use universal2, because they all depend on numpy and they don't distribute a universal2 wheel.

I do think that in a lot of cases, universal2 "just works" and in those cases it would be best to recommend it strongly, with a clear fallback for the cases where it's more complex.

As you say, it's not clear-cut. But ultimately, the loudest cry for universal2 comes from people that want to distribute macOS apps, and delocate-fuse exists for that use-case now.

I will also say that since we've already discussed this in #1204 and released arm64 support with this feature, we unlikely to change it unless there's a very strong community push for universal2.

[joerick]

improved documentation stating the advantages and disadvantages of both options could be useful

We already have some info in this section of the docs - happy to look at PRs improving / adding more nuance here.

[henryiii]

You really should never use pip install -U pip to replace your system's pip; that is officially not supported, and in the past has caused breakages (upgrading off of pip 9 to pip 10 broke the system pip command, for example). It's perfectly safe inside a virtual environment, but it is not supported to overwrite the files provided by you system package manager. It mostly might work, but it is not supported by pip or the package managers.

You can also install this in your user directory, which is a little ugly IMO, but a pretty reasonable middle ground especially for something like pip - and newer versions of pip automatically switch to the user directory if they can't write to the system packages (not sure about upgrades, but I think it applies there too). Doesn't help if you are upgrading from an older pip, though.

[Czaki]

You can't just brew install libs for a universal build

You never should do this. Otherwise, platform tag will be broken.

[rgommers]

The packager burden is the main reason I think native wheels are the default.

That is a pain, but I'd say disk and download space are reason number 1. The way universal2 was introduced was basically "this is easiest to implement, so let's do that" - pushing the burden on folks with slow networks, onto PyPI, and onto the disk space of every user. All for little good reason. The py2app case is extremely niche compared to the people who just want a working Python environment.

By design, universal2 is a weird idea, at least as a general wheel build and shipping on PyPI option. The few people who need this can grab regular wheels from PyPI and do:

pip install delocate
delocate-fuse $amd64_wheel $arm64_wheel -w .

There is now also freely available M1 CI (Cirrus CI), so folks will more and more build x86-64 wheels on GitHub Actions and M1 wheels on Cirrus CI. Fusing should be done in py2app or by the devs who are building dual-arch dmg installers for nontechnical users.

It looks to me like the status quo is fine, and this issue can be closed.

[mattip]

xref the NumPy discussion of this topic, which came down on the side of separate wheels: numpy/numpy#18143