New --no-build-isolation check in Pip 22.1 broke oldest-supported-numpy

[rgommers]

Description

The change in gh-10886 to check for presence of build dependencies when building with --no-build-isolation broke users of oldest-supported-numpy (see scipy/oldest-supported-numpy#53).

For context, oldest-supported-numpy is a meta-package which ensures that the correct numpy version is installed at build time, handling platform and Python-version specific details so those don't have to be replicated in each package which depends on NumPy's C API. Also important: it is a PyPI-specific package; other packaging systems have different ways of dealing with NumPy ABI issues and do not necessarily use the same NumPy versions to build against as the NumPy team recommends as the default for wheels.

A use case like this does not seem to have been considered at all in gh-10886 or the issue for it (gh-9794). It doesn't make sense to have a metapackage like this installed in many cases, nor is it desired to force users to build with exactly one numpy version - in CI for example, it makes perfect sense to test with multiple versions; the only requirement is that the version at runtime is >= the version used at build time.

I don't see a good way to fix this up in either oldest-supported-numpy or in the pyproject.toml files from users of that package, and I think that therefore the change in Pip's behavior should be reverted.

Expected behavior

I expect the reproducer to result in a successful build of scikit-learn. Users must be able to specify oldest-supported-numpy in their build dependencies without the new check being triggered.

This is probably not the only use case that broke, but in case you really want to keep this check: an alternative could be to special-case oldest-supported-numpy inside the code performing the check; instead just check that a version of numpy is installed.

pip version

22.1

Python version

all

OS

all

How to Reproduce

Example for one of a number of packages that are affected:

pip install cython scipy 'setuptools<60'
git clone https://github.com/scikit-learn/scikit-learn
cd scikit-learn
pip install --no-build-isolation .

Output

Processing /Users/rgommers/code/tmp/scikit-learn
ERROR: Some build dependencies for file:///Users/rgommers/code/tmp/scikit-learn are missing: 'oldest-supported-numpy'.

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[rgommers]

Cc @q0w, @pradyunsg as author and reviewer of gh-10886

[q0w]

Maybe add new flag to validate build deps?

[rgommers]

Maybe add new flag to validate build deps?

It sounds reasonable to do this as opt-in to me. Note that pypa/build has it as opt-out (--skip-dependency-check) and I've found a need to use that on multiple occasions as well. These checks are only applicable in a subset of circumstances where you'd want to use a tool like Pip or build, so opt-in seems much preferred to me.

Side note (I'll expand on that elsewhere): there's a bigger conceptual issue here, which I only noticed once I started to use pyproject.toml on large projects with complex dependencies. That is that you really need two sets of hooks:

1. Unpinned build dependencies for development, building wheels locally, conda packages, rpm's, etc.
2. Pinned dependencies specifically for the sdist that you upload to PyPI as part of a release.

These two are very different, and there's no place to put two sets. Right now checks like the one under discussion here simply assume that (1) and (2) are the same.

[uranusjr]

Hmm, I guess this applies to all metapackage approaches in general. We should probably make an opt-in or out mechanism (I’m leaning toward an opt-in personally).

Also more broadly, perhaps we should propose some sort of mechnism in packaging metadata to handle metapackages. This is sort of the opposite problem of detecting a specified extra—an extra is a requirement that is specified but can’t be detected, while a metapackage is something that can be specified but should not be considered at runtime. It may be possible to abstract both ideas into a special “virtual package” concept (I think this is Debian’s terminology?) or something.

[pradyunsg]

Let’s change this to an opt-in. Looking back at this, I think that’s what we should have done when we implemented this.

[uranusjr]

I think we should not validate by default, and add a flag to enable validation.

[tacaswell]

These two are very different, and there's no place to put two sets. Right now checks like the one under discussion here simply assume that (1) and (2) are the same.

I very much agree with this!

[henryiii]

FYI (from another thread), build has had this from the beginning (or at least a long time), and uses:

  --skip-dependency-check, -x
                        do not check that build dependencies are installed
  --no-isolation, -n    do not isolate the build in a virtual environment

So pipx run build -nx is how you disable isolation and skip the dependency check. Other use cases include cmake and ninja on Conda-forge (the packages there don't install the PyPI packages), and in something like Pyodide where you are locked to a single local version and package authors pin to old versions (including things like numpy - old pins are not requires if you build the whole stack).

[jakirkham]

Yeah for validation in some of these cases, we are not terribly concerned with validation at build time, but do care at install time. Typically we are handling this by running things like pip check once things are in place (so after pip install and possibly other steps like bundling artifacts).