Explain oldest-supported-numpy

[BastianZim]

Where should the content be added?

FAQ

What should be added?

NumPy introduced oldest-supported-numpy: https://github.com/scipy/oldest-supported-numpy

This can be used by packages instead of specifying a NumPy range when building.

The package doesn't exist in conda-forge and instead numpy should be used without any versions specified.

Additional information

https://gitter.im/conda-forge/conda-forge.github.io?at=62a8fc44deea5616bbf9b99c

[ngam]

oldest-supported-numpy shows up in pyproject.toml too; in that case, either a patch is needed or alternatively deleting --no-deps from the python install call so that pip can install/find numpy. The patch is safer

[ngam]

Maybe we should consider making an alias package, name it oldest-supported-numpy and set it equal exactly to numpy

[chrisburr]

Maybe a linter rule in conda-smithy would be better if oldest-supported-numpy is found in the recipe?

[ngam]

Maybe a linter rule in conda-smithy would be better if oldest-supported-numpy is found in the recipe?

Yes, definitely. The alias package idea is really a bad hack. It's just that we already have whatever oldest-supported-numpy is supposed to do implemented seamlessly in conda-forge, so we could simply just replace occurrences of oldest-supported-numpy with numpy (e.g. in a patch) and all should be good. If that could be done with a linter, even better!

[h-vetinari]

I actually think the wrapper (or an empty package that just tells pip oldest-supported-numpy is installed) could be an interesting idea. I'm patching pyproject.toml files on a couple of feedstocks, and it's painful. If a wrapper package would allow avoiding that, it would be very nice!

[hmaarrfk]

- rm pyproject.toml
- pip install . -vv --no-build-isolation --no-deps

why not adapt your build script to completely remove pyproject.toml

[h-vetinari]

why not adapt your build script to completely remove pyproject.toml

I've started doing that, but it's still a bandaid IMO. More and more functionality ecosystem-wide is moving into pyproject.toml (e.g. scipy starting to require meson to build), so it would be nice to be able to install packages out-of-the box (as in the staged-recipes example-package) just by providing the right dependencies.

[chrisburr]

more functionality ecosystem-wide is moving into pyproject.toml

Longer term I think we should start using pyproject.toml to improve the update of recipes (likely using grayskull + some conservative logic to compare to the previous recipe). In this case grayskull can be adapted to convert oldest-supported-numpy to something suitable for conda-forge.

[BastianZim]

For now, we can just add it to the matching as a new quirk: https://github.com/conda-incubator/grayskull/blob/main/grayskull/strategy/config.yaml

[eli-schwartz]

why not adapt your build script to completely remove pyproject.toml

Because that would be like removing setup.cfg / setup.py when they specify oldest-supported-numpy in setup_requires in a pre-PEP 517/518 package. Removing huge swathes of the build system because one tiny part of it includes a version pin will just break everything because, well, huge swathes of the build system are there.

pip cannot install anything unless it either:

• uses setuptools
• has a pyproject.toml for better or worse

Depending on the build backend, it may be possible to run that directly instead. e.g. flit build --format wheel / poetry build --format wheel / python setup.py bdist_wheel. This is a problem for things like Meson (SciPy) where the build backend doesn't have a frontend CLI (and meson install does not handle dist-info yet).

[ngam]

fyi, xref: conda-forge/staged-recipes#16514 & conda-forge/staged-recipes#18982

[hmaarrfk]

Removing huge swathes of the build system

I think this is somewhat overblown. Much of the dependencies are rewritten in the meta file. Pyproject.toml may help in a pypi context, but in a conda forge centric context, it somewhat duplicates alot of the effort of the CIs and docker.

Maybe I'm not so versed in the more recent additions to pyproject.toml, but i recall it was made to help declare build time dependencies, as such, this is duplicated in the meta file, and can likely safely be removed, or replaced by a shim for conda-forge purposes.

[eli-schwartz]

Maybe I'm not so versed in the more recent additions to pyproject.toml, but i recall it was made to help declare build time dependencies, as such, this is duplicated in the meta file

There's a new PEP that says that the meta file is now pyproject.toml, in the [project] section.

There's (experimental?) support for "no setup.cfg, just pyproject.toml" in setuptools. Flit and poetry never use anything else. Meson-python (used by e.g. SciPy) never uses anything else. Other build backends will follow if they haven't already.

So yes, that file is now a hard requirement a lot of the time.

EDIT: see https://peps.python.org/pep-0621/

[ngam]

So yes, that file is now a hard requirement a lot of the time.

That's my understanding too. Well, at least in the "recommended" setup (i.e. using setup.cfg AND pyproject.toml instead of setup.py). I am rather inexperienced in this area, so I have no idea if "recommended" is legit or I just happen to remember it this way.

but i recall it was made to help declare build time dependencies, as such, this is duplicated in the meta file, and can likely safely be removed, or replaced by a shim for conda-forge purposes.

I think you're correct about the build time thing, but the larger problem is that newer projects may simply fail if pyproject.toml is deleted (not that it is actually of any use in conda-forge). For example, if a package uses setup.cfg, then it must also include pyproject.toml; so if we delete pyproject.toml and run {{ PYTHON }} -m pip install . -vv, we get this error:

ERROR: Directory '.' is not installable. Neither 'setup.py' nor 'pyproject.toml' found.

(example implementation: conda-forge/particula-feedstock#13)

[BastianZim]

Just to add: I have seen some projects now that only have a pyproject.toml so removing is not an option anymore.

[hmaarrfk]

Ok. It seems to have evolved to be more than just build system declaration.

Does pip have a flag to ignore the build system stuff? Does the --no-build-isolation flag do what you need?

[ngam]

Does the --no-build-isolation flag do what you need?

No, but even if it did, the problem is now more complicated anyway, I just wanted to give you a minimal live example of it failing.

[ngam]

Here is one with only pyproject.toml: conda-forge/staged-recipes#19364 (btw, pls merge it if you can, thanks!)

[hmaarrfk]

I'm sorry, that project doesn't actually have any required dependencies nor does it use oldest-supported-numpy.