How to pass former --install-option flags via --config-settings since pip 23.1?

[cacti77]

Description

I don't know if this is a bug or just a question....

Prior to pip 23.1 (e.g., 23.0.1) I was able to install my custom Python package with C++ extensions successfully on Windows using:

pip install --install-option=build_ext --install-option="--library-dirs=/path/to/library" *.zip

But the --install-option flag was removed in pip 23.1 via #11858.
So how do I now specify --install-option flags (such as --library-dirs) via --config-settings (or --config-setting) please?

I also asked this on SO here, but have got no answer yet.

Expected behavior

If this is possible with pip 23.1, please consider expanding on the docs for --config-settings at https://pip.pypa.io/en/stable/cli/pip_install/#cmdoption-C, or in the Examples section below.

pip version

23.1.2

Python version

3.7, 3.8

OS

Windows

How to Reproduce

1. Install pip 23.1 (e.g., in a Python 3.7 or 3.8 virtual environment created with venv.)
2. Try pip install of a source archive on Windows, attempting to pass in any former --install-option flag via --config-settings. E.g., --library-dirs or --libraries.

Output

Successful installation of a Python package from a source archive.

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[cacti77]

I should have said this doesn't work for example:
pip install --config-settings="--install-option=build_ext" --config-settings="--install-option=--library-dirs=/path/to/lib" *.zip

The Windows linker, link.exe, complains it can't find the library I'm trying to link to as there is no /LIBPATH entry for /path/to/lib, unlike when I used pip < 23.1.

[pfmoore]

You should probably ask in the setuptools tracker, as it's up to them how they map config settings to the legacy options.

[cacti77]

You should probably ask in the setuptools tracker, as it's up to them how they map config settings to the legacy options.

Thanks. I've added a comment at pypa/setuptools#3654.

However my CI/CD build pipeline broke when pip upgraded from 23.0.1 to 23.1; setuptools was 47.1.0 in both cases. I'm not a Python packaging expert so I don't know how the tools relate to each other really. From my point of view, things simply stopped working when pip upgraded to 23.1:
no such option: --install-option

[pfmoore]

That's probably because pip stopped falling back to running setup.py install. From the changelog:

When the wheel package is not installed, pip now uses the default build backend instead of setup.py install and setup.py develop for project without pyproject.toml. (#8559)

As a result, pip will be asking setuptools (via the PEP 517 interface) to build a wheel, and an install option doesn't make much sense when building a wheel. So you'll need to look at what you're trying to achieve, and work out how to achieve that when building a wheel.

[cacti77]

Ok, thanks, that makes more sense now. I didn't know it tried to build a wheel first, but I see that in the pip output now. So it sounds like I should be using something like:
pip install --config-settings="--build-option=blah"
rather than:
pip install --config-settings="--install-option=blah"
I haven't found the magic combination of options that works yet, but at least I think you've set me on the right path now, thanks. Yes, my project doesn't have a pyproject.toml at the moment.

[elupus]

Getting similar issues. Here. uv pip install still passes these through.

Edit.. Seems to be working again with 24. So sorry for the noise.

[44yu5h]

--config-settings="--build-option=/*blah*/" worked. Thanks!

[smemsh]

I can't for the life of me figure out the correct syntax. I can do a pip install, no wheel found, the build fails, then I can go into the tmp build dir and use setup.py build_ext --libraries=foo fix and the build works. However I cannot get the same --libraries to be passed via pip (24.2) to the build step no matter which variation I try:

--config-settings=libraries=foo
--config-settings --libraries=foo
--config-settings=--build-option=build_ext --config-settings=--build-option=--libraries=foo
--config-settings=--global-option=build_ext --config-settings=--global-option=--libraries=foo
--config-settings=--install-option=install --config-settings=--install-option=--libraries=foo
--config-settings --build-option=--libraries=
--config-settings --libraries=foo

None of these variations have had any effect on the linker line that gets run (I need -lfoo to appear at the right place, $LDFLAGS is too early but --libraries works right and $LIBS and $LDLIBS seems to be ignored) so I can only build this from the build root by directly invoking setup.py myself and adding --libraries, it's the only way for me to install this package.

It's confusing, there are many references to --config-settings in web searches and discussion about the deprecations, but few working examples of old (global-options, build-options, install-options) to new syntax conversion, just a lot of variations that people try. Passing args to build_ext would seem to be a simple, common case...

[pfmoore]

Passing args to build_ext would seem to be a simple, common case...

You should probably ask on the setuptools tracker. Pip does very little here except pass what you supply to setuptools. I think the problem is the fact that it's not obvious what config settings setuptools uses in place of the old --build-option (etc.) flags.

[smemsh]

Pip does very little here except pass what you supply to setuptools

How does Pip "pass" it though? In this case I know that a single --libraries argument to setup.py fixes the problem, but can't figure out how to pass arguments to setup.py via--config-settings.

[pfmoore]

If you say --config-settings a=b pip passes {‘a’: ‘b’} to the config_settings argument of the backend hooks.

It’s up to setuptools how they interpret that.