Improve `build.util.project_wheel_metadata` #553

atugushev · 2022-12-13T23:50:01Z

What's the problem this feature will solve?

Currently, pip-tools uses project_wheel_metadata() to parse setup files and get install requirements of a package. The issue is that the function suppresses any errors if a setup file (or pyproject.toml) is invalid. See the example below.

Example

Note
There is no ZeroDivisionError in the output.

$ python --version
Python 3.11.1

$ python -m build --version
build 0.9.0 (/usr/local/lib/python3.11/site-packages/build)

$ mkdir /tmp/foo

$ echo '1/0' > /tmp/foo/setup.py

$ python -c 'from build.util import project_wheel_metadata; project_wheel_metadata("/tmp/foo")'
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/build/__init__.py", line 494, in _handle_backend
    yield
  File "/usr/local/lib/python3.11/site-packages/build/__init__.py", line 381, in get_requires_for_build
    return set(get_requires(config_settings))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pep517/wrappers.py", line 173, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pep517/wrappers.py", line 309, in _call_hook
    self._subprocess_runner(
  File "/usr/local/lib/python3.11/site-packages/build/__init__.py", line 327, in _runner
    self._hook_runner(cmd, cwd, extra_environ)
  File "/usr/local/lib/python3.11/site-packages/pep517/wrappers.py", line 76, in quiet_subprocess_runner
    check_output(cmd, cwd=cwd, env=env, stderr=STDOUT)
  File "/usr/local/lib/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/tmp/build-env-v0bbw1mc/bin/python', '/usr/local/lib/python3.11/site-packages/pep517/in_process/_in_process.py', 'get_requires_for_build_wheel', '/tmp/tmppk54orfx']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.11/site-packages/build/util.py", line 54, in project_wheel_metadata
    env.install(builder.get_requires_for_build('wheel'))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/build/__init__.py", line 380, in get_requires_for_build
    with self._handle_backend(hook_name):
  File "/usr/local/lib/python3.11/contextlib.py", line 155, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.11/site-packages/build/__init__.py", line 502, in _handle_backend
    raise BuildBackendException(  # noqa: B904 # use raise from
build.BuildBackendException: Backend subprocess exited when trying to invoke get_requires_for_build_wheel

That's because project_wheel_metadata uses pep517.quiet_subprocess_runner under the hood which suppresses the output.

Describe the solution you'd like

It would be nice to have a parameter to control which runner will be used. A few options:

project_wheel_metadata(srcdir, isolated, quiet=false) - will be used pep517.default_subprocess_runner
project_wheel_metadata(srcdir, isolated, runner=pep517default_subprocess_runner - pass a runner directly

Example

Note
There is ZeroDivisionError in the output.

$ python -c 'from build.util import project_wheel_metadata; project_wheel_metadata("/tmp/foo")'
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/pep517/in_process/_in_process.py", line 351, in <module>
    main()
  File "/usr/local/lib/python3.11/site-packages/pep517/in_process/_in_process.py", line 333, in main
    json_out['return_val'] = hook(**hook_input['kwargs'])
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pep517/in_process/_in_process.py", line 118, in get_requires_for_build_wheel
    return hook(config_settings)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/build-env-rdlyy__u/lib/python3.11/site-packages/setuptools/build_meta.py", line 338, in get_requires_for_build_wheel
    return self._get_build_requires(config_settings, requirements=['wheel'])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/build-env-rdlyy__u/lib/python3.11/site-packages/setuptools/build_meta.py", line 320, in _get_build_requires
    self.run_setup()
  File "/tmp/build-env-rdlyy__u/lib/python3.11/site-packages/setuptools/build_meta.py", line 485, in run_setup
    self).run_setup(setup_script=setup_script)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/build-env-rdlyy__u/lib/python3.11/site-packages/setuptools/build_meta.py", line 335, in run_setup
    exec(code, locals())
  File "<string>", line 1, in <module>
ZeroDivisionError: division by zero

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/build/__init__.py", line 494, in _handle_backend
    yield
  File "/usr/local/lib/python3.11/site-packages/build/__init__.py", line 381, in get_requires_for_build
    return set(get_requires(config_settings))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pep517/wrappers.py", line 173, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pep517/wrappers.py", line 309, in _call_hook
    self._subprocess_runner(
  File "/usr/local/lib/python3.11/site-packages/build/__init__.py", line 327, in _runner
    self._hook_runner(cmd, cwd, extra_environ)
  File "/usr/local/lib/python3.11/site-packages/pep517/wrappers.py", line 67, in default_subprocess_runner
    check_call(cmd, cwd=cwd, env=env)
  File "/usr/local/lib/python3.11/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/build-env-rdlyy__u/bin/python', '/usr/local/lib/python3.11/site-packages/pep517/in_process/_in_process.py', 'get_requires_for_build_wheel', '/tmp/tmprcu10f6m']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.11/site-packages/build/util.py", line 55, in project_wheel_metadata
    env.install(builder.get_requires_for_build('wheel'))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/build/__init__.py", line 380, in get_requires_for_build
    with self._handle_backend(hook_name):
  File "/usr/local/lib/python3.11/contextlib.py", line 155, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.11/site-packages/build/__init__.py", line 502, in _handle_backend
    raise BuildBackendException(  # noqa: B904 # use raise from
build.BuildBackendException: Backend subprocess exited when trying to invoke get_requires_for_build_wheel

Additional context

Mysterious error with optional dependencies in pyproject.toml jazzband/pip-tools#1711

The text was updated successfully, but these errors were encountered:

) Co-authored-by: Henry Schreiner <[email protected]> Co-authored-by: layday <[email protected]> Closes #553

While `tox` doesn't need to be in the venv, it DOES currently need to be less than version 4. `build` is useful for debugging pip errors and pip-compile trouble. Whenever pip-compile (via `tox -e upgrade-deps` for instance) fails with Backend subprocess exited when trying to invoke get_requires_for_build_wheel Failed to parse /PATH/pyproject.toml run `python -m build -w` to se what `build` is actually complaining about. See also pypa/build#553

While `tox` doesn't need to be in the venv, it DOES currently need to be less than version 4. `build` is useful for debugging pip errors and pip-compile trouble. Whenever pip-compile (via `tox -e upgrade-deps` for instance) fails with Backend subprocess exited when trying to invoke get_requires_for_build_wheel Failed to parse /PATH/pyproject.toml run `python -m build -w` to see what `build` is actually complaining about. See also pypa/build#553

atugushev mentioned this issue Dec 13, 2022

Mysterious error with optional dependencies in pyproject.toml jazzband/pip-tools#1711

Closed

q0w mentioned this issue Jan 22, 2023

util: allow passing alternative runner to project_wheel_metadata #566

Merged

layday closed this as completed in #566 Jan 25, 2023

layday pushed a commit that referenced this issue Jan 25, 2023

util: allow passing alternative runner to project_wheel_metadata (#566

e18ce1c

) Co-authored-by: Henry Schreiner <[email protected]> Co-authored-by: layday <[email protected]> Closes #553

nikhilweee mentioned this issue May 4, 2023

Log errors for incorrect pyproject.toml jazzband/pip-tools#1860

Closed

atugushev mentioned this issue Aug 9, 2023

Add options for including build dependencies in compiled output jazzband/pip-tools#1681

Merged

4 tasks

hmpf mentioned this issue Feb 8, 2024

Add two development dependencies Uninett/Argus#719

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve `build.util.project_wheel_metadata` #553

Improve `build.util.project_wheel_metadata` #553

atugushev commented Dec 13, 2022 •

edited

Loading

Improve build.util.project_wheel_metadata #553

Improve build.util.project_wheel_metadata #553

Comments

atugushev commented Dec 13, 2022 • edited Loading

What's the problem this feature will solve?

Describe the solution you'd like

Additional context

Improve `build.util.project_wheel_metadata` #553

Improve `build.util.project_wheel_metadata` #553

atugushev commented Dec 13, 2022 •

edited

Loading