Python.h No such file or directory

[perryitay]

setuptools version

65.2.0

Python version

Python 3.9

OS

Ubuntu

Additional environment information

No response

Description

when trying to update setuptools to a ver >= 65.2.0, tried various versions. (65.1.1 works fine)

I get this errors for different extensions:

   INFO:build_extension:building 'M2Crypto._m2crypto' extension^M
      INFO:swig_sources:swigging src/SWIG/_m2crypto.i to src/SWIG/_m2crypto_wrap.c^M
      INFO:spawn:swig -python -py3 -I/usr/lib/gcc/x86_64-linux-gnu/7/include -I/usr/local/include -I/usr/lib/gcc/x86_64-linux-gnu/7/include-fixed -I/usr/include/x86_64-linux-gnu -I/usr/include -D__x86_64__ -I/opt/redislabs/include/python3.9 -I/usr/include/openssl -includeall -modern -builtin -outdir /tmp/pip-req-build-6_7oepv5/src/M2Crypto -o src/SWIG/_m2crypto_wrap.c src/SWIG/_m2crypto.i^M
      INFO:mkpath:creating build/temp.linux-x86_64-cpython-39^M
      INFO:mkpath:creating build/temp.linux-x86_64-cpython-39/src^M
      INFO:mkpath:creating build/temp.linux-x86_64-cpython-39/src/SWIG^M
      INFO:spawn:gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/opt/redislabs/include/python3.9 -I/tmp/pip-req-build-6_7oepv5/src/SWIG -c src/SWIG/_m2crypto_wrap.c -o build/temp.linux-x86_64-cpython-39/src/SWIG/_m2crypto_wrap.o -Wno-deprecated-declarations -DTHREADING^M
      src/SWIG/_m2crypto_wrap.c:151:11: fatal error: Python.h: No such file or directory^M
       # include <Python.h>^M
                 ^~~~~~~~~~^M
      compilation terminated.^M
      error: command '/usr/bin/gcc' failed with exit code 1^M
      [end of output]^M
  ^M
  note: This error originates from a subprocess, and is likely not a problem with pip.^M
  ERROR: Failed building wheel for M2Crypto^M
  error: subprocess-exited-with-error^M
  ^M
  × python setup.py bdist_wheel did not run successfully.^M
  │ exit code: 1^M
  ╰─> [7 lines of output]^M
      /tmp/cache/bazel/_bazel_root/e65c5d2d008a6d538a6299494bc1c8f6/sandbox/processwrapper-sandbox/29/execroot/redis_enterprise/bazel-out/k8-fastbuild/bin/deps/dependencies/opt/redislabs/lib/python3.9/site-packages/setuptools/dist.py:529: UserWarning: Normalizing '2014.08.15' to '2014.8.15'^M
        warnings.warn(tmpl.format(**locals()))^M
      Python3/_regex.c:46:10: fatal error: Python.h: No such file or directory^M
       #include "Python.h"^M

Expected behavior

I except Python.h to exist

How to Reproduce

create a build that includes cffi, mcripto etc. and setuptools larger then 65.2.0

Output

[abravalheri]

Hi @perryitay have you installed the python3.9-dev package on Ubuntu (via apt install)?

(OSs normally split the Python distribution into several packages and only provide a minimal subset by default. On Ubuntu python3.9-dev is required for building c-extensions).

[perryitay]

I do have python3.9-dev installed. and the problem happens only from setuptools >= 65.2.0. why could be the reason for that ?

[jaraco]

Thanks for bisecting the release. According to the changelog, that version introduced one change:

• #3553: Sync with pypa/distutils@22b9bcf, including fixed cross-compiling support and removing deprecation warning per pypa/distutils#169.

[jaraco]

Fixing cross-compiling support was pypa/distutils#173. I suspect that change has broken some assumptions in your environment.

[jaraco]

I'm unable to replicate the issue using pip.

 ~ $ docker run -it jaraco/multipy-tox sh -c 'apt install -qqq -y libffi-dev; py -3.9 -m pip install --no-binary cffi cffi'
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libffi-dev:arm64.
(Reading database ... 28526 files and directories currently installed.)
Preparing to unpack .../libffi-dev_3.4.2-4_arm64.deb ...
Unpacking libffi-dev:arm64 (3.4.2-4) ...
Setting up libffi-dev:arm64 (3.4.2-4) ...
DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
Collecting cffi
  Downloading cffi-1.15.1.tar.gz (508 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 508.5/508.5 kB 6.8 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting pycparser
  Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 kB 22.8 MB/s eta 0:00:00
Building wheels for collected packages: cffi
  Building wheel for cffi (pyproject.toml) ... done
  Created wheel for cffi: filename=cffi-1.15.1-cp39-cp39-linux_aarch64.whl size=384502 sha256=5089f40465bf8dd77513b944e0ca62747704331fa2dc7f51b699b23e093eebef
  Stored in directory: /root/.cache/pip/wheels/f7/2f/c9/36969207bdb71d6830e29c5ecdcc6f54aee2c5631ce15f724e
Successfully built cffi
Installing collected packages: pycparser, cffi
Successfully installed cffi-1.15.1 pycparser-2.21

My guess is there's something special about the sysconfig on that system.

What do you get for python3 -c "import distutils.sysconfig as sc; print(sc.get_python_inc())"?

Can you work out what steps are needed to replicate the failure (on a clean system, maybe Docker)?

[jaraco]

See also pypa/distutils#178 where issues with related logic are being discussed.

[jaraco]

Likely this report is a duplicate of #3657.