Can not execute setup.py since setuptools is not available in the build environment

[dlesnoff]

Is there an existing issue for this?

• I have searched the existing issues for a bug report that matches the one I want to file, without success.

I found this issue that might be related but I did not understand its contents:
#30371

In sagelib-10.0.log, I have the error « Can not execute setup.py since setuptools is not available in the build environment ».
The exact error status seem to be this paragraph:

    error: Command "g++ -std=gnu++11 -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -g -fwrapv -O2 -Wl,-rpath-link,/home/lesnoff/sage/local/lib -L/home/lesnoff/sage/local/lib -Wl,-rpath,/home/lesnoff/sage/local/lib -Wl,-rpath-link,/home/lesnoff/sage/local/lib -L/home/lesnoff/sage/local/lib -Wl,-rpath,/home/lesnoff/sage/local/lib -g -O2 build/temp.linux-x86_64-cpython-310/sage/libs/eclib/homspace.o -L///usr/local/lib -L/usr/lib/x86_64-linux-gnu -lec -lntl -lpari -lgmp -lm -o build/lib.linux-x86_64-cpython-310/sage/libs/eclib/homspace.cpython-310-x86_64-linux-gnu.so -lpari" failed with exit status 1
    error: subprocess-exited-with-error
    
    × python setup.py develop did not run successfully.
    │ exit code: 1
    ╰─> See above for output.
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
    full command: /home/lesnoff/sage/local/var/lib/sage/venv-python3.10/bin/python3 -c '
    exec(compile('"'"''"'"''"'"'
    # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
    #
    # - It imports setuptools before invoking setup.py, to enable projects that directly
    #   import from `distutils.core` to work with newer packaging standards.
    # - It provides a clear error message when setuptools is not installed.
    # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
    #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
    #     manifest_maker: standard file '"'"'-c'"'"' not found".
    # - It generates a shim setup.py, for handling setup.cfg-only projects.
    import os, sys, tokenize
    
    try:
        import setuptools
    except ImportError as error:
        print(
            "ERROR: Can not execute `setup.py` since setuptools is not available in "
            "the build environment.",
            file=sys.stderr,
        )
        sys.exit(1)
    
    __file__ = %r
    sys.argv[0] = __file__
    
    if os.path.exists(__file__):
        filename = __file__
        with tokenize.open(__file__) as f:
            setup_py_code = f.read()
    else:
        filename = "<auto-generated setuptools caller>"
        setup_py_code = "from setuptools import setup; setup()"
    
    exec(compile(setup_py_code, filename, "exec"))
    '"'"''"'"''"'"' % ('"'"'/home/lesnoff/sage/src/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' --no-user-cfg develop --no-deps
    cwd: /home/lesnoff/sage/src/
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Did you read the documentation and troubleshoot guide?

• I have read the documentation and troubleshoot guide

Environment

- **OS**: 
OS: Ubuntu 22.04.2 LTS x86_64 
Host: HP EliteBook 840 G8 Notebook P
- **Sage Version**: 10.0 (tag on develop branch) / same on 10.1.beta4
- **Python version**: Python 3.10.9 (main, Jan 11 2023, 15:21:40) [GCC 11.2.0] / after an apt update: Python 3.10.6 (main, May 29 2023, 11:10:38) [GCC 11.3.0]
(The patch number decreased).
- **GCC version**:
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/11/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 11.3.0-1ubuntu1~22.04.1' --with-bugurl=file:///usr/share/doc/gcc-11/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-11 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --enable-cet --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-11-aYxV0E/gcc-11-11.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-aYxV0E/gcc-11-11.3.0/debian/tmp-gcn/usr --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-build-config=bootstrap-lto-lean --enable-link-serialization=2
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 11.3.0 (Ubuntu 11.3.0-1ubuntu1~22.04.1) 

PATH, PKG_CONFIG_PATH, LD_LIBRARY_PATH, CFLAGS, CPPFLAGS, CXXFLAGS, and LDFLAGS environment variables
are almost all not defined apart this specific flint version I use:
(I use on other projects fflas-ffpack, flint and ntl).

- **env**:
LD_LIBRARY_PATH=/home/lesnoff/flint/lib/
PATH=/home/lesnoff/miniconda3/condabin:/usr/local/texlive/2023/bin/x86_64-linux:/home/lesnoff/.nimble/bin:/usr/local/texlive/2022/bin/x86_64-linux:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin

Steps To Reproduce

git clone https://github.com/sagemath/sage/
cd sage
git checkout 10.0
./configure
make -j6

Terminal crash when not specifying the number of threads for make. Terminal stays open with make -j6.

Config log

config.log

Package logs

sagelib-10.0.log

Additional Information

pip setuptools seems installed:
pip show setuptools
Name: setuptools
Version: 65.6.3
Summary: Easily download, build, install, upgrade, and uninstall Python packages
Home-page: https://github.com/pypa/setuptools
Author: Python Packaging Authority
Author-email: [email protected]
License:
Location: /home/lesnoff/miniconda3/lib/python3.10/site-packages
Requires:
Required-by: python-lsp-server, sip, spyder

I used conda deactivate first.

command for future compilations (personal reminders):
make -j6 SAGE_KEEP_BUILT_SPKGS=yes

[mkoeppe]

Using make -j is not a good idea for building Sage. You'll likely run out of memory. If the terminal itself crashed, it was probably killed by the OOM killer.

[dlesnoff]

@mkoeppe I cleaned the issue from all references to make -j. The load was too high without a specified number of threads of course.

[mkoeppe]

The bit of the log that you posted above begins with error: Command "g++ -std=gnu++11 ....

The relevant error messages appear before this line. Best to attach the relevant full build log.

[dlesnoff]

In sagelib-10.0.log

@mkoeppe The file sagelib-10.0.log is attached below. Which one do you want exactly ?

[mkoeppe]

Thanks, I missed that. I'll take a look

[mkoeppe]

    /usr/bin/ld: ///usr/local/lib/libntl.a(RR.o): relocation R_X86_64_TPOFF32 against symbol `_ZN3NTL2RR4precE' can not be used when making a shared object; recompile with -fPIC
    /usr/bin/ld: failed to set dynamic section sizes: bad value
    collect2: error: ld returned 1 exit status

[mkoeppe]

This means you have a bad installation of NTL that is sneaking into the build. Delete it and try again

[dlesnoff]

Thanks for pointing out this!
Am I forced to delete my NTL library? Wouldn't it better to force the use of the NTL SPKG?

[mkoeppe]

That should also work.

[dlesnoff]

I ran:

./configure --without-system-ntl --without-system-flint
make -j7

The build is still erroneous, but I can launch a Sage shell. Should I open an issue for this new error or share it here?

Should I rename this issue? I did not find the correct error line in the config.log and I guess it already helps people.

If I give a more precise name, people will not know about this and will keep opening issues named « Can not execute setup.py since setuptools is not available in the build environment ».

I found plenty other issues by searching relocation against symbol, but none that had a similar title.

[mkoeppe]

Let's just continue here. No need to rename the issue.

[dlesnoff]

I forgot to add the config files for the second problem. I upgraded Sagemath and can not reproduce the issue.

[mkoeppe]

Thanks for the update! Glad it's working now.