meson build issue with ibm-clang_r compiler

[Ganeshkumhar1]

Describe the bug
while building the numpy 2.0.1 on AIX with ibm-clang_r compiler , meson build is hitting issue while detecting the linker.

To Reproduce
$ python -m build --wheel --no-isolation .

• Getting build dependencies for wheel...
• Building wheel...

• /tmp/3392379693/python/bin/python /home/user/numpy/2.0.1/numpy-2.0.1/vendored-meson/meson/meson.py setup /home/user/numpy/2.0.1/numpy-2.0.1 /home/user/numpy/2.0.1/numpy-2.0.1/.mesonpy-5yatrh9i -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=/home/user/numpy/2.0.1/numpy-2.0.1/.mesonpy-5yatrh9i/meson-python-native-file.ini
  The Meson build system
  Version: 1.4.99
  Source dir: /home/user/numpy/2.0.1/numpy-2.0.1
  Build dir: /home/user/numpy/2.0.1/numpy-2.0.1/.mesonpy-5yatrh9i
  Build type: native build
  Project name: NumPy
  Project version: 2.0.1

../meson.build:1:0: ERROR: Unable to detect linker for compiler ibm-clang_r -Wl,--version -L/tmp/3392379693/python/lib -I/tmp/3392379693/python/include -I/tmp/3392379693/python/include/openssl
stdout:
stderr: ld: 0706-012 The -- flag is not recognized.
.ibm-clang: error: linker command failed with exit code 255 (use -v to see invocation)

Expected behavior
it should build numpy

system parameters

• Is this a cross build or just a plain native build (for the same computer)? plain native build
• what operating system (e.g. MacOS Catalina, Windows 10, CentOS 8.0, Ubuntu 18.04, etc.) - AIX Version 7.2
• what Python version are you using e.g. 3.8.0 - Python 3.12.6
• what meson --version. - 1.5.2

numpy/numpy#27531

[Ganeshkumhar1]

Hi, Is this queue getting monitored for support.

[QuLogic]

cc @KamathForAIX

[KamathForAIX]

@QuLogic @Ganeshkumhar1

As of now ibm-clang_r compiler does not work in AIX with meson. Only GCC is. So any package that uses meson [numpy/scipy/matplotlib/p11-kit and so on] builds using GCC only in AIX.

You may try with GCC.

[eli-schwartz]

If someone wants to add support, by the way, we'd be happy to merge it. We may be able to mentor for it.

I definitely cannot do it myself -- I don't have an AIX system to experiment with. The GCC Compile Farm does, and I actually tried applying, but I got a rather strange answer and no account...

[KamathForAIX]

@eli-schwartz We can work together on this.

One thing I observe is this is seen only in meson version < 16.
export OBJECT_MODE=64
export CC="/opt/IBM/openxlC/17.1.2/bin/ibm-clang_r"
export CXX="/opt/IBM/openxlC/17.1.2/bin/ibm-clang++_r"
export CFLAGS="-pthread -m64"
export CXXFLAGS="-pthread -m64"
meson --version
1.2.1

For example,

# /opt/freeware/bin/meson setup build --wipe
The Meson build system
Version: 1.2.1
Source dir: /meson_test
Build dir: /meson_test/build
Build type: native build
Project name: hello_world_lib
Project version: undefined

meson.build:1:0: ERROR: Unable to detect linker for compiler `/opt/IBM/openxlC/17.1.2/bin/ibm-clang_r -Wl,--version -pthread -m64`
stdout: 
stderr: ld: 0706-012 The -- flag is not recognized.
.ibm-clang: error: linker command failed with exit code 255 (use -v to see invocation)

But with the master branch we are able to build :),

meson --version

1.6.0.rc2

# meson setup build --wipe
The Meson build system
Version: 1.6.0.rc2
Source dir: /meson_test
Build dir: /meson_test/build
Build type: native build
Project name: hello_world_lib
Project version: undefined
C compiler for the host machine: /opt/IBM/openxlC/17.1.2/bin/ibm-clang_r (clang 17.1.2 "IBM Open XL C/C++ for AIX 17.1.2 (5725-C72, 5765-J18), version 17.1.2.5, clang version 17.0.6 (build 483fa9b)")
C linker for the host machine: /opt/IBM/openxlC/17.1.2/bin/ibm-clang_r ld.aix 7.2.5
Host machine cpu family: ppc
Host machine cpu: powerpc
Build targets in project: 2

Found ninja-1.11.1 at /opt/freeware/bin/ninja

Unknowingly something has fixed this in AIX. Let me check.

[KamathForAIX]

I suspect it to be this fix: b7ce7c2

That has fixed AIX linker detection with clang as well. :)

I will test without this commit in AIX and then confirm

[KamathForAIX]

@eli-schwartz I can confirm from AIX that after I revoked b7ce7c2 from my friendly fork I reproduced the error.

So b7ce7c2 fixes this issue as well.

# meson setup build --wipe
The Meson build system
Version: 1.6.0.rc2
Source dir: /meson_test
Build dir: /meson_test/build
Build type: native build
Project name: hello_world_lib
Project version: undefined

meson.build:1:0: ERROR: Unable to detect linker for compiler `/opt/IBM/openxlC/17.1.2/bin/ibm-clang_r -Wl,--version -pthread -m64`
stdout: 
stderr: ld: 0706-012 The -- flag is not recognized.
.ibm-clang: error: linker command failed with exit code 255 (use -v to see invocation)


A full log can be found at /meson_test/build/meson-logs/meson-log.txt

[KamathForAIX]

@QuLogic @rgommers Kindly add to the vendored meson in numpy the commit b7ce7c2 so that AIX users can build numpy and you can close the issue in numpy's end. Attaching the succeessful tail of build log of numpy using ibm-clang below this message. I had patched the meson to the commit during my build.

export CC="/opt/IBM/openxlC/17.1.2/bin/ibm-clang_r -pthread"
export CXX="/opt/IBM/openxlC/17.1.2/bin/ibm-clang++_r -pthread"
export CFLAGS=-m64
export CXXFLAGS=-m64
pip3.11 install . --root $PWD/buildroot --ignore-installed -I --no-deps -v

One can use the above instructions to build numpy in AIX. Can change m64 to maix64 and clang to gcc if they want to try in GCC.

[940/940] /numpy/numpy/_core/tests/examples/limited_api/setup.py
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: numpy
  Running command Building wheel for numpy (pyproject.toml)
  Building wheel for numpy (pyproject.toml) ... done
  Created wheel for numpy: filename=numpy-2.2.0.dev0-cp311-cp311-aix_7205_2419_64.whl size=8361993 sha256=6c7fae0d3178d84a03483a0e7c4da26de558bf9218fe7a96bb8ad10c05a7b283
  Stored in directory: /tmp/pip-ephem-wheel-cache-fzp1vcea/wheels/9f/a0/f8/2ec58081d66e1a4b23039073b242a0986904857ea63b6894c4
Successfully built numpy
Installing collected packages: numpy
  Creating /numpy/%{buildroot}/.local/bin
  changing mode of %{buildroot}/.local/bin/f2py to 755
  changing mode of %{buildroot}/.local/bin/numpy-config to 755
  WARNING: The scripts f2py and numpy-config are installed in '%{buildroot}/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed numpy-2.2.0.dev0

[rgommers]

Great, thanks @KamathForAIX. I'll update the vendored Meson in NumPy.

[KamathForAIX]

@eli-schwartz

Kindly let me know if we can update release notes or write in documents like here given commit b7ce7c2 makes meson work with ibm-clang.

[eli-schwartz]

Yes, please add a release notes entry for 1.6.0.

[Ganeshkumhar1]

@KamathForAIX ,
I am getting this error now -

bash-5.2$ meson --version
1.6.0.rc2
bash-5.2$ ```
meson setup build_dir --buildtype=release
The Meson build system
Version: 1.6.0.rc2
Source dir: /home/user/numpy/2.0.1/numpy-2.0.1
Build dir: /home/user/numpy/2.0.1/numpy-2.0.1/build_dir
Build type: native build
Project name: NumPy
Project version: 2.0.1
C compiler for the host machine: ibm-clang_r (clang 17.1.2 "IBM Open XL C/C++ for AIX 17.1.2 (5725-C72, 5765-J18), version 17.1.2.11, clang version 17.0.6 (build 89643fc 89643fc7f8e51cc3bfdf69b4b65ea9f42c71dd66)")
C linker for the host machine: ibm-clang_r ld.aix 7.2.5
C++ compiler for the host machine: ibm-clang++_r (clang 17.1.2 "IBM Open XL C/C++ for AIX 17.1.2 (5725-C72, 5765-J18), version 17.1.2.11, clang version 17.0.6 (build 89643fc 89643fc7f8e51cc3bfdf69b4b65ea9f42c71dd66)")
C++ linker for the host machine: ibm-clang++_r ld.aix 7.2.5
Cython compiler for the host machine: cython (cython 3.0.10)
Host machine cpu family: ppc64
Host machine cpu: powerpc
Program python3 found: YES (/tmp/2796265743/python/bin/python3.12.bin)
Found pkg-config: YES (/usr/bin/pkg-config) 0.29.2
Run-time dependency python found: YES 3.12
Has header "Python.h" with dependency python-3.12: YES
Compiler for C supports arguments -fno-strict-aliasing: YES
Compiler for C supports arguments -ftrapping-math: YES

meson_cpu/x86/meson.build:2:15: ERROR: Module "features" does not exist

A full log can be found at /home/user/numpy/2.0.1/numpy-2.0.1/build_dir/meson-logs/meson-log.txt

[Ganeshkumhar1]

I had installed module features as well but still same issue .

$ python -m pip list | grep features
features 0.5.12

[KamathForAIX]

@Ganeshkumhar1 You are going to get that error since vendored meson is not updated. Also, features is a module within vendored meson here and not features module from pypi.

Can you try one thing.

Run "git submodule update --init " and then in the vendored meson,

/numpy/vendored-meson/meson/mesonbuild/linkers/linkers.py,

Update changes to the above file from the commit b7ce7c2

Then you will be able to build.

[KamathForAIX]

Thank you @jpakkane