[R] fix OpenMP detection on macOS

[jameslamb]

Detection of OpenMP on macOS in the R package's configure script is currently not working.

This PR proposes switching from a C compiler to a C++ compiler in R-package/configure to fix that.

Why using a C compiler breaks this check

This code in configure.ac...

xgboost/R-package/configure.ac

Line 23 in e7d612d

AC_LANG(C++)

xgboost/R-package/configure.ac

Line 57 in e7d612d

AC_LANG_CONFTEST([AC_LANG_PROGRAM([[#include <omp.h>]], [[ return (omp_get_max_threads() <= 1); ]])])

... results in autoconf generating a file conftest.cpp

xgboost/R-package/configure

Line 2745 in e7d612d

ac_ext=cpp

xgboost/R-package/configure

Line 2849 in e7d612d

cat confdefs.h - <<_ACEOF >conftest.$ac_ext

... but then XGBoost's configure file is looking instead for a file named conftest.c

xgboost/R-package/configure.ac

Line 58 in e7d612d

${CC} -o conftest conftest.c ${CPPFLAGS} ${LDFLAGS} ${OPENMP_LIB} ${OPENMP_CXXFLAGS} 2>/dev/null && ./conftest && ac_pkg_openmp=yes

How I tested this

On my mac tonight (macOS 12.2.1, R 4.2.2, compiling with clang++ 13.0.0), I ran the following on master

cd R-package
R CMD INSTALL .

And saw the following in logs

checking whether OpenMP will work in a package... no
*****************************************************************************************
         OpenMP is unavailable on this Mac OSX system. Training speed may be suboptimal.
         To use all CPU cores for training jobs, you should install OpenMP by running

             brew install libomp
*****************************************************************************************

@trivialfis noted the same thing on recent R Hub checks, #8669 (comment)

Repeating that on this branch, the R package compiles, links to OpenMP, and its unit tests succeed.

cd tests
Rscript testthat.R

Notes for Reviewers

Sorry for not noticing this while reviewing #8669. It wasn't until today that I had time to look closely and test.

In LightGBM, we try to catch issues like this by grep-ing the build logs: https://github.com/microsoft/LightGBM/blob/5df7d516f3e64e8431aecfea892a850afeb787bb/.ci/test_r_package.sh#L250-L267. I'm not proposing that here, but you might want to consider it in the future.

Thanks for your time and consideration.

[trivialfis]

Thank you for the fix! Small question inlined in the comments.

I don't have a macos device, @hcho3 please help look into this. ;-)

[trivialfis]

We might need the CC flag here due to function registration in R-package/src/init.c.

[hcho3]

Yes, let's retain this line.

[hcho3]

Let's retain this line too.

[hcho3]

LGTM. But please keep the two lines with CC and CFLAGS.

[hcho3]

@trivialfis I think we should include this in our CRAN submission, if you plan to re-submit.

[jameslamb]

Sure, sorry about that! Just pushed 2f59118 adding the C compiler back.

And regenerated configure like this:

autoconf \
    --output configure \
    ./configure.ac

[trivialfis]

In LightGBM, we try to catch issues like this by grep-ing the build logs

That's a really good suggestion. Will open an issue for this.

[leedrake5]

@jameslamb I've been strugglebus with xgboost for months on MacOS, see issue here. Minor update to that is that XGBoost in python can fully use CPU with Mac's M1 line of processors, but no evidence that the R version can yet.

I ran the following install conditions you noted with the latest build:

cd R-package R CMD INSTALL .

But trying out xgboost in R, I got the following error:

OMP: Error #15: Initializing libomp.dylib, but found libomp.dylib already initialized. OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://openmp.llvm.org/

I disabled makevars to make this a vanilla test. It's possible this is still specific to Mac's ARM architecture, but wanted to note the problem in this thread.

[jameslamb]

Just like that error says, you probably have multiple versions of OpenMP installed on your Mac.

You could find them like this:

find /usr -name 'libomp.dylib' -type f
find ${HOME} -name 'libomp.dylib' -type f

One common cause of this is the use of conda, which may put many copies of OpenMP onto your system (see this discussion in the LightGBM docs.

For example, on my mac I see the following running those commands

/usr/local/Cellar/libomp/15.0.7/lib/libomp.dylib
${HOME}/miniconda3/pkgs/llvm-openmp-14.0.4-ha654fa7_0/lib/libomp.dylib
${HOME}/miniconda3/pkgs/llvm-openmp-15.0.6-h61d9ccf_0/lib/libomp.dylib
${HOME}/miniconda3/lib/libomp.dylib
${HOME}/miniconda3/envs/xgboost-dev/lib/libomp.dylib
${HOME}/miniconda3/envs/pycharm-lightgbm/lib/libomp.dylib
${HOME}/miniconda3/envs/pycharm-lightgbm/lib/python3.9/site-packages/sklearn/.dylibs/libomp.dylib

If you're using R from conda, you can see the suggestions from #1715 for how to work around that.

If not, and you want to be sure R processes always find the Homebrew one, you could run something like the following.

export LD_LIBRARY_PATH="/usr/local/opt/libomp/lib:${LD_LIBRARY_PATH}"

Or achieve similar in ~/.bashrc, ~/.bashprofile, etc.

---

Please also note that I'm not a maintainer here. But I suspect that maintainers would prefer you search the open issues for similar error messages (like #5496 which you commented on a few years ago), and then if you don't find what you're looking for, open a new issue (instead of commenting on a closed pull request) with questions like this. And ideally providing a minimal, reproducible example showing exactly what you did and describing your environment.

[leedrake5]

@jameslamb Thanks for the very helpful feedback, but apologies if the original message went astray, just wanted to report the problem I ran into following the installation instructions here. I think you've correctly identified the problem on my setup, currently working to resolve that.

[barracuda156]

@trivialfis @jameslamb It is still broken, at least with GCC, and gives a wrong recommendation (libomp should only be used with Clang and libc++):

checking whether OpenMP will work in a package... no
*****************************************************************************************
         OpenMP is unavailable on this Mac OSX system. Training speed may be suboptimal.
         To use all CPU cores for training jobs, you should install OpenMP by running\n
             brew install libomp

P. S. Perhaps “should” should not be used with an optional package manager.