[ci] CUDA 11.8 wheel (gcc) CI jobs failing: 'libomp.so.5: no such file or directory'

[jameslamb]

Description

All CUDA 11.8 wheel CI jobs are currently failing, with this error showing up in tests (the first time import lightgbm is run):

__________________ ERROR collecting tests/c_api_test/test_.py __________________
tests/c_api_test/test_.py:10: in <module>
    from lightgbm.basic import _LIB as LIB
/github/home/.local/lib/python3.10/site-packages/lightgbm/__init__.py:8: in <module>
    from .basic import Booster, Dataset, Sequence, register_logger
/github/home/.local/lib/python3.10/site-packages/lightgbm/basic.py:278: in <module>
    _LIB = _load_lib()
/github/home/.local/lib/python3.10/site-packages/lightgbm/basic.py:262: in _load_lib
    lib = ctypes.cdll.LoadLibrary(lib_path[0])
/tmp/miniforge/envs/test-env/lib/python3.10/ctypes/__init__.py:452: in LoadLibrary
    return self._dlltype(name)
/tmp/miniforge/envs/test-env/lib/python3.10/ctypes/__init__.py:374: in __init__
    self._handle = _dlopen(self._name, mode)
E   OSError: libomp.so.5: cannot open shared object file: No such file or directory

(example build link)

Reproducible example

CI on every branch.

Environment info

Latest master.

Additional Comments

I see this in the CMake logs:

-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")

So OpenMP is being found in builds.

It's also surprising to see LightGBM linking against libomp, not libgomp, in a build supposedly using gcc. Maybe this is a result of building inside a conda environment but not using conda's compilers (ref: #5106 (comment)).

libomp is LLVM's OpenMP implementation, that accompanies clang.

Discussion that led to this: #6459 (comment)