[BUG] rmm v22.12 does not build from source

[ccoulombe]

Describe the bug
Latest stable version (v0.15) does not build from source as it is missing a dependency.

../include/rmm/mr/device/logging_resource_adaptor.hpp:22:10: fatal error: spdlog/sinks/basic_file_sink.h: No such file or directory
   22 | #include <spdlog/sinks/basic_file_sink.h>

SPDlog is not fetched hence causing build failure. It is also missing from listed/known dependencies: https://github.com/rapidsai/rmm/blob/v0.15.0/README.md#get-rmm-dependencies

Steps/Code to reproduce bug

git clone --recurse-submodules https://github.com/rapidsai/rmm.git -b v0.15.0 && cd rmm
bash build.sh

Expected behavior
Build, fetch missing dependencies, install, all successfully

This PR #580 reference SPDLog as a dep but it should still be listed.

[ccoulombe]

Building spdlog with

git clone https://github.com/gabime/spdlog.git -b v1.8.0 && cd spdlog
mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=..
make -j
make install
export SPDLOG_ROOT=$HOME/spdlog

I managed to build and install rmm in a virtual env. with :

cd $HOME
git clone --recurse-submodules https://github.com/rapidsai/rmm.git -b branch-0.15 && cd rmm
export RMM_ROOT=$HOME/rmm
export INSTALL_PREFIX=$PWD
CPATH=$CPATH:$SPDLOG_ROOT/include LIBRARY_PATH=$LIBRARY_PATH:$RMM_ROOT/lib PARALLEL_LEVEL=16 bash build.sh

The librmm is not found locally when building python with

rmm/build.sh

Line 137 in 8005ca5

python setup.py build_ext --inplace --library-dir="${LIBRMM_BUILD_DIR}"

, hence the LIBRARY_PATH as quick fix.

[kkraus14]

@ccoulombe the spdlog dependency is resolved as of 0.16 where CPM will automatically fetch it as needed: https://github.com/rapidsai/rmm/blob/branch-0.16/cmake/RMM_thirdparty.cmake#L6-L11

As of 0.16 RMM is now header only so there's no longer a need to find librmm in the Python build where the only thing is finding the headers which we look for via a relative path.

Any chance you could try 0.16 and see if your issues are resolved?

[ccoulombe]

@kkraus14 Yes, I can try 0.16 but I'll first need to install cmake >= v0.17 locally. I went for the stable tagged released version instead of the head of the main branch.

I'll let you know

[germasch]

Something seems odd to me. In 0.15, spdlog is fetched via FetchContent. I just tried, and it works as intended for me:

# ...
-- RMM: Enabling the GLIBCXX11 ABI
-- Build spdlog: 1.6.0
-- Build type: Release
# ...

spdlog is present after unning cmake:

root@86b0f1e3d9fb:~/src/ttt/rmm# ls build/_deps/
cnmem-build  cnmem-src  cnmem-subbuild  spdlog-build  spdlog-src  spdlog-subbuild  thrust-build  thrust-src  thrust-subbuild

I'm not quite sure what could have gone wrong for @ccoulombe, unfortunately.

[ccoulombe]

Well, yes I agree @germasch , spdlog is in build/_deps but building the python binding it is not found. Hence the CPATH hint.

The include_dirs in setup.py should include the location of build/_deps/spdlog-src/include, or it should be moved to this

rmm/python/setup.py

Line 83 in 8005ca5

"../build/include",

Note : this directory does not even exists in the build directory.

Or am I missing something? (I am no cmake knowledgable human). Like spdlog include is not added or actually excluded?

rmm/thirdparty/CMakeLists.txt

Line 34 in 8005ca5

add_subdirectory(${spdlog_SOURCE_DIR} ${spdlog_BINARY_DIR} EXCLUDE_FROM_ALL)

[germasch]

Well, yes I agree @germasch , spdlog is in build/_deps but building the python binding it is not found. Hence the CPATH hint.

I see, I didn't realize you the error during the python part of the build. That makes more sense now.

The include_dirs in setup.py should include the location of build/_deps/spdlog-src/include, or it should be moved to this

rmm/python/setup.py

Line 83 in 8005ca5

"../build/include",

Yeah, so this used to be an issue -- the C++ rmm would include spdlog during its build, but then during the python build, it would rely on a separately installed spdlog, usually provided by conda.

In 0.16, the C++ rmm will check whether spdlog is already available, and if so, use it, rather than downloading its own copy. If it is not available, it will download spdlog, and install it together with rmm. This is pending PR #580 being merged. But even in that case, I'll look at whether this will work correctly in the pyhon part of the build, I have a feeling there's still a piece missing to make that work.

[germasch]

So yeah, even after #580 there's still a problem in your case. I have a fix in progress, but I'll do it as a separate PR (after #580) since it affects only the python side of things.

[ccoulombe]

@germasch At the moment, on 0.16, spdlog is not fetched as cmake believes it is locally available (which is not). I'll wait for your PR/fix.

@kkraus14 using 0.16, my issue finding librmm resolved itself as you explained, so it builds/installs when spdlog is correctly found.

[germasch]

@germasch At the moment, on 0.16, spdlog is not fetched as cmake believes it is locally available (which is not). I'll wait for your PR/fix.

@ccoulombe, what likely happened is that you somehow ended up installing spdlog somewhere where cmake now finds it, and that then does also resolve the issue of finding spdlog during the python build. You wouldn't get the message

-- CPM: using local package [email protected]

unless find_package() found spdlog locally and identified its version. I can still reproduce your original issue when there is no locally installed spdlog (and not using conda).

[harrism]

@ccoulombe @germasch #588 has been merged. Can you verify whether or not it fixes this issue?

[germasch]

#588 should fix the python build to find spdlog if it has been fetched during the librmm (C++) build (it does for me).

It may still not fix @ccoulombe's original issue, which comes from the fact that he installed spdlog locally, and then told cmake about it (by setting SPDLOG_ROOT). So the cmake part of the build used it, but the python part of the build still has no idea where to find spdlog, so that's why I'm afraid the situation will be the same as before. It's basically a consequence of using two different build systems (cmake and setuptools) which don't know how to talk to each other.

I don't think there's an easy solution, but @kkraus14 said there's a plan to look into scikit-build, which may (hopefully, but I don't know) improve on that situation. Right now, there are basically three scenarios:

• A thirdparty dependency is missing: the cmake build will fetch it, and will also install it together with RMM, so after [REVIEW] use installed C++ RMM in python build #588, the python build will find it since it knows where RMM is installed. (However, this currently doesn't work for Thrust, since Thrust was missing the option to install when used via subdirectory, but they've now merged my fixes, so this should work in the future.)
• A thirdparty lib is found in a standard location (because it's installed using the distro package manager, or, I guess, conda). In this case, cmake will find it, and the python build will find it because they look in the same standard locations.
• A thirdparty lib is installed in a non-standard location, e.g., installed manually as @ccoulombe did. It's found by cmake because of setting <pkg>_ROOT or CMAKE_PREFIX_PATH, but the python build does not understand these variables, and in fact I believe the python build doesn't have any way to add additional include or library paths.

[ccoulombe]

@harrism Great.
Yes, it does fix the issue. Thanks! Building current HEAD (47457f5); spdlog is correctly found.and rmm builds and installs correctly.

I'll need to wait for the release of v0.17 (what happenned to 0.16?) but still can build rmm without fiddling around.

[kkraus14]

I'll need to wait for the release of v0.17 (what happenned to 0.16?) but still can build rmm without fiddling around.

0.16 release is currently in process where the code is frozen but packages haven't quite been released yet.

[harrism]

@ccoulombe Thanks for the update. I will close this issue now. Please comment or reopen if there's still a problem.

[ccoulombe]

I am reopening this as v22.12 does not build without cuda-python which is needed to build from source.
Idem for scikit-build, but those deps are not well documented, unless you look up the pyproject.toml or environment.yml.

#1146 fixes this but if one would be warn directly from the setup that could be better as well.

[harrism]

In the future, please open a new issue. Especially since this is a very different issue from the old one.