[review] Open source nvgraph in cugraph repository

[afender]

This is the PR for #33

Contains:

• Copy nvgraph code in cugraph repo under cpp/nvgraph
• Integrate recent changes from perforce and minor cleanup after reviews
• Add a path to produce nvgraph library only
• Add a path to automatically build nvpraph code when building cugraph
• rename nvgraph stand-alone library nvgraph_st into nvgraph_rapids
• Reduce nvgraph build time by disabling undocumented features when built from cugraph (got 3x speedup)
• update readme(s)

[afender]

Hi @raydouglass ,

I can repro the error I see on Jenkins only when I get CUB submodule with the command: git submodule update --init --recursive --remote cpp/nvgraph/cpp/thirdparty/cub .

It works fine when I get cub using git submodule update --init --recursive or even git clone --recurse-submodules [email protected]:afender/cugraph.git.

That sounds like an easy fix we could do on Jenkins, correct?

[raydouglass]

Hi @raydouglass ,

I can repro the error I see on Jenkins only when I get CUB submodule with the command: git submodule update --init --recursive --remote cpp/nvgraph/cpp/thirdparty/cub .

It works fine when I get cub using git submodule update --init --recursive or even git clone --recurse-submodules [email protected]:afender/cugraph.git.

That sounds like an easy fix we could do on Jenkins, correct?

Unfortunately not. Jenkins uses the former method to update submodules.

You should be able to resolve this by add branch information to .gitmodules like cuDF: https://github.com/rapidsai/cudf/blob/branch-0.7/.gitmodules
You likely want to use the same cub repository and branch as cuDF.

[afender]

You should be able to resolve this by add branch information to .gitmodules

Thanks, I added branches to submodules and C++ builds pass now.

However, we have another problem on Jenkins for the python part. Cugraph now builds and installs nvgraph (under ${CMAKE_INSTALL_PREFIX}) but the python setup cannot find it on Jenkins later on (it searches in CONDA_PREFIX/lib and LD_LIBRARY_PATH).

@raydouglass how should we update the setup.py so that it can find libnvgraph_rapids.so on Jenkins?

[afender]

@raydouglass ⬆️

[raydouglass]

@afender I opened a PR against your branch: afender#8
I think that will resolve the problem.

[raydouglass]

LGTM

[afender]

@raydouglass I was just talking to @rlratzel about conda packaging changes resulting from this PR in 0.7 release. I lost track if the updates have been done or not. There are basically two things I wanted to double check with you:

1. We should keep releasing nvgraph as a requirement. We should just make sure that conda install nvgraph reflects what's in https://github.com/rapidsai/cugraph/tree/branch-0.7/cpp/nvgraph rather than https://gitlab-master.nvidia.com/RAPIDS/nvgraph.

2. Now that nvgraph lives inside cugraph and is built along with it, should we update the way the cugraph conda packadge is built in 0.7? I suspect there is something to do so that conda install cugraph also installs nvgraph lib and headers?