CMake build system (feature development tracking PR)

[jeffdonahue]

This PR will officially track the development of the CMake build system for Caffe until it's ready to be merged into dev. I've taken #573 from @akosiorek (currently rebased on the latest dev, thanks!) and force pushed it to BVLC's cmake branch. Future changes to the cmake build system should be PRed against the cmake branch (rather than the normal dev branch) until its official merge into dev. The cmake branch will periodically need to be rebased on dev by someone developing it if it is to be merged; I will force update the cmake branch (and therefore this PR) when that's done. See @shelhamer's very nice explanation of this process in his 2nd followup comment on #610 for details.

TODOs for merge into dev:

• Rebase onto dev
• Verify that (or fix such that) the Travis CI build checks still pass in the current state of the .travis.yml.
• Modify .travis.yml to also successfully perform the CMake build steps, keeping all the current make build steps intact. (You should probably add make clean after the make steps are done, and then add the CMake build steps)
• Add libgflags

[jeffdonahue]

Hey guys, I apologize for the delay and general neglect. Due to #667 and #681, the process of guiding this along has been vastly simplified for me though. So here is the TODO list for merge into dev: [edit: moved to main PR comment, see above]

[jeffdonahue]

Awesome, Travis successfully builds with both CMake and make. I will give this a final review soon. Thanks for all your hard work @akosiorek.

[akosiorek]

I've added CPU-Only build in #784

[bhack]

@jeffdonahue @akosiorek Is there something else in the TODO list?

[akosiorek]

@bhack The commit I've just pushed to my fork's cmake branch introduces a CMake runtest target which does two things:

• it builds tests if they haven't been built earlier
• it performs tests with filters based on whether it is a CPU-Only build or not.

This commit also excludes building tests from the ALL target.

We should probably add lint target, too. But I'd need an explanation of how it is implemented in the Makefile.

[jeffdonahue]

The way the Makefile lint target works is to run cpp_lint.py on each source files (Makefile:322) and output to a .lint.txt file in the build dir, e.g.:

python ./scripts/cpp_lint.py src/path/to/source/file.cpp > build/src/path/to/source/file.cpp.lint.txt

There are a couple pipes into grep -v to ignore the messages that lint prints on success so we get an empty .lint.txt file if there are no lint errors.

Then, once all the per-file .lint.txt files have been created, they are concatenated into one big file (Makefile:310) build/.lint.txt. If the file is non-empty, i.e. there are lint errors, it is moved to a "visible" location build/lint.txt. The Makefile target itself is the empty lint file (build/.lint.txt), so if there are any lint errors, make won't print "nothing to do for target 'lint'" because the target won't exist.

I wouldn't expect a cmake version of lint to be implemented in this exact same way.

[akosiorek]

@jeffdonahue I added runtest and lint targets in #809. May I count for a final review and a merge into dev soon?

[jeffdonahue]

I haven't tested this yet, but while I'm thinking about it, one definite TODO before merge into dev is to add gflags as this is a new dependency as of #776. I'll add a new item to the checklist.

[akosiorek]

I added gflags and rebased onto BVLC/dev in #818

[jeffdonahue]

Thanks, force pushed #818. Doesn't seem to build on Travis though.

[akosiorek]

Sorry about that, I went a little bit too fast with this one. I'll try to figure it out.

[bhack]

@jeffdonahue @akosiorek Please take a look #822.
We need also to limit package install in travis. libopencv-dev libboost-all-dev install too many packages and slow down every builds.

[bhack]

Is there some other bullet on the todo list? Can be merged?

[jeffdonahue]

Sorry again for the delay in reviewing this. I rebased this myself on the latest dev and tried to build using it myself. I did the following:

mkdir build
cd build
cmake ..
make -j8

At first I got an error on cmake .. as I had CMake version 2.8.7 instead of 2.8.8. Since this seems to be the Ubuntu 12.04 default, I added a commit to allow version 2.8.7, which didn't seem to cause any problems -- did you have any particular reason to require >= 2.8.8, @akosiorek?

After that change, cmake .. seemed to work fine, and make -j8 mostly worked but then failed because it couldn't find the curand.h header file I guess (see errors below)? After looking around a bit I couldn't find any obvious reason this might be happening -- it looks like my CUDA include directory was correctly included; I had SET(CMAKE_C_INCLUDE_PATH "../include" "../src" "/usr/local/cuda-5.5/include") in my CMakeFiles/CMakeDirectoryInformation.cmake. Any ideas @akosiorek or anyone else?

[/home/jdonahue/dev/caffe-bvlc/build 274]$ make -j8 all
[  0%] [  2%] Built target gtest
Built target proto
[  5%] Built target gtest_main
[  8%] Building NVCC (Device) object src/caffe/CMakeFiles/caffe_cu.dir/util/./caffe_cu_generated_math_functions.cu.o
Linking CXX static library libcaffe_cu.a
[ 34%] Built target caffe_cu
[ 82%] Built target caffe
Linking CXX executable convert_imageset
Linking CXX executable compute_image_mean
Linking CXX executable device_query
Linking CXX executable extract_features
Linking CXX executable caffe_tool
Linking CXX executable finetune_net
Linking CXX executable net_speed_benchmark
Linking CXX executable dump_network
[ 82%] Built target net_speed_benchmark
[ 82%] Built target device_query
[ 82%] Built target finetune_net
Linking CXX executable test_net
Linking CXX executable train_net
Linking CXX executable upgrade_net_proto_binary
[ 85%] Built target train_net
../lib/libcaffe.a(common.cpp.o): In function `caffe::Caffe::Caffe()':
common.cpp:(.text+0x252): undefined reference to `curandCreateGenerator'
common.cpp:(.text+0x271): undefined reference to `curandSetPseudoRandomGeneratorSeed'
../lib/libcaffe.a(common.cpp.o): In function `caffe::Caffe::~Caffe()':
common.cpp:(.text+0x40e): undefined reference to `curandDestroyGenerator'
../lib/libcaffe.a(common.cpp.o): In function `caffe::Caffe::set_random_seed(unsigned int)':
common.cpp:(.text+0x540): undefined reference to `curandDestroyGenerator'
common.cpp:(.text+0x60c): undefined reference to `curandCreateGenerator'
common.cpp:(.text+0x6d8): undefined reference to `curandSetPseudoRandomGeneratorSeed'
../lib/libcaffe.a(common.cpp.o): In function `caffe::Caffe::SetDevice(int)':
common.cpp:(.text+0xb32): undefined reference to `curandDestroyGenerator'
common.cpp:(.text+0xccd): undefined reference to `curandCreateGenerator'
common.cpp:(.text+0xda5): undefined reference to `curandSetPseudoRandomGeneratorSeed'
../src/caffe/libcaffe_cu.a(caffe_cu_generated_math_functions.cu.o): In function `caffe::caffe_gpu_rng_uniform(int, unsigned int*)':
/home/jdonahue/dev/caffe-bvlc/src/caffe/util/math_functions.cu:380: undefined reference to `curandGenerate'
...
/home/jdonahue/dev/caffe-bvlc/src/caffe/util/math_functions.cu:412: undefined reference to `curandGenerateNormal'
../src/caffe/libcaffe_cu.a(caffe_cu_generated_math_functions.cu.o): In function `void caffe::caffe_gpu_rng_gaussian<double>(int, double, double, double*)':
/home/jdonahue/dev/caffe-bvlc/src/caffe/util/math_functions.cu:419: undefined reference to `curandGenerateNormalDouble'
collect2: ld returned 1 exit status
make[2]: *** [tools/caffe_tool] Error 1
make[1]: *** [tools/CMakeFiles/caffe_tool.dir/all] Error 2
../lib/libcaffe.a(common.cpp.o): In function `caffe::Caffe::Caffe()':
common.cpp:(.text+0x252): undefined reference to `curandCreateGenerator'
common.cpp:(.text+0x271): undefined reference to `curandSetPseudoRandomGeneratorSeed'
../lib/libcaffe.a(common.cpp.o): In function `caffe::Caffe::~Caffe()':
common.cpp:(.text+0x40e): undefined reference to `curandDestroyGenerator'
../lib/libcaffe.a(common.cpp.o): In function `caffe::Caffe::set_random_seed(unsigned int)':
common.cpp:(.text+0x540): undefined reference to `curandDestroyGenerator'
common.cpp:(.text+0x60c): undefined reference to `curandCreateGenerator'
common.cpp:(.text+0x6d8): undefined reference to `curandSetPseudoRandomGeneratorSeed'
../lib/libcaffe.a(common.cpp.o): In function `caffe::Caffe::SetDevice(int)':
common.cpp:(.text+0xb32): undefined reference to `curandDestroyGenerator'
common.cpp:(.text+0xccd): undefined reference to `curandCreateGenerator'
common.cpp:(.text+0xda5): undefined reference to `curandSetPseudoRandomGeneratorSeed'
../src/caffe/libcaffe_cu.a(caffe_cu_generated_math_functions.cu.o): In function `caffe::caffe_gpu_rng_uniform(int, unsigned int*)':
/home/jdonahue/dev/caffe-bvlc/src/caffe/util/math_functions.cu:380: undefined reference to `curandGenerate'
../src/caffe/libcaffe_cu.a(caffe_cu_generated_math_functions.cu.o): In function `void caffe::caffe_gpu_rng_uniform<float>(int, float, float, float*)':
/home/jdonahue/dev/caffe-bvlc/src/caffe/util/math_functions.cu:386: undefined reference to `curandGenerateUniform'
../src/caffe/libcaffe_cu.a(caffe_cu_generated_math_functions.cu.o): In function `void caffe::caffe_gpu_rng_uniform<double>(int, double, double, double*)':
/home/jdonahue/dev/caffe-bvlc/src/caffe/util/math_functions.cu:399: undefined reference to `curandGenerateUniformDouble'
../src/caffe/libcaffe_cu.a(caffe_cu_generated_math_functions.cu.o): In function `void caffe::caffe_gpu_rng_gaussian<float>(int, float, float, float*)':
/home/jdonahue/dev/caffe-bvlc/src/caffe/util/math_functions.cu:412: undefined reference to `curandGenerateNormal'
../src/caffe/libcaffe_cu.a(caffe_cu_generated_math_functions.cu.o): In function `void caffe::caffe_gpu_rng_gaussian<double>(int, double, double, double*)':
/home/jdonahue/dev/caffe-bvlc/src/caffe/util/math_functions.cu:419: undefined reference to `curandGenerateNormalDouble'
collect2: ld returned 1 exit status
make[2]: *** [tools/extract_features] Error 1
make[1]: *** [tools/CMakeFiles/extract_features.dir/all] Error 2
../lib/libcaffe.a(common.cpp.o): In function `caffe::Caffe::Caffe()':
common.cpp:(.text+0x252): undefined reference to `curandCreateGenerator'
common.cpp:(.text+0x271): undefined reference to `curandSetPseudoRandomGeneratorSeed'
../lib/libcaffe.a(common.cpp.o): In function `caffe::Caffe::~Caffe()':
common.cpp:(.text+0x40e): undefined reference to `curandDestroyGenerator'
../lib/libcaffe.a(common.cpp.o): In function `caffe::Caffe::set_random_seed(unsigned int)':
common.cpp:(.text+0x540): undefined reference to `curandDestroyGenerator'
common.cpp:(.text+0x60c): undefined reference to `curandCreateGenerator'
common.cpp:(.text+0x6d8): undefined reference to `curandSetPseudoRandomGeneratorSeed'
../lib/libcaffe.a(common.cpp.o): In function `caffe::Caffe::SetDevice(int)':
common.cpp:(.text+0xb32): undefined reference to `curandDestroyGenerator'
common.cpp:(.text+0xccd): undefined reference to `curandCreateGenerator'
common.cpp:(.text+0xda5): undefined reference to `curandSetPseudoRandomGeneratorSeed'
../src/caffe/libcaffe_cu.a(caffe_cu_generated_math_functions.cu.o): In function `caffe::caffe_gpu_rng_uniform(int, unsigned int*)':
/home/jdonahue/dev/caffe-bvlc/src/caffe/util/math_functions.cu:380: undefined reference to `curandGenerate'
../src/caffe/libcaffe_cu.a(caffe_cu_generated_math_functions.cu.o): In function `void caffe::caffe_gpu_rng_uniform<float>(int, float, float, float*)':
/home/jdonahue/dev/caffe-bvlc/src/caffe/util/math_functions.cu:386: undefined reference to `curandGenerateUniform'
../src/caffe/libcaffe_cu.a(caffe_cu_generated_math_functions.cu.o): In function `void caffe::caffe_gpu_rng_uniform<double>(int, double, double, double*)':
/home/jdonahue/dev/caffe-bvlc/src/caffe/util/math_functions.cu:399: undefined reference to `curandGenerateUniformDouble'
../src/caffe/libcaffe_cu.a(caffe_cu_generated_math_functions.cu.o): In function `void caffe::caffe_gpu_rng_gaussian<float>(int, float, float, float*)':
/home/jdonahue/dev/caffe-bvlc/src/caffe/util/math_functions.cu:412: undefined reference to `curandGenerateNormal'
../src/caffe/libcaffe_cu.a(caffe_cu_generated_math_functions.cu.o): In function `void caffe::caffe_gpu_rng_gaussian<double>(int, double, double, double*)':
/home/jdonahue/dev/caffe-bvlc/src/caffe/util/math_functions.cu:419: undefined reference to `curandGenerateNormalDouble'
collect2: ld returned 1 exit status
make[2]: *** [tools/dump_network] Error 1
make[1]: *** [tools/CMakeFiles/dump_network.dir/all] Error 2
make: *** [all] Error 2

[akosiorek]

@jeffdonahue CMake 2.8.8 adds support for curand. That's why @bhack and me added a step to install the newest CMake on Travis-CI before building.

[jeffdonahue]

Ah...well that would explain it, haha. Sorry; I unpushed that commit. It seems odd to me that support for curand had to specifically be added (just to add -lcurand to the compiler args?), but I have no idea what CMake is doing, so yeah.

Anyway, I tried using sudo add-apt-repository ppa:ubuntu-sdk-team/ppa -y (the line that was added to the .travis.yml), and then did sudo apt-get install cmake -y hoping it would upgrade, but nothing happened (cmake is already the newest version.\n0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.). I also tried removing my current version of cmake with sudo apt-get remove cmake and then reinstalling, but this just reinstalled version 2.8.7. Any suggestions @akosiorek or anyone else?

(edit: I'm using Ubuntu 12.04 btw, which is what Travis's build bots use)

[akosiorek]

@jeffdonahue have you executed sudo apt-get update between adding the ppa and trying to install CMake? I share your doubts whether this CMake version is really necessary. I suspect that it is only a matter of the FindCUDA.cmake scripts which where updated in release 2.8.8. I haven't investigated it any deeper, though

[jeffdonahue]

d'oh... thanks, sudo apt-get update did it. OK, make -j8 completes successfully after updating to cmake 2.8.8. Next I tried to do ctest -v, but that didn't work (see below) -- did I need to do anything extra to build the tests?

[/home/jdonahue/dev/caffe-bvlc/build 14]$ ctest -v
ctest: /home/jdonahue/anaconda/lib/libcurl.so.4: no version information available (required by /usr/lib/libxmlrpc_client.so.3)
Test project /home/jdonahue/dev/caffe-bvlc/build
No tests were found!!!

[akosiorek]

Yes, I removed the ctest support a few commits ago and added a runtest target which builds tests and executes them, pretty much as Make's runtest would do. Just type make runtest -j8

[jeffdonahue]

Got it. Thanks. Everything installed and all the tests passed. In the last commit I added some basic documentation of the cmake build process and a note on installation of cmake 2.8.9 in ubuntu 12.04.

I still need to go through the diff before merging though, sorry.

[akosiorek]

I'm glad that it works for you. I'm sorry for not providing any documentation myself.

[jeffdonahue]

After going through the diff, I did some minor cleanup and moved caffe_tool back to its original caffe name (which required changing the target name, re-adding the .testbin extension for tests, and adding an OUTPUT_NAME).

Thanks for all your great work on this @akosiorek! Well done. The documentation I added is pretty thin -- if you wanted to add a few more details in a later PR (e.g., of the -D options) that would be awesome. But for now I'll merge this to end my saga of neglect.

[bhack]

Tanks to all. We will maintain two build systems or we will remove Makefile from dev branch?

[akosiorek]

Thanks guys!

[kloudkl]

Great job, congratulations!