New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add cpp trainer lib and demo #10681

Merged

jacquesqiao merged 33 commits into PaddlePaddle:develop from jacquesqiao:add-cpp-trainer-demo

May 22, 2018

Member

jacquesqiao commented May 16, 2018 •

edited

Loading

task list: #10574

jacquesqiao added 12 commits

May 11, 2018 10:42


          add demo_network.py

55bc1e1


          Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

07ef3dd

… add-cpp-trainer-demo


          python can run

2becc31


          add demo cmake list

1c56311


          Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

e6399d6

… add-cpp-trainer-demo

tmp

a62093f

tmp

9944bf8


          complete CMakeLists.txt

c1b5502


          optimize CMakeLists.txt

865c951


          update boost dir

8690e08


          can build and run


          Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

531a223

… add-cpp-trainer-demo

jacquesqiao changed the title ~~Add cpp trainer lib and demo~~ [wip]Add cpp trainer lib and demo

jacquesqiao added 6 commits

May 16, 2018 16:15


          delete optimizer, optimize cmake

6310f05


          optimize code, add document

7c84808


          revert code

e31546a


          code clean

a8f0b43


          get loss after run program


          optimize code

da5e462

jacquesqiao changed the title ~~[wip]Add cpp trainer lib and demo~~ Add cpp trainer lib and demo

jacquesqiao added 9 commits

May 16, 2018 18:25


          Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

1479a25

… add-cpp-trainer-demo


          fix compile on server

acadffa


          Merge branch 'add-cpp-trainer-demo' of ssh://github.com/jacquesqiao/P…

436e45b

…addle into add-cpp-trainer-demo


          update cmake

0652d3d


          Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

… add-cpp-trainer-demo


          Merge branch 'add-mkldnn-to-paddle-lib' of ssh://github.com/jacquesqi…

6b6f7b5

…ao/Paddle into add-cpp-trainer-demo


          update document


          update cmake

3368d9a


          Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

454e9b3

… add-cpp-trainer-demo

jacquesqiao requested a review from luotao1

May 17, 2018 05:33

luotao1 reviewed

View reviewed changes

paddle/fluid/train/test/CMakeLists.txt Outdated

		@@ -0,0 +1,64 @@
		cmake_minimum_required(VERSION 3.0)

Contributor

luotao1 May 17, 2018

demo的话，是不是放到fluid/demo下比较好呢？或者doc/fluid下呢？这样以后官网可以展示demo。

Member Author

jacquesqiao May 21, 2018

done, 放到 fluid/train/demo下了，这个和c++训练有关，就先放到train目录下了

paddle/fluid/train/test/CMakeLists.txt Outdated


		set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")

		set(PADDLE_LIB "${PROJECT_SOURCE_DIR}/lib")

Contributor

luotao1 May 17, 2018

PROJECT_SOURCE_DIR请问是什么呢？inference_dist按照后就只有一个目录：PADDLE_LIB就够了。

Member Author

jacquesqiao May 21, 2018

done

paddle/fluid/train/test/CMakeLists.txt Outdated

+              set(PADDLE_LIB "${PROJECT_SOURCE_DIR}/lib")
+              set(MATH_TYPE $ENV{LIB_TYPE} CACHE STRING "Choose the Math library type: openblas mkl")
+              option(WITH_MKLDNN     "Compile PaddlePaddle with MKLDNN"             OFF)

Contributor

luotao1 May 17, 2018

10行可以先去掉，mkldnn的现在效果不好，以后加如何？

Member Author

jacquesqiao May 21, 2018

保留了，默认false

paddle/fluid/train/test/CMakeLists.txt Outdated

+              add_executable(demo_trainer demo_trainer.cc)
+              if(MATH_TYPE STREQUAL "mklml")
+                include_directories("${PADDLE_LIB}/third_party/install/mklml/include")

Contributor

luotao1 May 17, 2018

33行需要加么

Member Author

jacquesqiao May 21, 2018

我自己测试的时候是需要的。

paddle/fluid/train/test/README.md Outdated

+              ### step 1. build paddle lib
+              ```
+              # option MATH_TYPE=mklml

Contributor

luotao1 May 17, 2018 •

edited

Loading

第5行还要加openblas
第5-6行不是编译paddle lib需要的，是第四步的时候才需要，需要移动下位置。

Member Author

jacquesqiao May 21, 2018

done

paddle/fluid/train/test/README.md Outdated

+              ### step 2. copy lib to this dir
+              ```
+              cp -r /paddle/src/dir/paddle/fluid/train/lib .

Contributor

luotao1 May 17, 2018

第2步可以去掉，库安装完不用拷贝，只要在cmake里面加上库的路径即可。

Member Author

jacquesqiao May 21, 2018

done

paddle/fluid/train/test/README.md Outdated

+              This will generate two files:
+                - startup_program: used to init all parameters
+                - main_program: main logic of the network

Contributor

luotao1 May 17, 2018

第三步，需要安装完paddlepaddle whl包的。才能跑。这里也需要说明一下。
这一步跑完有什么打印的信息么？可以写一下，这样用户知道这一步成功了。

paddle/fluid/train/test/README.md Outdated

+              mkdir build
+              cd build
+              cmake ..
+              make

Contributor

luotao1 May 17, 2018

这一步的例子中，cmake选项没有实例，可以参考https://github.com/luotao1/fluid_inference_example#inference-example-project

Member Author

jacquesqiao May 21, 2018

updated

paddle/fluid/train/test/README.md Outdated

+              make
+              cp ../startup_program .
+              cp ../main_program .
+              ./demo_trainer

Contributor

luotao1 May 17, 2018

这里跑完有什么结果么？可以写一下结果，这样用户知道例子跑通了。

Member Author

jacquesqiao May 21, 2018

done

paddle/fluid/train/test/README.md Outdated

+              ### step 4. build demo_trainer and run it.
+              ```
+              mkdir build

Contributor

luotao1 May 17, 2018

这里需要说明下是在当前目录mkdir build，用户不知道去哪儿新建目录。
也需要说明整个demo目录可以放在任意一个地方。

Member Author

jacquesqiao May 21, 2018

done

jacquesqiao added 5 commits

May 21, 2018 09:35


          update lib install dir

44dc25c


          Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

86212a8

… add-cpp-trainer-demo


          optimize code and document

d3882c3


          optimize code

dc03009


          optimize document

b59082d

luotao1 reviewed

View reviewed changes

paddle/fluid/train/demo/demo_trainer.cc

+              }
+              }  // namespace train
+              }  // namespace paddle

Contributor

luotao1 May 21, 2018

ReadBinaryFile和load函数既然在paddle/train的namespace里面，是否应该放在paddle代码里，而不是demo代码里。

Member Author

jacquesqiao May 21, 2018

打算后面统一整理一下

paddle/fluid/train/demo/README.md

		@@ -0,0 +1,66 @@

Contributor

luotao1 May 21, 2018

需要注明是使用CPU静态库的版本。GPU/动态库有点区别，但类似。

Member Author

jacquesqiao May 21, 2018

后面统一整理？目前业务方只需要用CPU的部分，可以先给他们用起来

paddle/fluid/train/demo/README.md

+                       -DWITH_MKL=OFF \
+                       -DWITH_MKLDNN=OFF
+              make -j8
+              make -j8 inference_lib_dist

Contributor

luotao1 May 21, 2018

@Superjomn 训练也使用的话，叫inference_lib_dist好么？改成fluid_lib_dist?

Member Author

jacquesqiao May 21, 2018

打算后面统一整理一下，这个demo可以尽快merge

paddle/fluid/train/demo/README.md

+              ```
+              step: 0 loss: 1069.02
+              step: 1 loss: 1069.02
+              step: 2 loss: 1069.02

Contributor

luotao1 May 21, 2018

WARNING: Logging before InitGoogleLogging() is written to STDERR
W0521 20:55:43.756713 24692 init.cc:84] 'CUDA' is not supported, Please re-compile with WITH_GPU option
W0521 20:55:43.756901 24692 init.cc:100] 'CUDA' is not supported, Please re-compile with WITH_GPU option
step: 0 loss: 58.8651
step: 1 loss: 58.8651
step: 2 loss: 58.8651

我打出来是这样，每个阶段的loss都一样？

Member Author

jacquesqiao May 21, 2018

对，因为没有加入optimize op，有一个参数控制，https://github.com/PaddlePaddle/Paddle/pull/10681/files#diff-7e8d0736b2aff0b2bc699d05f454e0a3R19
这是业务的需求

Member Author

jacquesqiao May 21, 2018

加上之后可以正常收敛

paddle/fluid/train/demo/demo_trainer.cc Outdated


		auto loss_var = scope.Var(loss_name);

		for (int i = 0; i < 100; ++i) {

Contributor

luotao1 May 21, 2018

这里i=10，10个循环就够了


          follow comment, change 100 to 10

luotao1 approved these changes

View reviewed changes

Contributor

luotao1 left a comment

LGTM 后续可以进一步优化demo

jacquesqiao merged commit be26b71 into PaddlePaddle:develop

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet