2017 06 07

wangkuiyi

Porting Majel and improving the building system

• https://github.com/PaddlePaddle/Paddle/pull/2339
• https://github.com/PaddlePaddle/Paddle/pull/2326
• https://github.com/PaddlePaddle/Paddle/pull/2343#pullrequestreview-41423553
• https://github.com/dmlc/mshadow/issues/254

Errands

• https://github.com/PaddlePaddle/Paddle/pull/2414#pullrequestreview-42735001

helinwang

• Setup Teamcity CI to check PaddlePaddle Pull Requests.

  • It runs build and GPU/CPU tests. There are 3 GPU enabled build agents, build statistics (click login as guest).
  • Fix frequent Teamcity CI test failures: https://github.com/PaddlePaddle/Paddle/pull/2414
  • Make Teamcity visible outside of Baidu: http://19196ef4.ngrok.io/ , people can login as a guest.

• Create recordio repo, add Python binding, wheel packaging and CI test:

  • https://github.com/PaddlePaddle/recordio/pull/1
  • https://github.com/PaddlePaddle/recordio/pull/3
  • https://github.com/PaddlePaddle/recordio/pull/4
  • https://github.com/PaddlePaddle/recordio/pull/5

• Code Reviews

  • Optimizer: https://github.com/PaddlePaddle/Paddle/pull/2386#pullrequestreview-42115122
  • File Manager: https://github.com/PaddlePaddle/cloud/pull/112#pullrequestreview-41651168
  • New remote parameter updater: https://github.com/PaddlePaddle/Paddle/pull/2413#pullrequestreview-42699278

gongweibao

• Code Reviews
  • add script to package recordio into wheel: https://github.com/PaddlePaddle/recordio/pull/5
  • improve doc and travis CI: https://github.com/PaddlePaddle/recordio/pull/4
  • Submit job with upload user's package:https://github.com/PaddlePaddle/cloud/pull/128
• Pull Request
  • change restclient file: https://github.com/PaddlePaddle/cloud/pull/123
  • Add convert function: https://github.com/PaddlePaddle/Paddle/pull/2407
  • merge filemanager module: https://github.com/PaddlePaddle/cloud/pull/112

dongzhihong

• Develop

  • polish and add more features to the optimizer library https://github.com/PaddlePaddle/Paddle/pull/2386
  • develop go part based optimizer library, will send PR after optimizer merged.
  • trainer design doc discussion https://github.com/PaddlePaddle/Paddle/pull/2363 https://github.com/PaddlePaddle/Paddle/issues/2364 https://github.com/PaddlePaddle/Paddle/issues/2347

• Code Review

  • trainer updater https://github.com/PaddlePaddle/Paddle/pull/2330
  • https://github.com/PaddlePaddle/Paddle/pull/2413

Yancey1989(yanxu)

• PaddleCloud
  • Add a demo fit_a_line in PaddleCloud, https://github.com/PaddlePaddle/cloud/pull/130
  • [Reviewing] A doc about how to develop cloud dataset,https://github.com/PaddlePaddle/cloud/pull/134/files
  • PaddleCloud mount public dataset, https://github.com/PaddlePaddle/cloud/pull/122
• Code Reviews:
  • https://github.com/PaddlePaddle/cloud/pull/128
• MPI
  • [WIP]Test V2 API receiver and deploy for hlan and 10g MPI server

GangLiao

• Github PR

  • add OBJECT property in generic CMake https://github.com/PaddlePaddle/Paddle/pull/2359
  • rename CMAKE_CURRENT_LIST_DIR to CMAKE_CURRENT_SOURCE_DIR https://github.com/PaddlePaddle/Paddle/pull/2383
  • Survey of Eigen in order to compare with Majel https://github.com/PaddlePaddle/Paddle/wiki/A-Survey-and-Taxonomy-of-Majel https://github.com/PaddlePaddle/Paddle/wiki/A-Survey-and-Taxonomy-of-Eigen
  • TeamCity CI only output failure https://github.com/PaddlePaddle/Paddle/pull/2390
  • update docker docs https://github.com/PaddlePaddle/Paddle/pull/2362
  • update book docker docs https://github.com/PaddlePaddle/book/pull/329

• GitHub issue

  • Is cuDNN/BLAS, MKL, Neon all BLAS compatible? https://github.com/PaddlePaddle/Paddle/issues/2397

Xingzhaolong

• pr
  • improve pruning https://github.com/PaddlePaddle/Paddle/pull/2354 user need to specify a sparisity_ratio for each layer.
• auto pruning
  • Investigation on dynamic network surgery
  • Investigation on direct convolution
  • the whole pruning process for a model takes several stages. For example, if u want to reach the ratio of sparse of 0.7, u must not directly specify the ratio of 0.7 for fine-tuning. It will cause a lot of accuracy drop. So, it would be better to increase the sparsity in several stages. This will keep the accuracy loss within a reasonable range, but it will take a lot of time. I have did a demo on caffe platform to complete it in a whole process. it's have a perfect result on oxford flowers 102 reasult see here, the test on larger dataset is in the process. If test well, i'll update it to paddle.

wanghaoshuang

• DeepSpeech2:
  • Make DS2 run on Kubernates clusters: https://github.com/PaddlePaddle/Paddle/issues/2381
• other:
  • simple document about attributes of nnvm: doc

qiaolongfei

ParameterUpdater using go:

• issue: https://github.com/PaddlePaddle/Paddle/issues/2274
• can use go cclient to communicate with pserver now: https://github.com/PaddlePaddle/Paddle/pull/2413

Code review:

• Optimizer Lib: https://github.com/PaddlePaddle/Paddle/pull/2339

survey and ducument:

• survey of tensorflow: https://github.com/PaddlePaddle/Paddle/wiki/TensorFlow%E8%B0%83%E7%A0%94
• document of parameter updater in Paddle: https://github.com/PaddlePaddle/Paddle/wiki/Paddle%E7%9B%AE%E5%89%8D%E7%9A%84%E5%AE%9E%E7%8E%B0%E7%9A%84%E5%90%84%E4%B8%AA%E6%A8%A1%E5%9D%97%E7%BB%86%E8%8A%82#parameterupdater

luotao

• remove duplicated examples among demo/models/book, and rename remain demo to v1_api_demo: #2357
• fix bugs:
  • fix Broken link to DL 101 book: #2358 #2391
  • remove top_k argument in classification_cost #2412
• Wechat PaddlePaddle:
  • support artical with latex formulas
  • 179 fans -> 216 fans

qijun

• survey dynet: https://github.com/PaddlePaddle/Paddle/wiki/dynet%E8%B0%83%E7%A0%94
• survey caffe2: https://github.com/PaddlePaddle/Paddle/wiki/Caffe2%E8%B0%83%E7%A0%94
• survey operator register framework of mxnet

fengjiayi

• survey on mshadow and lazy operation: https://github.com/PaddlePaddle/Paddle/wiki/mshadow%E8%B0%83%E7%A0%94
• survey on Caffe2, with Qijun: https://github.com/PaddlePaddle/Paddle/wiki/Caffe2%E8%B0%83%E7%A0%94
• update the demo script of cluster job, fix bugs: http://wiki.baidu.com/pages/viewpage.action?pageId=327596461

livc(Zhao Li)

• Issue:https://github.com/PaddlePaddle/book/issues/328
• 手音：Began to use convolution neural network training model, the accuracy rate rose to 67%

Dang qingqing

• DeepSpeech2

  • Finish the row convolution operation for both CPU and GPU implementation.
    • https://github.com/PaddlePaddle/Paddle/pull/2407
  • Support variable-length input and SortaGrad.
    • https://github.com/PaddlePaddle/models/pull/74
    • https://github.com/PaddlePaddle/models/issues/75
    • The speed of variable-length input is faster based on the train-clean-100 dataset of LibriSpeech.
      • 3752 sec/epoc: pad all samples to 2000
      • 3286 sec/epoc: shuffle all samples -> make mini-batch -> pad each batch to same size.
      • 2861 sec/epoc: sort all samples by length -> make mini-batch -> shuffle batches -> pad each batch to same size.

• Traffic lights detection

  • Verify the training on the refactored code.
  • help Yaming to solve some problems when porting the v1's demo to v2 API.

• PR View

  • https://github.com/PaddlePaddle/Paddle/pull/2357
  • https://github.com/PaddlePaddle/Paddle/pull/2282
  • https://github.com/PaddlePaddle/Paddle/pull/2242#pullrequestreview-41961043
  • https://github.com/PaddlePaddle/models/pull/58
  • https://github.com/PaddlePaddle/models/pull/52

Xinghai Sun

• DeepSpeech2
  • Solved the training convergence problem.
    • https://github.com/PaddlePaddle/models/pull/55
    • Trained a toy model with full LibriSpeech 960-Hours dataset (WER 22, without beam search and LM).
  • Reading speech_dl codes from SVAIL.
  • Working on decoder, batch arrangement, augmentation pipeline, Kubernetes running (with Qingqing, Yibing, Yaming, Shaoyong, Haoshuang).
  • Other Pull Requests
    • https://github.com/PaddlePaddle/models/pull/69
    • https://github.com/PaddlePaddle/models/pull/78
  • PR Reviews
    • https://github.com/PaddlePaddle/models/pull/71#pullrequestreview-41991713
    • https://github.com/PaddlePaddle/models/pull/59#pullrequestreview-41982390
    • https://github.com/PaddlePaddle/models/pull/72#pullrequestreview-42558801

Yibing Liu

• DS2: CTC Beam Search Decoder

  • PR: https://github.com/PaddlePaddle/models/pull/59
  • Design Doc: https://github.com/PaddlePaddle/Paddle/pull/2423
  • Confirm correctness by comparing with the decoder in Tensorflow
  • Read speech_dl code for decoder part
  • Integrate with CTC network and language model
  • Two things remaining:
    • Optimizie decoding efficiency
    • Tune parameters by quantitative index(WER/CER)

• Code Review:

  • https://github.com/PaddlePaddle/models/pull/69
  • https://github.com/PaddlePaddle/models/pull/55

Yu Yang

• ComputationGraph Refactorization.
  • Survey on Tensorflow/Caffe2/MXNet/DyNet/PyTorch and give two talks this week.
    1. design overview of these frameworks
    2. Computation graph implementation survey of these frameworks.
• Keep servuy on mxnet, caff2's computation graph implementation.
• Writting a toy project to help me thinking how to implement a compuatation graph.
  • https://github.com/reyoung/NaiveNet

Caoying

• A simple survey of torch, the second part is written by @yanchunwei

  • https://github.com/PaddlePaddle/Paddle/wiki/PyTorch-Survey
  • Pytorch is designed in quite a different way to PaddlePaddle, I think we can learn from it, but not in a direct way. So I currently lower the priority of this work.

• some modifications to refine the model project

  • review and merge CTR example
    • https://github.com/PaddlePaddle/models/pull/57#pullrequestreview-41400273
  • https://github.com/PaddlePaddle/models/pull/61
  • https://github.com/PaddlePaddle/models/pull/62
  • https://github.com/PaddlePaddle/models/pull/76

• fix bugs in PaddlePaddle:

  • fix the bug in parsing the network configuration for machine translation and SRL
    • https://github.com/PaddlePaddle/Paddle/pull/2384
  • add an helper for prelu layer and refine the API doc:
    • https://github.com/PaddlePaddle/Paddle/pull/2412
  • Last week, we've discussed to fix the way V2 api parses the network configuration. This work is already done by this commit. I learn this commit, and fix some small bugs.

Shaoyong Xu

• DeepSpeech: Audio data augmentation
  • Read speech_dl code for augmentation part
  • Construct the code of augmentation part of speech_dl：https://github.com/chrisxu2016/models/tree/develop/deep_speech_2/augmentation
  • Design doc: https://github.com/chrisxu2016/models/blob/develop/deep_speech_2/augmentation/README.md
• code view:
  • https://github.com/PaddlePaddle/models/pull/55

typhoonzero(wuyi)

• Paddle cloud:
  • bug fixes and enhancements and public datasets: https://github.com/PaddlePaddle/cloud/pulls?q=is%3Apr+is%3Aclosed
  • pfs review and merged. need more tests
• MPI cluster v2 training issues
• Test for running kubernetes on baidu cloud bare-metal servers with hardware VxLAN environment
• Document for distributed training with v2 API update: https://github.com/PaddlePaddle/Paddle/pull/2072
• Look into Tensorflow to catch up current NN framework designs, layer->op, tensor->variable

hedaoyuan

• Convolution Function and Reconstruction Convolution Layer
  https://github.com/PaddlePaddle/Paddle/pull/2282
  https://github.com/PaddlePaddle/Paddle/issues/2424
  https://github.com/PaddlePaddle/Paddle/issues/2425

• Code Review
  https://github.com/PaddlePaddle/Paddle/pull/2373
  https://github.com/PaddlePaddle/Paddle/pull/2299
  https://github.com/PaddlePaddle/Paddle/pull/2341

yangyaming

• Feature
  • DetectionUtil (Some utility functions)
    https://github.com/PaddlePaddle/Paddle/pull/2357
  • DS2 (Support initializing model from a pre-trained model)
    https://github.com/PaddlePaddle/models/pull/72
• Traffic light detection demo for ADU
  • v2 network configuration (done)
  • training、test、eval and infer scripts (done)
  • usage document (MIP)

juliecbd

• PR
  • https://github.com/PaddlePaddle/Paddle/pull/2417
• Created Reinforcement learning demo using v2 api: to be uploaded

Liu Yiqun

• Refine the docmentation of layers
  • fix some typo and add more detail about the usage of warp_ctc layer https://github.com/PaddlePaddle/Paddle/pull/2376
• Build and install Paddle on NVIDIA DRIVE PX2
  • Remove DYNAMIC_ARCH option of buliding openblas https://github.com/PaddlePaddle/Paddle/pull/2393
  • Remove the dependency of opencv-python for arm-based platforms https://github.com/PaddlePaddle/Paddle/pull/2394
• Code review
  • https://github.com/PaddlePaddle/Paddle/pull/2354
• issue
  • https://github.com/PaddlePaddle/Paddle/issues/2379

Yan Chunwei

• finish CTC tutorial, waiting for PR
  • https://github.com/PaddlePaddle/models/pull/63
• finish CTR tutorial, merged into paddle/models
  • https://github.com/PaddlePaddle/models/tree/develop/ctr
• survey PyTorch from user's view
  • wiki was merged into PyTorch-Survey
• survey mxnet::engine