2018 03 21

Yang Yang (tonyyang-svail)

PR:

1. C++ Multi GPU Executor: https://github.com/PaddlePaddle/Paddle/pull/9035

Review

1. [Speed] Refine parallel_do_grad: https://github.com/PaddlePaddle/Paddle/pull/9299
2. Shrink batch_norm_grad's inputs: https://github.com/PaddlePaddle/Paddle/pull/9072

ranqiu

• API doc problems (nn and ops)

  https://github.com/PaddlePaddle/Paddle/issues?q=is%3Aissue+is%3Aopen+label%3Adocumentation+label%3Apythonapi

• PR review

  https://github.com/PaddlePaddle/Paddle/pull/9157

gongweibao

• Reuduce memory copy when communication between trainer and pserver
  • https://github.com/PaddlePaddle/Paddle/pull/9271

tangwei

• EDL

  • DeepSpeech2 on Sys Kubernetes Cluster
    • Distribute with CPU and EDL, done
    • Distribute with GPU, debugging

• GPUDriect RDMA

  • Nvidia/gdrcopy research
  • module install and deploy
  • test sample structuring

• PR [BUG FIX]:

  • https://github.com/PaddlePaddle/cloud/issues/646
  • https://github.com/PaddlePaddle/cloud/issues/648

• PR [DOC]

  • https://github.com/PaddlePaddle/cloud/issues/644
  • https://github.com/PaddlePaddle/Paddle/issues/8637

• Review

  • https://github.com/PaddlePaddle/edl/pull/14

• ISSUE [USER]

  • https://github.com/PaddlePaddle/cloud/issues/645

helinwang

• Simple baseline parallel executor and debugging with YangYang
• Reviews:
  • Distributed lookup table
    • https://github.com/PaddlePaddle/Paddle/pull/9075#pullrequestreview-104311819
    • https://github.com/PaddlePaddle/Paddle/issues/9211#issuecomment-374322850
  • Distributed training optimizations
    • https://github.com/PaddlePaddle/Paddle/issues/9161#issuecomment-373853897
    • https://github.com/PaddlePaddle/Paddle/pull/9209#pullrequestreview-105164276
    • https://github.com/PaddlePaddle/Paddle/pull/9303#pullrequestreview-105865481
  • EDL
    • https://github.com/PaddlePaddle/edl/pull/12#pullrequestreview-104741291
  • 2018 milestone: https://github.com/PaddlePaddle/Paddle/issues/9108#issuecomment-373481203
  • CI
    • https://github.com/PaddlePaddle/Paddle/pull/9150#issuecomment-374413918
    • https://github.com/PaddlePaddle/Paddle/pull/9150#pullrequestreview-105584339
  • [Speed] scope.Findvar maybe slow down all the operators: https://github.com/PaddlePaddle/Paddle/issues/9232#event-1530105632
  • FP16
    • https://github.com/PaddlePaddle/Paddle/pull/9267#pullrequestreview-105903136
    • https://github.com/PaddlePaddle/Paddle/pull/9269#pullrequestreview-105912235
• Help AMD developers on integrating AMD GPU build and test onto CI (WIP)

luotao

• fix AttributeError: 'module' object has no attribute 'framework_pb2': https://github.com/PaddlePaddle/Paddle/pull/9128
• [WIP]fuse batch norm
• code review:
  • MKLDNN:
    • Implementation of MKLDNN LRN: https://github.com/PaddlePaddle/Paddle/pull/9123
    • MKLDNN Relu Tanh Sqrt Abs activations added: https://github.com/PaddlePaddle/Paddle/pull/9081
    • [Merge] Softmax MKLDNN FLUID operator: https://github.com/PaddlePaddle/Paddle/pull/9214
  • [AMD] Demostration of cmake refine for HIP support: https://github.com/PaddlePaddle/Paddle/pull/9165
  • doc:
    • https://github.com/PaddlePaddle/Paddle/pull/9226
    • https://github.com/PaddlePaddle/Paddle/pull/9235
    • https://github.com/PaddlePaddle/Paddle/pull/9241
    • https://github.com/PaddlePaddle/Paddle/pull/9233
    • https://github.com/PaddlePaddle/Paddle/pull/9236
  • add math_function to selected_rows_functor and softmax dependency list:
    • https://github.com/PaddlePaddle/Paddle/pull/9134
    • https://github.com/PaddlePaddle/Paddle/pull/9219

qijun

Memory optimization:

Model	no optimize	forward memory	reuse memory	release memory	recomputation and release memory
Resnet	169791488	77500416	92602368	78020608	59027456

• [Memory][WIP] Recompute policy in memory optimization
  • https://github.com/PaddlePaddle/Paddle/pull/9089

Code Review:

• https://github.com/PaddlePaddle/Paddle/pull/9080#discussion_r175673679

Yan Xu

• [WIP] Turn on cmaka flag WITH_DISTRIBUTE on CI
  • issue: https://github.com/PaddlePaddle/Paddle/issues/9046
  • PR: https://github.com/PaddlePaddle/Paddle/pull/9150
• [WIP] Parallel send gradients and execution of backward ops
  • issue: https://github.com/PaddlePaddle/Paddle/issues/9161
  • RP: https://github.com/PaddlePaddle/Paddle/pull/9303
• PR Review:
  • https://github.com/PaddlePaddle/cloud/pull/647#pullrequestreview-104844594
  • https://github.com/PaddlePaddle/Paddle/pull/9154#discussion_r175679001
  • https://github.com/PaddlePaddle/Paddle/pull/9288#pullrequestreview-105684088

Yu Yang

• [WIP] C++ ParallelExecutor
  • https://github.com/PaddlePaddle/Paddle/pull/9080
• Several enhancements
  • https://github.com/PaddlePaddle/Paddle/pull/9299
  • https://github.com/PaddlePaddle/Paddle/pull/9293

kexinzhao

• FP16 inference
  • Can successfully run vgg16 inference on CIFAR10 in float16 mode
  • Add float16 support for cudnn conv2d: https://github.com/PaddlePaddle/Paddle/pull/9143
  • Add float16 support for cast op: https://github.com/PaddlePaddle/Paddle/pull/9148
  • Add float16 support for pool 2d operator: https://github.com/PaddlePaddle/Paddle/pull/9167
  • Add float16 support to batch norm operator: https://github.com/PaddlePaddle/Paddle/pull/9176
  • Add float16 support to dropout operator: https://github.com/PaddlePaddle/Paddle/pull/9223
  • Add float16 support to Elementwise Add op: https://github.com/PaddlePaddle/Paddle/pull/9231
  • Add float16 support to relu op: https://github.com/PaddlePaddle/Paddle/pull/9267
  • Add float16 support to cudnn softmax kernel: https://github.com/PaddlePaddle/Paddle/pull/9269
• Review
  • Inference:
    • https://github.com/PaddlePaddle/Paddle/pull/9242#pullrequestreview-105600284
    • https://github.com/PaddlePaddle/Paddle/pull/9207#pullrequestreview-105575228
    • https://github.com/PaddlePaddle/Paddle/pull/9043#pullrequestreview-104788445

zhangchao

• Update the RNN Search Model for multi-gpu.
  • https://github.com/PaddlePaddle/models/pull/729
  • https://github.com/dzhwinter/benchmark/issues/87
  • https://github.com/PaddlePaddle/models/issues/757
• Refine the input of sequence_softmax in machine_translation.py.
  • https://github.com/dzhwinter/benchmark/pull/89

wuyi

• fluid dist train perf enhancements:
  • pserver runs parrallel https://github.com/PaddlePaddle/Paddle/pull/9154
  • prepare before run https://github.com/PaddlePaddle/Paddle/pull/9287
  • review: https://github.com/PaddlePaddle/Paddle/pull/9271
  • Bug: https://github.com/PaddlePaddle/Paddle/issues/9284
• remote lookup table discussions: https://github.com/PaddlePaddle/Paddle/issues/9211

yangyaming

• [Merged] Enhance LoDResetOp
  https://github.com/PaddlePaddle/Paddle/pull/9204
• [Merged] Enhance SequenceExpandOp
  https://github.com/PaddlePaddle/Paddle/pull/9100
• [WIP] Evaluate BLEU for RNN Search model
• Survey for ONNX
  https://github.com/PaddlePaddle/Paddle/pull/9296

fengjiayi

• Multi-threaded C++ Reader:
  • https://github.com/PaddlePaddle/Paddle/pull/9182
• Multi-pass C++ Reader:
  • https://github.com/PaddlePaddle/Paddle/pull/9259
• Bugs fix:
  • C++ Readers: https://github.com/PaddlePaddle/Paddle/pull/9178
  • RecordIO: https://github.com/PaddlePaddle/Paddle/pull/9240
• [WIP] C++ Reader profile

Dang Qingqing

• SSD on Fluid:
  • Fix a critical bug in softmax_with_cross_entropy_op.
    • https://github.com/PaddlePaddle/Paddle/pull/9120
  • MobileNet-SSD on single GPU:
    • mAP: Fluid 71.78% with part data argumentaion Vs Caffe 72.7% with all data argumentation.
  • Expose RMSProp optimizer.
    • https://github.com/PaddlePaddle/Paddle/pull/9247
  • Parallel training for MobileNet-SSD.
    • https://github.com/PaddlePaddle/models/pull/746
  • Temporarily fix bug for backward tanspiler when using parallel_do operator.
    • https://github.com/PaddlePaddle/Paddle/pull/9282
  • Delete the detection_output_op, which had been split into several operators.
    • https://github.com/PaddlePaddle/Paddle/pull/9121
• Other:
  • Fix bug in LRN operator.
    • https://github.com/PaddlePaddle/Paddle/pull/9124
  • Always synchronize when copy data on GPU from C++ to Numpy array.
    • https://github.com/PaddlePaddle/Paddle/pull/9110
  • Problem of BatchNorm in Fluid.
    • https://github.com/PaddlePaddle/Paddle/issues/9273
  • [WIP] Profiling parallel_do op.

wanghaoshuang

• Merge model average optimizer

  • https://github.com/PaddlePaddle/Paddle/pull/9082
  • validation on ctc model: https://github.com/PaddlePaddle/Paddle/issues/9172

• Add python wrapper for Adadelta optimizer

  • https://github.com/PaddlePaddle/Paddle/pull/9213

• Add model average option for OCR CTC model

  • https://github.com/PaddlePaddle/models/pull/740

• Add parallel option for OCR CTC training.

  • https://github.com/PaddlePaddle/models/pull/744

• Review:

  • https://github.com/PaddlePaddle/Paddle/pull/9247
  • https://github.com/PaddlePaddle/Paddle/pull/9121
  • https://github.com/PaddlePaddle/Paddle/pull/9120
  • https://github.com/PaddlePaddle/Paddle/pull/9100
  • https://github.com/PaddlePaddle/models/pull/746

guosheng

• NMT:
  • Refine and fix inference program for Transformer(Merged).
    • https://github.com/PaddlePaddle/models/pull/727
  • Refine the ReshapeOp enhancement.
    • https://github.com/PaddlePaddle/Paddle/pull/9008
  • Evaluate Transformer on WMT16.
    • (Dict Size 3000) BLEU vs. PyTorch: 28.57 vs. 28.10
    • (Dict Size 10000) BLEU: 30.8
• Review:
  • https://github.com/PaddlePaddle/models/pull/689

Chenxi

• PR

  • https://github.com/PaddlePaddle/Paddle/pull/9130
  • https://github.com/PaddlePaddle/edl/pull/13
  • https://github.com/PaddlePaddle/Paddle/pull/9137
  • https://github.com/PaddlePaddle/Paddle/pull/9134

• Issues

  • https://github.com/PaddlePaddle/Paddle/issues/9133
  • https://github.com/PaddlePaddle/Paddle/issues/9135

Liu Yiqun

• Inference Framework
  • Add relu cudnn kernel
    • https://github.com/PaddlePaddle/Paddle/pull/9207
  • Add an argument in Executor.Run to allow users to choose whether to create and destroy variables every time
    • https://github.com/PaddlePaddle/Paddle/pull/9242
  • Enable the test of not creating variables every time
    • https://github.com/PaddlePaddle/Paddle/pull/9301
  • Add multi-thread inference example which shares the inference_program and parameters
    • https://github.com/PaddlePaddle/Paddle/pull/9302
  • Review
    • add MKL for fluid static and shared library: https://github.com/PaddlePaddle/Paddle/pull/8887

qiaolongfei

fluid

• Fluid support Abacus(discuss with @wuyi @yanxu @helin @wangyi @lidong)

• Project: https://github.com/PaddlePaddle/Paddle/projects/56

• Overall Plan:

  • two ways：https://github.com/PaddlePaddle/Paddle/pull/9075
    1. Abacus kv-store migration by @LiDong, we provide technical support.
    2. Fluid will provide an open source implementation.

• Fluid implementation:

  1. Add design doc for lookup remote table https://github.com/PaddlePaddle/Paddle/pull/9068
  2. Add table design doc https://github.com/PaddlePaddle/Paddle/pull/9210
  3. model asq_and_ubmq check, we only need to add distribute lookup table.
  4. TODO: https://github.com/PaddlePaddle/Paddle/issues/9211

• Others:

  • Discuss the way to speed up the download of boost.

• Review

  • Fix a program copy regression. https://github.com/PaddlePaddle/Paddle/pull/9141
  • Add float16 support to batch norm operator https://github.com/PaddlePaddle/Paddle/pull/9176
  • Shrink batch_norm_grad's inputs https://github.com/PaddlePaddle/Paddle/pull/9299

Yibing Liu

• Refactor and wrap decoder for DeepAsr model
  • https://github.com/PaddlePaddle/models/pull/753
  • https://github.com/PaddlePaddle/models/pull/736
• Train model on aishell dataset (public, Madarin, 178h)
• [WIP] Add design doc for onnx convertor
  • https://github.com/PaddlePaddle/Paddle/pull/9296

Code Review:

• https://github.com/PaddlePaddle/Paddle/pull/9008
• https://github.com/PaddlePaddle/models/pull/749
• https://github.com/PaddlePaddle/DeepSpeech/pull/185

dongzhihong

• [Speed] accelerate 6x sequence expand/grad op, time cost per minibatch 0.9 -> 0.17
  • https://github.com/PaddlePaddle/Paddle/pull/9289
• [Speed] acclerate sequence pooling op, speed up the sequence average, sum, chip
  • https://github.com/PaddlePaddle/Paddle/pull/9217
• [NLP/lego] support sequence tagging model. fix bug in dropout
  • https://github.com/PaddlePaddle/Paddle/issues/9144
• [NLP/lego] support seq2seq model, fix bug in gpu mode
  • https://github.com/dzhwinter/benchmark/issues/87
• [NLP/lego]
  • https://github.com/PaddlePaddle/Paddle/pull/9304

Xin Pan

• Fix a regression, restore 10% on resnext
  • https://github.com/PaddlePaddle/Paddle/pull/9141
• Enable P2P memory copy, reduce 4.0 to 3.8 on K40
  • https://github.com/PaddlePaddle/Paddle/pull/9190
• Add memory caching to transformer, reduce 25% overhead
  • https://github.com/PaddlePaddle/models/pull/741
• Add multi-gpu to transfomer, 4 device speedup 1.33
  • https://github.com/PaddlePaddle/models/pull/745
• Review some NLP codes, such as beam search, expand sequence
• Follow up with QA team
• Follow up with performance test machines
• Collect information for the risk of 5.1 Paddle Cloud Launch

Yan Chunwei

• Model CE

  • https://github.com/PaddlePaddle/MacroCE/pull/1
  • https://github.com/PaddlePaddle/MacroCE/pull/2
  • https://github.com/PaddlePaddle/MacroCE/pull/4

• Conv Sequence to sequence

  • https://github.com/PaddlePaddle/models/pull/686

• VisualDL

  • https://github.com/PaddlePaddle/VisualDL/pull/328#pullrequestreview-105586724

zhaochengduo

• PR
  • [WIP] Add pinned memory
    • https://github.com/PaddlePaddle/Paddle/pull/9216
  • Add more times close test
    • https://github.com/PaddlePaddle/Paddle/pull/9215
  • Fix v2_pooling doc
    • https://github.com/PaddlePaddle/Paddle/pull/9314
• Review
  • Cpp parallel executor
  • Enable P2P memory copy
  • Add memcpy_stream and use multi-thread to issue parameter copies
  • support float16:
    • https://github.com/PaddlePaddle/Paddle/pull/9231
    • https://github.com/PaddlePaddle/Paddle/pull/9223
    • https://github.com/PaddlePaddle/Paddle/pull/9167
    • https://github.com/PaddlePaddle/Paddle/pull/9143
  • Fix compilation for gcc5.4
  • Support copy in Fluid channels
  • Update index_en.rst
  • Add default value of keyword argument to DocString in channel_send

Wei Xing

Bug fix:

• Set WITH_DOC=${WITH_DOC:-OFF} in paddle/scripts/docker/build.sh and remove paddle/scripts/tools/build_doc.sh
  • https://github.com/PaddlePaddle/Paddle/pull/9233

Doc:

• Build basic sphinx doctree for doc/fluid

  • https://github.com/PaddlePaddle/Paddle/pull/9236

• Adjust some contents in write_docs_en.rst for Contribute Documentation

  • https://github.com/PaddlePaddle/Paddle/pull/9147

• Add contents for manully build documentation(cn version)

  • https://github.com/PaddlePaddle/Paddle/pull/9298

abhinavarora

• PR
  • Fluid channels should match the semantics of Go Channels https://github.com/PaddlePaddle/Paddle/pull/9265
  • Add default value of keyword argument to DocString in channel_send https://github.com/PaddlePaddle/Paddle/pull/9262
  • Support copy in Fluid channels https://github.com/PaddlePaddle/Paddle/pull/9138
  • Fix unused variable build error https://github.com/PaddlePaddle/Paddle/pull/9140
  • Add modification to Channel to support Select OP https://github.com/PaddlePaddle/Paddle/pull/9084
• Issues
  • Move all Concurrency operators to paddle/fluid/operators/concurrency https://github.com/PaddlePaddle/Paddle/issues/9086
  • Channel Destroy should inform Select callback of destruction. https://github.com/PaddlePaddle/Paddle/issues/9087
• Reviews
  • https://github.com/PaddlePaddle/Paddle/pull/9130#pullrequestreview-104339031
  • https://github.com/PaddlePaddle/Paddle/pull/9132#pullrequestreview-104399634
  • https://github.com/PaddlePaddle/Paddle/pull/9136#pullrequestreview-104399519
  • https://github.com/PaddlePaddle/Paddle/pull/9147#pullrequestreview-105054759
  • https://github.com/PaddlePaddle/Paddle/pull/9157#pullrequestreview-105427071
  • https://github.com/PaddlePaddle/Paddle/pull/9215#pullrequestreview-105054058
  • https://github.com/PaddlePaddle/Paddle/pull/9280#pullrequestreview-105991130
  • https://github.com/PaddlePaddle/Paddle/pull/9286#pullrequestreview-105988048

jetfuel

Plan 0.0.3 release goals https://github.com/PaddlePaddle/VisualDL/projects/5

• PR

  • Provide manual Save feature: https://github.com/PaddlePaddle/VisualDL/pull/332
  • Adjust sync cycle: https://github.com/PaddlePaddle/VisualDL/pull/328
  • Fix the issue where Scalar always misses the last record: https://github.com/PaddlePaddle/VisualDL/pull/331

• Issue

  • https://github.com/PaddlePaddle/VisualDL/issues/322

daming-lu

Plan 0.0.3 release goals https://github.com/PaddlePaddle/VisualDL/projects/5

• In progress: Interactive ONNX Graphs
  • https://github.com/PaddlePaddle/VisualDL/issues/241
• Back from Beijing to Sunnyvale and rest for 1 day

cs2be (thuan)

• PR
  • Fix issue of Paddle API documentation not updating on website (https://github.com/PaddlePaddle/PaddlePaddle.org/pull/443)
  • Create select_op design document (https://github.com/PaddlePaddle/Paddle/pull/9139)
• Reviews
  • https://github.com/PaddlePaddle/Paddle/pull/9138
  • https://github.com/PaddlePaddle/Paddle/pull/9265
  • https://github.com/PaddlePaddle/Paddle/pull/9136
• Other
  • Svail AWS account maintenance

aroravarun

• Concurrency reorganizing and subsequent build failure on thread debugging with Thuan - https://github.com/PaddlePaddle/Paddle/pull/9136
• New concurrency test - https://github.com/PaddlePaddle/Paddle/pull/9132
• Document planning and review: https://github.com/PaddlePaddle/Paddle/pull/9139
• Assist with VDL user study for audio component

sidgoyal78

(PTO for most of the week)

• Following up with TensorRT errors with engineers from Nvidia
• PR review: https://github.com/PaddlePaddle/Paddle/pull/9280