[Kunlun]fix multi xpu dygraph hang, test=kunlun #32662
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
wangxicoding
requested changes
Apr 29, 2021
paddle/fluid/imperative/reducer.cc
Outdated
| @@ -762,7 +762,7 @@ void Reducer::MarkGroupReady(size_t group_index) { | |||
| // TODO(liuyuhui): Add try catch to deal with exception later, | |||
| // otherwise the main thread will continue to run when an exception is | |||
| // thrown in comm_pool_. | |||
| comm_pool_->enqueue([&] { | |||
| comm_pool_->enqueue([=, &group] { | |||
There was a problem hiding this comment.
一个个指定需要传入的变量。包括this，run_order，另外next_group_是this里面的成员变量，需要用值传入的方式，可以c++14可以[next_group = next_group_]传入，或者先定义好再值传入。
wangxicoding
requested changes
Apr 29, 2021
paddle/fluid/imperative/reducer.cc
Outdated
| @@ -762,10 +762,11 @@ void Reducer::MarkGroupReady(size_t group_index) { | |||
| // TODO(liuyuhui): Add try catch to deal with exception later, | |||
| // otherwise the main thread will continue to run when an exception is | |||
| // thrown in comm_pool_. | |||
| comm_pool_->enqueue([&] { | |||
| auto next_group = next_group_; | |||
| comm_pool_->enqueue([&, run_order, next_group] { | |||
There was a problem hiding this comment.
this, run_order, next_group, &group
明确到各个变量，不要直接用一个&
vslyu
added a commit
to vslyu/Paddle
that referenced
this pull request
Apr 29, 2021
fuyinno4
pushed a commit
that referenced
this pull request
May 1, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Bug fixes
PR changes
Others
Describe
Fix multi kunlun XPU cards dygraph running hang.
ref to PaddleClas [PR690]:PaddlePaddle/PaddleClas#690