You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#7838 implement batch barrier feature, and we also need to implement merging SelectedRows in server site dynamically, just because:
After the distributed transpiler, a SelectedRows variable block maybe contains an empty Tensor, and we don't need to send the empty Tensor in SendOp.
For the server site, we can not use an Op to merge the received SelectedRows, because the Input size should be a definite value, but the number of received SelectedRows variable is different between each batch.
So I think we also need to implement the following feature:
Send Op don't need to send the variable which was not initialized.
Merge SelectedRows in server site.
The text was updated successfully, but these errors were encountered:
Do not generate "empty variables" in transpiler. Since we have barrier we can just send any count of variables and signal the server the send is complete.
Yes, but I think it's not exactly, for the sparse update, the transpiler would split a SelectedRow into multiple SelectedRows by the attribution height_sections, for each batch, SelectedRows.value() could be empty.
So I think that SendOp doesn't need to send an empty variable is Okay.
#7838 implement batch barrier feature, and we also need to implement merging SelectedRows in server site dynamically, just because:
So I think we also need to implement the following feature:
The text was updated successfully, but these errors were encountered: