Skip to content

Commit

Permalink
fix comment in fsdp_tp_example.py
Browse files Browse the repository at this point in the history
  • Loading branch information
Mddct authored May 27, 2024
1 parent 37a1866 commit baabec4
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion distributed/tensor_parallelism/fsdp_tp_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@
dp_mesh = device_mesh["dp"]

# For TP, input needs to be same across all TP ranks.
# while for SP, input can be different across all ranks.
# while for DP, input can be different across all ranks.
# We will use dp_rank for setting the random seed
# to mimic the behavior of the dataloader.
dp_rank = dp_mesh.get_local_rank()
Expand Down

0 comments on commit baabec4

Please sign in to comment.