Skip to content

Commit

Permalink
improve error msg when checking target_blocks in activation_checkpoin…
Browse files Browse the repository at this point in the history
…ting_target (#977)
  • Loading branch information
cli99 authored Feb 15, 2024
1 parent 73033ee commit 2e8982e
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion llmfoundry/models/utils/act_ckpt.py
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ def check_mapping_blocks_overlap(mapping: dict, max_block_idx: int) -> None:
else:
if all_blocks[vv] is not None:
raise ValueError(
f'Block {vv} is assigned to both {k} and {all_blocks[vv]}.'
f'Block {vv} is assigned to both {k} and {all_blocks[vv]}. Each block can only have one granularity of activation checkpointing. Make sure the target_blocks in activation_checkpointing_target do not overlap. For more details, refer to the docs of activation_checkpointing_fn.'
)
else:
all_blocks[vv] = k
Expand Down

0 comments on commit 2e8982e

Please sign in to comment.