add deallocate pipeline output optimization (NVIDIA#8279)

* add deallocate pipeline output optimization Signed-off-by: Jimmy Zhang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <[email protected]> Co-authored-by: Jimmy Zhang <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <[email protected]>
akoumpa · Feb 5, 2024 · cc8d888 · cc8d888
1 parent d758ffb
commit cc8d888
Showing 1 changed file with 1 addition and 0 deletions.
diff --git a/nemo/collections/nlp/models/language_modeling/megatron_base_model.py b/nemo/collections/nlp/models/language_modeling/megatron_base_model.py
@@ -475,6 +475,7 @@ def build_transformer_config(self) -> TransformerConfig:
             'recompute_num_layers': recompute_num_layers,
             'distribute_saved_activations': False,  # not currently used in NeMo
             'fp8': None,
+            'deallocate_pipeline_outputs': True,
         }
 
         # populate the transformer config dict