diff --git a/training/distributed_training/pytorch/model_parallel/gpt-j/11_train_gptj_smp_tensor_parallel_notebook.ipynb b/training/distributed_training/pytorch/model_parallel/gpt-j/11_train_gptj_smp_tensor_parallel_notebook.ipynb index 7f67abd03d..516d06bf7f 100644 --- a/training/distributed_training/pytorch/model_parallel/gpt-j/11_train_gptj_smp_tensor_parallel_notebook.ipynb +++ b/training/distributed_training/pytorch/model_parallel/gpt-j/11_train_gptj_smp_tensor_parallel_notebook.ipynb @@ -11,7 +11,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "This notebook walks you through how to use the tensor parallelism feature provided by the SageMaker model parallelism library. You'll learn how to train the GPT-J model with tensor parallelism on the GLUE sst2 dataset.\n", + "This notebook walks you through how to use the tensor parallelism feature provided by the SageMaker model parallelism library. You'll learn how to run FP16 training of the GPT-J model with tensor parallelism on the GLUE sst2 dataset.\n", "\n", "## Install and Upgrade Libraries\n", "\n", @@ -810,7 +810,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Create a SageMaker HuggingFace 🤗 Estimator\n", + "### Create a SageMaker PyTorch Estimator\n", "\n", "The following cell constructs a PyTorch estimator using the parameters defined above. To see how the SageMaker tensor parallelism modules and functions are applied to the script, see the `train_gptj_smp_tensor_parallel_script.py` file and the private preview documentation. " ] diff --git a/training/distributed_training/pytorch/model_parallel/gpt2/smp-train-gpt-simple.ipynb b/training/distributed_training/pytorch/model_parallel/gpt2/smp-train-gpt-simple.ipynb index 26ef87196f..861fa5b2bd 100644 --- a/training/distributed_training/pytorch/model_parallel/gpt2/smp-train-gpt-simple.ipynb +++ b/training/distributed_training/pytorch/model_parallel/gpt2/smp-train-gpt-simple.ipynb @@ -23,7 +23,7 @@ "- `requirements.txt`: This will install the dependencies, like the right version of huggingface transformers.\n", "- `data_prep_512.py`: This will download and preprocess the openwebtext dataset.\n", "- `memory_tracker.py`: This contains the functions to track memory usage.\n", - "- `sharded_data_parallel_checkpoint.py`: This contains the checkpoint utils for sharded dat parallelism\n", + "- `sharded_data_parallel_checkpoint.py`: This contains checkpoint util functions for sharded data parallelism\n", "\n", "### Additional Resources\n", "If you are a new user of Amazon SageMaker, you may find the following helpful to learn more about SMP and using SageMaker with PyTorch.\n", @@ -528,9 +528,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Create a SageMaker HuggingFace Estimator\n", + "### Create a SageMaker PyTorch Estimator\n", "\n", - "The following cell constructs a HuggingFace estimator using the parameters defined above. To see how the SageMaker tensor parallelism modules and functions are applied to the script, see the `train_gpt_simple.py` file." + "The following cell constructs a PyTorch estimator using the parameters defined above. To see how the SageMaker tensor parallelism modules and functions are applied to the script, see the `train_gpt_simple.py` file." ] }, {