[bug] smdistributed is not included in pytorch-training:2.4.0-gpu-py311

[gui-miotto]

Checklist

• I've prepended issue tag with type of change: [bug]
• (If applicable) I've attached the script to reproduce the bug
• (If applicable) I've documented below the DLC image/dockerfile this relates to
• (If applicable) I've documented below the tests I've run on the DLC image
• I'm using an existing DLC image listed here: https://docs.aws.amazon.com/deep-learning-containers/latest/devguide/deep-learning-containers-images.html
• I've built my own container based off DLC (and I've attached the code used to build my own image)

Concise Description:

Running

import smdistributed.dataparallel.torch.torch_smddp

Results in

ModuleNotFoundError: No module named 'smdistributed'

DLC image/dockerfile:

763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:2.4.0-gpu-py311-cu124-ubuntu22.04-sagemaker

[sirutBuasai]

Hi, SMDDP releases are not coupled with DLC which is why it is not currently present in PyTorch 2.4 Training DLC. However, we are working on releasing a new SMDDP binary compatible with PyTorch 2.4 in the coming weeks.

I will follow up once we have a PR up for adding the SMDDP binary on PyTorch 2.4 DLC.

[sirutBuasai]

Hi, to follow up, we have a PR open for adding SMDDP to PT 2.4 Training SM DLC #4403

[gui-miotto]

Hey @sirutBuasai , thanks for the kind reply.
I'm happy that you guys shipped an PyTorch 2.5.1 DLC!
Do you know when the SMDDP for PyTorch 2.5 will be released? I'm talking about the .whl itself, not necessarily a DLC containing it.
Checking the release notes, it seems to me that cadence is so slow. :-(

[sirutBuasai]

@gui-miotto, unfortunately we don't have direct insight into the SMDDP release schedule.
For PT 2.4 SM DLC, we have a RC build with SMDDP and we're planning on releasing a new version on 12/9/24