Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SageMaker config stopped working in Accelerate version 0.29+ #2744

Closed
ivan-khvostishkov opened this issue May 5, 2024 · 1 comment · Fixed by #2753
Closed

SageMaker config stopped working in Accelerate version 0.29+ #2744

ivan-khvostishkov opened this issue May 5, 2024 · 1 comment · Fixed by #2753

Comments

@ivan-khvostishkov
Copy link

System Info

The `enable_cpu_affinity` property is missing in `SageMakerConfig` class:

https://github.com/huggingface/accelerate/blob/v0.30.0/src/accelerate/commands/config/config_args.py#L228-L243

But this property is expected by the config parser:

https://github.com/huggingface/accelerate/blob/v0.30.0/src/accelerate/commands/config/config_args.py#L148-L155

As a result, SageMaker config doesn't parse anymore producing the exception:


ValueError: The config file at .../.cache/huggingface/accelerate/default_config.yaml had unknown keys (['enable_cpu_affinity']), please try upgrading your `accelerate` version or fix (and potentially remove) these keys from your config file.


### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] One of the scripts in the examples/ folder of Accelerate or an officially supported `no_trainer` script in the `examples` folder of the `transformers` repo (such as `run_no_trainer_glue.py`)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Create config with `compute_environment: AMAZON_SAGEMAKER` .
2. Run `accelerate config update`

### Expected behavior

Message `Sucessfully updated the configuration file`
@muellerzr muellerzr mentioned this issue May 7, 2024
5 tasks
@muellerzr
Copy link
Collaborator

Thanks @ivan-khvostishkov, can you test it out via this PR? #2753

You can do pip install git+https://github.com/huggingface/accelerate@fix-sagemaker

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants