Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migrate HFDeepSpeedConfig from trfrs to accelerate #432

Merged
merged 7 commits into from
Jun 15, 2022
Merged

Migrate HFDeepSpeedConfig from trfrs to accelerate #432

merged 7 commits into from
Jun 15, 2022

Conversation

pacman100
Copy link
Contributor

@pacman100 pacman100 commented Jun 9, 2022 •
edited
Loading

What does this PR do?

  1. Migrates HFDeepSpeedConfig from transformers repo to accelerate repo as it is generic enough and specific bits remain in transformers.
  2. Fix affected code and test; refactoring and improving few things.

@pacman100 pacman100 requested review from sgugger and stas00 June 9, 2022 10:23
@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Jun 9, 2022 •
edited
Loading

The documentation is not available anymore as the PR was closed or merged.

@pacman100 pacman100 mentioned this pull request Jun 9, 2022
Merged
5 tasks
sgugger
sgugger reviewed Jun 9, 2022
View reviewed changes
Copy link
Collaborator

@sgugger sgugger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM except the change of argument name. Can we find a way to keep backward compatibility on this?
Also would it affect old accelerate configs of users (in which case we should add a conversion old config to new config to make their lives easier)

src/accelerate/utils/dataclasses.py Show resolved Hide resolved
@pacman100
update state.py to resolve comments
26a66a3 
1. Adds static method to have a simple API for integrating deepspeed config in transformers trainer.
sgugger
sgugger approved these changes Jun 14, 2022
View reviewed changes
Copy link
Collaborator

@sgugger sgugger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice new utilities! Could you add docstrings on those four new static methods?

src/accelerate/state.py Outdated Show resolved Hide resolved
src/accelerate/state.py Outdated Show resolved Hide resolved
@stas00 stas00 removed their request for review June 15, 2022 14:44
@pacman100 pacman100 merged commit 873dcc6 into huggingface:main Jun 15, 2022
@pacman100 pacman100 deleted the smangrul/migrate-HFDeepSpeedConfig branch July 13, 2022 02:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sgugger sgugger sgugger approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@pacman100 @HuggingFaceDocBuilderDev @sgugger