Fix ds nvme #34444
Fix ds nvme #34444
Conversation
|
@SunMarc @muellerzr Do you have any feedback? |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
pinging @muellerzr ! |
There was a problem hiding this comment.
Hey @eljandoubi , thanks for the contribution. Could you explain why this is needed ? Thanks !
|
Hey @SunMarc. When using NVMe offloading, recursively calling |
|
Thanks for the additional context ! I think that we do support multiple models with deepspeed https://huggingface.co/docs/accelerate/en/usage_guides/deepspeed_multiple_model . cc @muellerzr |
|
Anyway, we might use a context manager solution to prevent embedded calls to DeepSpeed.zero.Init. |
eljandoubi
force-pushed
the
fix_ds_nvme
branch
from
468152c to
87174b9
Compare
|
@ArthurZucker @SunMarc @muellerzr Is there any update on the pull request? |
|
@SunMarc can you underscore them? |
There was a problem hiding this comment.
this and all ipynb are unrelated
| @@ -223,6 +224,15 @@ def set_quantized_state(): | |||
| finally: | |||
| _is_quantized = False | |||
|
|
|||
| @contextmanager | |||
| def set_zero3_state(): | |||
There was a problem hiding this comment.
can we have a small comment documentation about that (explaning the issue we fixed with this basically) 😉
|
Just missing |
* skip nested deepspeed.zero.Init call * make fixup * solve conflict * solve conflict * put back local * use context mangers instead of local thread * Skip recursive calls to deepspeed.zero.Init * Skip recursive calls to deepspeed.zero.Init * back to old notebooks * make style
What does this PR do?
Fixes #34429
Who can review?
Models:
Integrations: