-
Notifications
You must be signed in to change notification settings - Fork 27.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use new parametrization based weight norm if available #24030
Conversation
See pytorch/pytorch#103001 Signed-off-by: Edward Z. Yang <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM as it's properly gated. @sanchit-gandhi can you have a second look?
You'll just need to run make style
and make fix-copies
on your branch to make the CI happy.
The documentation is not available anymore as the PR was closed or merged. |
Signed-off-by: Edward Z. Yang <[email protected]>
Signed-off-by: Edward Z. Yang <[email protected]>
PyTorch side PR has landed! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM too - thanks for the PR @ezyang! Here's the PR in the PyTorch repo for reference: pytorch/pytorch#103001
…4030) * Use new parametrization based weight norm if available See pytorch/pytorch#103001 Signed-off-by: Edward Z. Yang <[email protected]> * handle copies Signed-off-by: Edward Z. Yang <[email protected]> * black Signed-off-by: Edward Z. Yang <[email protected]> --------- Signed-off-by: Edward Z. Yang <[email protected]>
With nightly pytorch, we get
when trying to load a pytorch model into a TF model. The TF model is looking for Question: In your PR pytorch/pytorch#103001, is this part
that deals with the backward compatibility? If so, we will copy it :-) |
* fix * fix * fix --------- Co-authored-by: ydshieh <[email protected]>
Yep. The change here is not FC so the ingester needs updating. |
* fix * fix * fix --------- Co-authored-by: ydshieh <[email protected]>
What does this PR do?
In pytorch/pytorch#103001 I introduce a new
parametrization based version of
weight_norm
. One big benefit ofthe new API is that the resulting model is deepcopy'able; today, you can't
deepcopy Wav2Vec2 models.
Since the new API isn't even in PyTorch main yet, I'd like to feature
gate it here, so that it gets used whenever PyTorch is recent enough to
support it. It would be a big help for me if you could take this change
earlier rather than later; otherwise I will have to patch transformers
in our own CI to get our benchmark harness working on Wav2Vec2.
Signed-off-by: Edward Z. Yang [email protected]
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
cc @sanchit-gandhi