Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FSMT] Fix non-shared weights #26187

Merged
merged 3 commits into from
Sep 18, 2023
Merged

[FSMT] Fix non-shared weights #26187

merged 3 commits into from
Sep 18, 2023

Conversation

LysandreJik
Copy link
Member

FSMT shares weights across three layers.

The tie_weights method wasn't having the intended effect on all three (only on the input and output embeddings), leading to errors when converting the model to safetensors.

See https://huggingface.co/facebook/wmt19-en-de/discussions/7

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Sep 15, 2023

The documentation is not available anymore as the PR was closed or merged.

Copy link
Collaborator

@ArthurZucker ArthurZucker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, apart for the tests/models/fsmt/test_modeling_fsmt.py::FSMTModelTest::test_tied_weights_keys test that is failing.

@LysandreJik LysandreJik merged commit 77ed9fa into main Sep 18, 2023
3 checks passed
@LysandreJik LysandreJik deleted the fsmt-fix branch September 18, 2023 14:58
MKhalusova pushed a commit to MKhalusova/transformers that referenced this pull request Sep 19, 2023
* Fix non-shared weights

* Add tests

* Edit tied weights keys
parambharat pushed a commit to parambharat/transformers that referenced this pull request Sep 26, 2023
* Fix non-shared weights

* Add tests

* Edit tied weights keys
blbadger pushed a commit to blbadger/transformers that referenced this pull request Nov 8, 2023
* Fix non-shared weights

* Add tests

* Edit tied weights keys
EduardoPach pushed a commit to EduardoPach/transformers that referenced this pull request Nov 18, 2023
* Fix non-shared weights

* Add tests

* Edit tied weights keys
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants