[AutoModel] Add AutoModelForTextEncoding #24305
Conversation
| @@ -1011,6 +1011,36 @@ | |||
| ] | |||
| ) | |||
|
|
|||
| MODEL_FOR_TEXT_ENCODING_MAPPING_NAMES = OrderedDict( | |||
There was a problem hiding this comment.
Is this list good for you @amyeroberts? It's the most popular encoder-only architectures, and enc-dec ones where we have the encoder model separated as an import
There was a problem hiding this comment.
Yep! We can always add as needed :)
| @@ -1011,6 +1011,36 @@ | |||
| ] | |||
| ) | |||
|
|
|||
| MODEL_FOR_TEXT_ENCODING_MAPPING_NAMES = OrderedDict( | |||
There was a problem hiding this comment.
Yep! We can always add as needed :)
| @@ -716,6 +716,13 @@ def __init__(self, *args, **kwargs): | |||
| requires_backends(self, ["torch"]) | |||
|
|
|||
|
|
|||
| class AutoModelForTextEncoding(metaclass=DummyObject): | |||
There was a problem hiding this comment.
There should be the equivalent TFAutoModelForTextEncoding in dummy_tf_objects.py
There was a problem hiding this comment.
Ah perfect! Thanks for flagging! Resolved in 8b003f7
| @@ -1084,6 +1084,7 @@ | |||
| "AutoModelForSequenceClassification", | |||
| "AutoModelForSpeechSeq2Seq", | |||
| "AutoModelForTableQuestionAnswering", | |||
| "AutoModelForTextEncoding", | |||
There was a problem hiding this comment.
The TF equivalent class should be in this init too
There was a problem hiding this comment.
Resolved in 8b003f7
|
The documentation is not available anymore as the PR was closed or merged. |
sanchit-gandhi
force-pushed
the
auto-text-encoder
branch
from
ebfc908 to
adc7782
Compare
What does this PR do?
Adds AutoModel for text encoding (used in the circumstance when you want to extract the text encoder from an encoder-decoder architecture).
This facilitates loading a t5 encoder from t5 enc-dec model weights (as is done in Music Gen in #24109)