Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tweaks to Chat Templates docs #26168

Merged
merged 4 commits into from
Sep 15, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/source/en/chat_templating.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ once you set the correct chat template, your model will automatically become com

Before the introduction of chat templates, chat handling was hardcoded at the model class level. For backwards
compatibility, we have retained this class-specific handling as default templates, also set at the class level. If a
model does not have a chat template set, but there is a default template for its model class, the `ConversationPipeline`
model does not have a chat template set, but there is a default template for its model class, the `ConversationalPipeline`
class and methods like `apply_chat_template` will use the class template instead. You can find out what the default
template for your tokenizer is by checking the `tokenizer.default_chat_template` attribute.

Expand Down Expand Up @@ -248,7 +248,7 @@ with an empty chat template, or one that's still using the default class templat
the model repository so that this attribute can be set properly!

Once the attribute is set, that's it, you're done! `tokenizer.apply_chat_template` will now work correctly for that
model, which means it is also automatically supported in places like `ConversationPipeline`!
model, which means it is also automatically supported in places like `ConversationalPipeline`!

By ensuring that models have this attribute, we can make sure that the whole community gets to use the full power of
open-source models. Formatting mismatches have been haunting the field and silently harming performance for too long -
Expand Down
4 changes: 2 additions & 2 deletions docs/source/en/main_classes/tokenizer.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,10 @@ to a given token).

[[autodoc]] PreTrainedTokenizer
- __call__
- apply_chat_template
- batch_decode
- decode
- encode
- apply_chat_template
- push_to_hub
- all

Expand All @@ -69,10 +69,10 @@ loaded very simply into 🤗 transformers. Take a look at the [Using tokenizers

[[autodoc]] PreTrainedTokenizerFast
- __call__
- apply_chat_template
- batch_decode
- decode
- encode
- apply_chat_template
- push_to_hub
- all

Expand Down
4 changes: 3 additions & 1 deletion src/transformers/pipelines/conversational.py
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,9 @@ def _forward(self, model_inputs, minimum_tokens=10, **generate_kwargs):

n = model_inputs["input_ids"].shape[1]
if max_length - minimum_tokens < n:
logger.warning(f"Conversation input is to long ({n}), trimming it to ({max_length} - {minimum_tokens})")
logger.warning(
f"Conversation input is too long ({n}), trimming it to {max_length - minimum_tokens} tokens. Consider increasing `max_length` to avoid truncation."
)
trim = max_length - minimum_tokens
model_inputs["input_ids"] = model_inputs["input_ids"][:, -trim:]
if "attention_mask" in model_inputs:
Expand Down