Skip to content

Commit

Permalink
Tweaks to Chat Templates docs (#26168)
Browse files Browse the repository at this point in the history
* Put tokenizer methods in the right alphabetical order in the docs

* Quick tweak to ConversationalPipeline

* Typo fixes in the developer doc

* make fixup
  • Loading branch information
Rocketknight1 authored Sep 15, 2023
1 parent d70fab8 commit 2518e36
Show file tree
Hide file tree
Showing 3 changed files with 7 additions and 5 deletions.
4 changes: 2 additions & 2 deletions docs/source/en/chat_templating.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ once you set the correct chat template, your model will automatically become com

Before the introduction of chat templates, chat handling was hardcoded at the model class level. For backwards
compatibility, we have retained this class-specific handling as default templates, also set at the class level. If a
model does not have a chat template set, but there is a default template for its model class, the `ConversationPipeline`
model does not have a chat template set, but there is a default template for its model class, the `ConversationalPipeline`
class and methods like `apply_chat_template` will use the class template instead. You can find out what the default
template for your tokenizer is by checking the `tokenizer.default_chat_template` attribute.

Expand Down Expand Up @@ -248,7 +248,7 @@ with an empty chat template, or one that's still using the default class templat
the model repository so that this attribute can be set properly!

Once the attribute is set, that's it, you're done! `tokenizer.apply_chat_template` will now work correctly for that
model, which means it is also automatically supported in places like `ConversationPipeline`!
model, which means it is also automatically supported in places like `ConversationalPipeline`!

By ensuring that models have this attribute, we can make sure that the whole community gets to use the full power of
open-source models. Formatting mismatches have been haunting the field and silently harming performance for too long -
Expand Down
4 changes: 2 additions & 2 deletions docs/source/en/main_classes/tokenizer.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,10 @@ to a given token).

[[autodoc]] PreTrainedTokenizer
- __call__
- apply_chat_template
- batch_decode
- decode
- encode
- apply_chat_template
- push_to_hub
- all

Expand All @@ -69,10 +69,10 @@ loaded very simply into 🤗 transformers. Take a look at the [Using tokenizers

[[autodoc]] PreTrainedTokenizerFast
- __call__
- apply_chat_template
- batch_decode
- decode
- encode
- apply_chat_template
- push_to_hub
- all

Expand Down
4 changes: 3 additions & 1 deletion src/transformers/pipelines/conversational.py
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,9 @@ def _forward(self, model_inputs, minimum_tokens=10, **generate_kwargs):

n = model_inputs["input_ids"].shape[1]
if max_length - minimum_tokens < n:
logger.warning(f"Conversation input is to long ({n}), trimming it to ({max_length} - {minimum_tokens})")
logger.warning(
f"Conversation input is too long ({n}), trimming it to {max_length - minimum_tokens} tokens. Consider increasing `max_length` to avoid truncation."
)
trim = max_length - minimum_tokens
model_inputs["input_ids"] = model_inputs["input_ids"][:, -trim:]
if "attention_mask" in model_inputs:
Expand Down

0 comments on commit 2518e36

Please sign in to comment.