Skip to content

Commit

Permalink
fix style
Browse files Browse the repository at this point in the history
  • Loading branch information
leandro committed Aug 4, 2022
1 parent 2bf870d commit ef3b87b
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion chapters/zh-CN/chapter6/8.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -403,7 +403,9 @@ tokenizer.decode(encoding.ids)
from transformers import PreTrainedTokenizerFast

wrapped_tokenizer = PreTrainedTokenizerFast(
tokenizer_object=tokenizer, bos_token="<|endoftext|>", eos_token="<|endoftext|>",
tokenizer_object=tokenizer,
bos_token="<|endoftext|>",
eos_token="<|endoftext|>",
)
```

Expand Down

0 comments on commit ef3b87b

Please sign in to comment.