Skip to content

Commit

Permalink
fix pyright
Browse files Browse the repository at this point in the history
  • Loading branch information
irenedea authored Oct 27, 2023
1 parent e994fed commit 5e688e7
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion llmfoundry/tokenizers/tiktoken.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ def pickle_Encoding(enc: tiktoken.Encoding):
mergeable_ranks=enc._mergeable_ranks,
special_tokens=enc._special_tokens), ())

copyreg.pickle(tiktoken.core.Encoding, pickle_Encoding)
copyreg.pickle(tiktoken.Encoding, pickle_Encoding)

if model_name is not None and encoding_name is not None:
raise ValueError(
Expand Down

0 comments on commit 5e688e7

Please sign in to comment.