Skip to content

Make TiktokenTokenizerWrapper compatible with convert_composer_to_hf.py#730

Merged
irenedea merged 7 commits intomosaicml:mainfrom irenedea:tiktoken-hf-checkpointsNov 13, 2023
+45-8

Commits

Commits on Nov 10, 2023

Commits on Nov 13, 2023