Fix packing script w Tiktokenizer on Mac and Windows

[samhavens]

We recently added tiktokenizer support, and it turns out this broke support for the packing script on Mac (and likely Windows, though that is untested). Why is explained in the comments.

I tested this change and it works on Mac and does not break Linux. I have not tested on Windows.

[dakinggg]

cc @irenedea since you're about to change all of this

[irenedea]

@dakinggg @samhavens I think the proper solution here would be to make TiktokenTokenizerWrapper picklable.
#700 What do you think of this? If it looks good to you, I think this would be a more robust solution.

[samhavens]

If the only reason it needs to be pickleable is because of the way multiprocessing works on Mac+Windows vs Linux, then I could go either way on which solution I prefer. But if there are any other cases where the tokenizer being pickleable is useful (storing state in some way) then I agree with you. Also, neither seems particularly harder and your PR is also ready, so I will close this PR.

[irenedea]

@samhavens Sounds good! Multiprocessing is the main case where pickling is required (particularly in dataloader), but even so we'd need to handle this at every entry point in our repo not just in packing. Plus, HF tokenizers are all picklable, so I'd rather not break this assumption.