Support for using tiktoken tokenizers

[dakinggg]

This PR adds support for tiktoken tokenizers via a huggingface compatible wrapper. It is possible that there are pieces of the wrapped implementation that are not perfect, but the unit test coverage should cover the main scenarios that we use.

The tokenizer can be loaded in a yaml via

tokenizer:
    name:
    kwargs:
        model_name: gpt-4

• merge implementation with jeremy's previous PR

[dakinggg]

wait for #616 before merging this

[irenedea]

LGTM, pending minor comments + pyright.import_or_skip() is skipping the tests on CI.

feel free to address in a follow up PR

[abhi-mosaic]

Reviewed the tests carefully, and added some comments.

Also, do we need to add tiktoken as a dependency in setup.py?

[dakinggg]

@abhi-mosaic tiktoken is added as an optional dep group already