Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use custom cache dir for tokenizer download, too #41

Merged
merged 2 commits into from
Nov 7, 2024

Conversation

erickpeirson
Copy link
Contributor

Presently, passing cache_dir: Path to WordLlama.load() has no impact on the cache directory where tokenizer assets are stored. This makes it impossible to use WordLlama in an environment where the default cache path (the user's home directory) is not writable, which is often the case in production scenarios.

This PR does two things:

  • Modifies the meaning of cache_dir parameter on the WordLlama.load() method to be the cache root directory, within which the tokenizers and weights subdirectories are created;
  • Ensures that the cache_dir is passed to check_and_download_tokenizer and used, so that all writes occur within a configurable cache directory;

Note that this will effectively bust the cache on upgrade. But I'm hoping that's a small price to pay for the fix.

@dleemiller
Copy link
Owner

Nice - definitely a necessary change for deploying to places like lambda functions. Thanks!

@dleemiller dleemiller merged commit d8810b8 into dleemiller:main Nov 7, 2024
3 checks passed
@dleemiller
Copy link
Owner

#42

I have decided to clean everything up and simplify the API by removing the weights_dir as well. That feels legacy and over-complicated to me now to have both keyword arguments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants