Skip to content
This repository has been archived by the owner on Sep 24, 2024. It is now read-only.

Add support for llamafile model names #117

Open
aittalam opened this issue Aug 20, 2024 · 0 comments
Open

Add support for llamafile model names #117

aittalam opened this issue Aug 20, 2024 · 0 comments

Comments

@aittalam
Copy link
Member

Currently llamafiles are supported as generic OAI-compatible servers. While this works, it forces us to specify a valid (i.e. hf-transformers loadable) model name - which is needed for vLLM but might not be the case here as one generally uses GGUF files or llamafile filenames. While not strictly required, these names might be useful to e.g. properly identify the model when reading evaluation results.
Supporting this boils down to adding a llamafile:// prefix to handle llamafiles differently and changing the validation methodology for the provided model names.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant