You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Sep 24, 2024. It is now read-only.
Currently llamafiles are supported as generic OAI-compatible servers. While this works, it forces us to specify a valid (i.e. hf-transformers loadable) model name - which is needed for vLLM but might not be the case here as one generally uses GGUF files or llamafile filenames. While not strictly required, these names might be useful to e.g. properly identify the model when reading evaluation results.
Supporting this boils down to adding a llamafile:// prefix to handle llamafiles differently and changing the validation methodology for the provided model names.
The text was updated successfully, but these errors were encountered:
Currently llamafiles are supported as generic OAI-compatible servers. While this works, it forces us to specify a valid (i.e. hf-transformers loadable) model name - which is needed for vLLM but might not be the case here as one generally uses GGUF files or llamafile filenames. While not strictly required, these names might be useful to e.g. properly identify the model when reading evaluation results.
Supporting this boils down to adding a
llamafile://
prefix to handle llamafiles differently and changing the validation methodology for the provided model names.The text was updated successfully, but these errors were encountered: