feat(forge/llm): Add `LlamafileProvider`#7091

Merged

Pwuts merged 51 commits intoSignificant-Gravitas:masterfrom Mozilla-Ocho:draft-llamafile-support

Jul 17, 2024

+680-10

Commits on Apr 18, 2024

Add minimal implementation of LlamafileProvider, a new ChatModelProvider for llamafiles. Currently it just extends OpenAIProvider and only overrides methods that are necessary to get the system to …
k8si
committed
Adapt model prompt message roles to be compatible with the Mistral-7b-Instruct chat template, which supports the 'user' & 'assistant' roles but does not support the 'system' role.
k8si
committed
In OpenAIProvider, change methods count_message_tokens, count_tokens, and get_tokenizer from classmethods so I can override them in LlamafileProvide (and so I can access instance instance a…
k8si
committed
misc cleanup
k8si
committed
add README for llamafile integration including setup instruction + notes on the integration; add helper scripts for downloading/running a llamafile + example env file.
k8si
committed

Commits on Apr 19, 2024

Commits on May 25, 2024

Commits on May 30, 2024

Merge branch 'master' into draft-llamafile-support
Pwuts
committed

Commits on May 31, 2024

Commits on Jun 2, 2024

Merge branch 'master' into draft-llamafile-support
Pwuts
committed

Commits on Jun 20, 2024

Commits on Jul 4, 2024

Commits on Jul 17, 2024