Add OpenAICompatibleChatExperiment class for enhanced LLM compatibility #125
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I have introduced a new class called OpenAICompatibleChatExperiment. This class aims to enhance the compatibility of prompttools with a wider range of large language models (LLMs) that support OpenAI-compatible inference endpoints. This enables usage of popular open source models hosted on Model as API service providers like together.ai, and self hosted models via Huggingface Text Generation Inference/ vLLM/ ollama etc, as they provide openai compatible rest api.
Key features and benefits of the OpenAICompatibleChatExperiment class:
Flexibility: The class allows users to specify the base URL and API key for any OpenAI-compatible inference endpoint, enabling seamless integration with various LLMs.
Ease of use: The class maintains a similar interface to the existing OpenAIChatExperiment class, ensuring a smooth transition for users familiar with prompttools.
Expanded LLM support: By leveraging OpenAI-compatible endpoints, prompttools can now interface with a broader ecosystem of LLMs, empowering users to explore and utilize different models effortlessly.