Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for self hosted LLM's #2

Open
RustyKobie opened this issue Jan 31, 2024 · 2 comments
Open

Support for self hosted LLM's #2

RustyKobie opened this issue Jan 31, 2024 · 2 comments

Comments

@RustyKobie
Copy link

RustyKobie commented Jan 31, 2024

Problem Statement:

At present, Wordflow lacks the capability to utilize self-hosted Language Models (LLMs), limiting users' flexibility in choosing or swapping models according to their specific needs or preferences.

Proposed Solution:

Integrating support for self-hosted LLMs within Wordflow would greatly enhance its functionality and utility. This enhancement would empower users to leverage a diverse range of models, adapt to evolving requirements, and potentially improve performance by selecting models tailored to specific prompt tasks.

Benefits:

Flexibility: Users can seamlessly switch between different models or select a specific model based on task requirements.
Customization: Allows users to fine-tune their experience by integrating custom or specialized models.
Scalability: Accommodates the rapid influx of new models, ensuring users can readily integrate them into their workflow.
Performance Optimization: Users can choose models optimized for specific tasks, potentially improving overall performance and accuracy.

Easy win would be to integrate with something like ollama, which exposes a standard api for a whole range of different models:
https://github.com/ollama/ollama

@xiaohk
Copy link
Member

xiaohk commented Feb 2, 2024

Good idea!! I will add this feature when I have time. PR is welcome! We can add a new option in the model drop-down menu and allow users to enter a URL (e.g., localhost:8080) to call local/self-hosted ollama.

image

@fishfree
Copy link

+1 pls

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants