Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Open WebUI as a Unified LLM Provider #37

Open
JBGruber opened this issue Feb 6, 2025 · 0 comments
Open

Support Open WebUI as a Unified LLM Provider #37

JBGruber opened this issue Feb 6, 2025 · 0 comments

Comments

@JBGruber
Copy link
Owner

JBGruber commented Feb 6, 2025

We only need to make a few changes to support Open WebUI. I already made the bare minimum changes in https://github.com/JBGruber/rollama/tree/open-webui. The authentication can already be attached as described in #7.

The main advatages are:

  • user authentication is possible, which allows for a central Ollama server, e.g., in a research lab. Everyone who has been granted access can then query models through their own account
  • we can compare the models available through Ollama to proprietary ones

This is the current syntax, which could use some improvement, but works:

options(rollama_headers = list(Authorization = "Bearer sk-6b82a5443e7145d4a779922592701ee3"))
query(q = "Why is the sky blue?", 
            model = "gpt-4o", 
            server = "http://localhost:3000", 
            endpoint = "/api/chat/completions")
#>                      
#>  ── Answer from gpt-4o-2024-08-06 ───────────────────────────────────────────────────────────────
#>  The sky appears blue due to a phenomenon called Rayleigh scattering. When sunlight enters Earth's
#>  atmosphere, it is made up of many different colors, each with different wavelengths. Blue light has a
#>  shorter wavelength and is scattered in all directions by the gases and particles in the atmosphere more
#>  than other colors with longer wavelengths, like red or yellow.
#>  
#>  As sunlight passes through the atmosphere, the blue light is scattered more widely across the sky and is
#>  more likely to be seen from any given point. This widespread scattering of blue light makes the sky
#>  appear predominantly blue to our eyes during the day. Near sunrise and sunset, the sky can take on red
#>  or orange hues because the sun is lower on the horizon, and its light passes through more of the
#>  atmosphere, scattering shorter wavelengths out of the direct line of sight and allowing longer
#>  wavelengths like red and orange to dominate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant