gr.load_chat: Allow loading any openai-compatible server immediately as a ChatInterface

[aliabid94]

gr.load_chat allows loading any openai chatbot immediately as a ChatInterface easily.

gr.load_chat("http://localhost:11434/v1/", "llama3.2").launch()

Can be customized as such:

import gradio as gr
gr.ChatInterface(
  gr.load_chat("http://localhost:11434/v1/", "llama3.2"),
  title="Etc"
).launch()

I think this will be an especially popular pairing with Ollama.

In a follow up PR, will add support for media input and responses.

[gradio-pr-bot]

🪼 branch checks and previews

•	Name	Status	URL
	Spaces	ready!	Spaces preview
	Website	ready!	Website preview
🦄	Changes	detected!	Details

---

Install Gradio from this PR

pip install https://gradio-pypi-previews.s3.amazonaws.com/0a1285dd8d60705f86ffe137f3d0a930f3e6b986/gradio-5.9.1-py3-none-any.whl

Install Gradio Python Client from this PR

pip install "gradio-client @ git+https://github.com/gradio-app/gradio@0a1285dd8d60705f86ffe137f3d0a930f3e6b986#subdirectory=client/python"

Install Gradio JS Client from this PR

npm install https://gradio-npm-previews.s3.amazonaws.com/0a1285dd8d60705f86ffe137f3d0a930f3e6b986/gradio-client-1.8.0.tgz

Use Lite from this PR

<script type="module" src="https://gradio-lite-previews.s3.amazonaws.com/0a1285dd8d60705f86ffe137f3d0a930f3e6b986/dist/lite.js""></script>

[gradio-pr-bot]

🦄 change detected

This Pull Request includes changes to the following packages.

Package	Version
gradio	minor

• Maintainers can select this checkbox to manually select packages to update.

With the following changelog entry.

gr.load_chat: Allow loading any openai-compatible server immediately as a ChatInterface

Maintainers or the PR author can modify the PR title to modify this entry.

Something isn't right?

• Maintainers can change the version label to modify the version bump.
• If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.

[abidlabs]

Very cool @aliabid94! I find this part kind of confusing though:

Can be customized as such:

import gradio as gr
gr.ChatInterface(
  gr.load_chat("http://localhost:11434/v1/", "llama3.2"),
  title="Etc"
).launch()

So gr.load_chat produces a ChatInterface, but now you can pass a ChatInterface as the fn parameter into a new ChatInterface? That's not intuitive and the DX is not optimal since many of the parameters in gr.ChatInterface (e.g. additional_inputs) don't make sense if you are loading an openai server.

What about if we did the following instead?

gr.ChatInterface.load(server_url). To let users customize, we could add title and whichever parameters actually make sense for loading an openai server to this factory method gr.ChatInterface.load() instead.

[aliabid94]

What about if we did the following instead?

gr.ChatInterface.load(server_url). To let users customize, we could add title and whichever parameters actually make sense for loading an openai server to this factory method gr.ChatInterface.load() instead.

Well we can't use load, that's the event listener for when the demo loads. So we could do something like gr.ChatInterface.load_from(server_url), but that feels too long. I also don't really like the idea of copying all the kwargs and docstrings over again from the constructor to this load_from method, that would be like 40 lines of code duplication.

gr.load_chat mirrors gr.load. Using gr.load as a function for a customized Interface is also the current way to customize a gr.Interface, e.g.

demo = gr.Interface(fn=gr.load("model/meta-ai/llama3.2"), title="My App", examples=["a", "b"])

mirrors

demo = gr.ChatInterface(fn=gr.load_chat("http://chat.com/v1"), title="My App", examples=["a", "b"])

So I think the existing API makes the most sense.

[abidlabs]

I see thanks for explaining 👍

[abidlabs]

lgtm works great @aliabid94! Can we add some documentation around this at the top of https://www.gradio.app/guides/creating-a-chatbot-fast?

[abidlabs]

Suggested change

	gr.load_chat("http://localhost:11434/v1/", "llama3.2").launch()
	gr.load_chat("http://localhost:11434/v1/", model="llama3.2", token="ollama").launch()