Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow Chatbot examples to show more than one image #10111

Merged
merged 28 commits into from
Dec 13, 2024
Merged

Allow Chatbot examples to show more than one image #10111

Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
b931817
change example image limit to 4
hannahblair Dec 3, 2024
e8d6684
add changeset
gradio-pr-bot Dec 3, 2024
44bf05e
refactor examples
hannahblair Dec 4, 2024
afbb7f2
redesign examples
hannahblair Dec 5, 2024
09defb5
Merge branch 'main' into example-images-chatbot
hannahblair Dec 5, 2024
64140ec
aria improvements
hannahblair Dec 5, 2024
96d0508
Merge branch 'example-images-chatbot' of github.com:gradio-app/gradio…
hannahblair Dec 5, 2024
7ff38e1
check for multimodal
hannahblair Dec 6, 2024
9245e68
use multimedia icons
hannahblair Dec 6, 2024
80f4db9
clean up
hannahblair Dec 6, 2024
519af65
remove min-height
hannahblair Dec 9, 2024
f858097
add param
hannahblair Dec 9, 2024
41570fa
fix test
hannahblair Dec 9, 2024
f4a57b4
Merge branch 'main' into example-images-chatbot
abidlabs Dec 10, 2024
0d60c67
Revert "add param"
hannahblair Dec 10, 2024
cbdf22a
Revert "fix test"
hannahblair Dec 10, 2024
b6f2be4
Merge branch 'example-images-chatbot' of github.com:gradio-app/gradio…
hannahblair Dec 10, 2024
4168f99
handle icon in chat_interface.py
hannahblair Dec 11, 2024
e6b9b9e
Merge branch 'main' into example-images-chatbot
hannahblair Dec 11, 2024
0ee87ad
Merge branch 'main' into example-images-chatbot
abidlabs Dec 11, 2024
3aec03b
remove unused css class
hannahblair Dec 11, 2024
64a50bd
Merge branch 'example-images-chatbot' of github.com:gradio-app/gradio…
hannahblair Dec 11, 2024
08a2336
fix types
hannahblair Dec 11, 2024
551ca95
format
hannahblair Dec 12, 2024
4616c85
fix test
hannahblair Dec 12, 2024
b4fd92e
- change logic to use example icon for text example if exists
hannahblair Dec 12, 2024
6c6fc4f
add stories
hannahblair Dec 12, 2024
2e22899
tweaks
hannahblair Dec 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/legal-aliens-burn.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
"@gradio/chatbot": minor
"gradio": minor
---

feat:Allow Chatbot examples to show more than one image
17 changes: 14 additions & 3 deletions gradio/chat_interface.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ def __init__(
self.provided_chatbot = chatbot is not None
self.examples = examples
self.examples_messages = self._setup_example_messages(
examples, example_labels, example_icons
examples, example_labels, example_icons, multimodal
hannahblair marked this conversation as resolved.
Show resolved Hide resolved
)
self.run_examples_on_click = run_examples_on_click
self.cache_examples = cache_examples
Expand Down Expand Up @@ -324,6 +324,7 @@ def _setup_example_messages(
examples: list[str] | list[MultimodalValue] | list[list] | None,
example_labels: list[str] | None = None,
example_icons: list[str] | None = None,
multimodal: bool = False,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
multimodal: bool = False,

) -> list[ExampleMessage]:
examples_messages = []
if examples:
Expand All @@ -338,8 +339,18 @@ def _setup_example_messages(
example_message["files"] = example.get("files", [])
if example_labels:
example_message["display_text"] = example_labels[index]
if example_icons:
example_message["icon"] = example_icons[index]
if example_icons and multimodal:
example_files = example_message.get("files")
if not example_files:
example_message["icon"] = {
"path": "",
"url": None,
"orig_name": None,
"mime_type": "text", # for internal use, not a valid mime type
Copy link
Collaborator Author

@hannahblair hannahblair Dec 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here's the backend implementation i'm using to indicate in the frontend whether the example is text only

"meta": {"_type": "gradio.FileData"},
}
else:
example_message["icon"] = example_icons[index]
hannahblair marked this conversation as resolved.
Show resolved Hide resolved
examples_messages.append(example_message)
return examples_messages

Expand Down
145 changes: 9 additions & 136 deletions js/chatbot/shared/ChatBot.svelte
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@
import { ShareError } from "@gradio/utils";
import { Gradio } from "@gradio/utils";

import Examples from "./Examples.svelte";
hannahblair marked this conversation as resolved.
Show resolved Hide resolved

export let value: NormalisedMessage[] | null = [];
let old_value: NormalisedMessage[] | null = null;

Expand Down Expand Up @@ -328,55 +330,13 @@
{/if}
</div>
{:else}
<div class="placeholder-content">
{#if placeholder !== null}
<div class="placeholder">
<Markdown message={placeholder} {latex_delimiters} {root} />
</div>
{/if}
{#if examples !== null}
<div class="examples">
{#each examples as example, i}
<button
class="example"
on:click={() => handle_example_select(i, example)}
>
{#if example.icon !== undefined}
<div class="example-icon-container">
<Image
class="example-icon"
src={example.icon.url}
alt="example-icon"
/>
</div>
{/if}
{#if example.display_text !== undefined}
<span class="example-display-text">{example.display_text}</span>
{:else}
<span class="example-text">{example.text}</span>
{/if}
{#if example.files !== undefined && example.files.length > 1}
<span class="example-file"
><em>{example.files.length} Files</em></span
>
{:else if example.files !== undefined && example.files[0] !== undefined && example.files[0].mime_type?.includes("image")}
<div class="example-image-container">
<Image
class="example-image"
src={example.files[0].url}
alt="example-image"
/>
</div>
{:else if example.files !== undefined && example.files[0] !== undefined}
<span class="example-file"
><em>{example.files[0].orig_name}</em></span
>
{/if}
</button>
{/each}
</div>
{/if}
</div>
<Examples
{examples}
{placeholder}
{latex_delimiters}
{root}
on:example_select={(e) => dispatch("example_select", e.detail)}
/>
{/if}
</div>

Expand All @@ -392,93 +352,6 @@
{/if}

<style>
.placeholder-content {
display: flex;
flex-direction: column;
height: 100%;
}

.placeholder {
align-items: center;
display: flex;
justify-content: center;
height: 100%;
flex-grow: 1;
}

.examples :global(img) {
pointer-events: none;
}

.examples {
margin: auto;
padding: var(--spacing-xxl);
display: grid;
grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
gap: var(--spacing-xxl);
max-width: calc(min(4 * 200px + 5 * var(--spacing-xxl), 100%));
}

.example {
display: flex;
flex-direction: column;
align-items: center;
padding: var(--spacing-xl);
border: 0.05px solid var(--border-color-primary);
border-radius: var(--radius-md);
background-color: var(--background-fill-secondary);
cursor: pointer;
transition: var(--button-transition);
max-width: var(--size-56);
width: 100%;
justify-content: center;
}

.example:hover {
background-color: var(--color-accent-soft);
border-color: var(--border-color-accent);
}

.example-icon-container {
display: flex;
align-self: flex-start;
margin-left: var(--spacing-md);
width: var(--size-6);
height: var(--size-6);
}

.example-display-text,
.example-text,
.example-file {
font-size: var(--text-md);
width: 100%;
text-align: center;
overflow: hidden;
text-overflow: ellipsis;
}

.example-display-text,
.example-file {
margin-top: var(--spacing-md);
}

.example-image-container {
flex-grow: 1;
display: flex;
justify-content: center;
align-items: center;
margin-top: var(--spacing-xl);
}

.example-image-container :global(img) {
max-height: 100%;
max-width: 100%;
height: var(--size-32);
width: 100%;
object-fit: cover;
border-radius: var(--radius-xl);
}

.panel-wrap {
width: 100%;
overflow-y: auto;
Expand Down
Loading
Loading