Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chat Interface flagging and chatbot feedback #10272

Merged
merged 21 commits into from
Jan 3, 2025
Merged

Chat Interface flagging and chatbot feedback #10272

merged 21 commits into from
Jan 3, 2025

Conversation

aliabid94
Copy link
Collaborator

@aliabid94 aliabid94 commented Jan 1, 2025

Allows chatbot and chatinterface to provide additional feedback. In chat interface, feedback is flagged to a CSV file.

Recording 2024-12-31 at 16 44 47

@gradio-pr-bot
Copy link
Collaborator

gradio-pr-bot commented Jan 1, 2025

🪼 branch checks and previews

Name Status URL
Spaces ready! Spaces preview
Website ready! Website preview
Storybook ready! Storybook preview
🦄 Changes detected! Details

Install Gradio from this PR

pip install https://gradio-pypi-previews.s3.amazonaws.com/936988dbbfd3cb1431ecd59fd7061e0217fe66f1/gradio-5.9.1-py3-none-any.whl

Install Gradio Python Client from this PR

pip install "gradio-client @ git+https://github.com/gradio-app/gradio@936988dbbfd3cb1431ecd59fd7061e0217fe66f1#subdirectory=client/python"

Install Gradio JS Client from this PR

npm install https://gradio-npm-previews.s3.amazonaws.com/936988dbbfd3cb1431ecd59fd7061e0217fe66f1/gradio-client-1.8.0.tgz

Use Lite from this PR

<script type="module" src="https://gradio-lite-previews.s3.amazonaws.com/936988dbbfd3cb1431ecd59fd7061e0217fe66f1/dist/lite.js""></script>

@gradio-pr-bot
Copy link
Collaborator

gradio-pr-bot commented Jan 1, 2025

🦄 change detected

This Pull Request includes changes to the following packages.

Package Version
@gradio/atoms minor
@gradio/chatbot minor
@gradio/utils minor
gradio minor
  • Maintainers can select this checkbox to manually select packages to update.

With the following changelog entry.

Chat Interface flagging and chatbot feedback

Maintainers or the PR author can modify the PR title to modify this entry.

Something isn't right?

  • Maintainers can change the version label to modify the version bump.
  • If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.

gradio/flagging.py Outdated Show resolved Hide resolved
gradio/chat_interface.py Outdated Show resolved Hide resolved
gradio/chat_interface.py Outdated Show resolved Hide resolved
gradio/chat_interface.py Outdated Show resolved Hide resolved
gradio/chat_interface.py Outdated Show resolved Hide resolved
@abidlabs abidlabs requested a review from pngwn January 1, 2025 01:29
@abidlabs
Copy link
Member

abidlabs commented Jan 1, 2025

Nice implementation @aliabid94! I tested it and everything looks good aside from one minor thing: the index gets flagged as a tuple [index, index] instead of a single element index. I assume this is not intentional?

For example, after flagging a couple of messages, here's what my csv looked like:

conversation,index,value,flag,timestamp
"[{""role"": ""user"", ""metadata"": {""title"": null}, ""content"": ""abc"", ""options"": null}, {""role"": ""assistant"", ""metadata"": {""title"": null}, ""content"": ""Yes"", ""options"": null}]","[1, 1]",Like,2024-12-31 20:35:57.236605
"[{""role"": ""user"", ""metadata"": {""title"": null}, ""content"": ""abc"", ""options"": null}, {""role"": ""assistant"", ""metadata"": {""title"": null}, ""content"": ""Yes"", ""options"": null}, {""role"": ""user"", ""metadata"": {""title"": null}, ""content"": ""def"", ""options"": null}, {""role"": ""assistant"", ""metadata"": {""title"": null}, ""content"": ""No"", ""options"": null}]","[3, 3]",incorrect,2024-12-31 20:36:00.349117
"[{""role"": ""user"", ""metadata"": {""title"": null}, ""content"": ""abc"", ""options"": null}, {""role"": ""assistant"", ""metadata"": {""title"": null}, ""content"": ""Yes"", ""options"": null}, {""role"": ""user"", ""metadata"": {""title"": null}, ""content"": ""def"", ""options"": null}, {""role"": ""assistant"", ""metadata"": {""title"": null}, ""content"": ""No"", ""options"": null}]","[3, 3]",other,2024-12-31 20:36:58.068836

Notice the [1, 1] and [3, 3]

@abidlabs
Copy link
Member

abidlabs commented Jan 1, 2025

Oh and we should create a demo showcasing this & include this in the ChatInterface docs!

Copy link
Member

@abidlabs abidlabs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm great PR @aliabid94! I'll go ahead and merge this in once CI is green to avoid conflicts with #10191

@abidlabs abidlabs enabled auto-merge (squash) January 3, 2025 13:57
@abidlabs abidlabs merged commit a1f2649 into main Jan 3, 2025
23 checks passed
@abidlabs abidlabs deleted the chatbot_feedback branch January 3, 2025 14:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants