Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate SpeechToText with ChatInterface #7021

Open
awesomebytes opened this issue Jul 26, 2024 · 1 comment
Open

Integrate SpeechToText with ChatInterface #7021

awesomebytes opened this issue Jul 26, 2024 · 1 comment

Comments

@awesomebytes
Copy link

Is your feature request related to a problem? Please describe.

I am working on a chatbot interface and a typical user request is to be able to just talk instead of typing. Holoviz has the SpeechToText widget, but it is currently not-so-easy to integrate into ChatInterface. (Or at least, when I've tried, I've had to do plenty of workarounds, I may be doing something wrong).

Using SpeechToText has the advantage of being free (mostly), and is already integrated in the browser.

Bonus: Reading out loud text would be interesting too given there is a TextToSpeech widget too. Maybe a little button next to the chat messages.

Describe the solution you'd like

I would like to have, next to the text input a microphone icon to give speech input, and that the recognized text would appear in the text input (extra cool if the recognized text appears as it is being transcribed? I think that depends on the browser, if it is even supported). Autosend should be optional.

Describe alternatives you've considered

Using third party services (marvin was mentioned in this thread) can be used, however, this implies that by default you must be running the app in the same computer with the microphone.

Otherwise, recording with JS browser APIs and sending it to some service (be it local or cloud) would be a good alternative too. The examples found in this thread seem good: https://stackoverflow.com/questions/16413063/html5-record-audio-to-file However, I'm not versed enough with JS, and new to Panel, to be able to do this myself at the moment.

Additional context

My workarounds make it look like this, which is more or less what I would expect:

Screenshot 2024-07-26 205642

@MarcSkovMadsen
Copy link
Collaborator

There is a very basic example of a AudioInput widget in #7035 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants