TaSTT, a free self-hosted STT for VRChat

[yum-food]

TaSTT is a free and open source speech-to-text for VRChat. It uses OpenAI's whisper to transcribe speech to text, and streams it into the game via OSC.

Notably, all language services are performed on the client. This allows the service to be free, reliable, and very fast. End-to-end latency (with reasonable settings, hardware, and using the custom chatbox) is typically under 0.5 seconds.

I'm using faster-whisper, a compute-optimized implementation of Whisper. Most VR users are able to handle using this library with the base.en model.

It has a bunch of features:

• Streaming to in-game chatbox
• Streaming to customizable chatbox
• Streaming to OBS browser source
• Speak in any of 100 languages
• Translate into any of 200 languages
• Text filters (upper/lowercase, profanity, uwu, etc)

I've been working on this as a passion project for almost a year. It started as a "weekend project" and grew into the largest software project I've ever made. There are about 170 users now.

Going forward, there are lots of things on the agenda:

• Improve appearance (in progress)
• Rewrite desktop UI
• Implement overlay
• Adopt svoice or similar to do transcription of others' audio (accessibility tool)
• Adopt coqui or similar to do speech synthesis

... so the point is, this is an active project with a future.

Links:

• https://github.com/yum-food/TaSTT
• https://youtu.be/SqOGTnKXgag?si=4VXEcPRdSY1SBwIh

Cheers!
yum

PS: Please add an OSC endpoint to explicitly dismiss the built-in chatbox, it would help improve the UX for mute players who use speech-to-text tools :)