Unity support

[nadir500]

[reuben]

Can you expand on what your needs are? We have .NET support and are working to add .NET Core support as well soon. Would that be enough?

[nadir500]

It would be very helpful to have a TTS running on Unity for voice commands and recognizing words the same way windows speech recognition works. Basically with my case you have an array of words already defined and the voice recognition just works with simple listening to mic functions but it's exclusive to windows machines unfortunately. if there's already .NET support then doing cross platform and being recognized on Unity .NET-4.x/.NET 2.0-Standard would be really great. Unity doesn't natively recognize all libraries and do cross platform when exporting as plugins/packages so it needs to be tested on different platforms first.

[reuben]

For TTS you want https://github.com/coqui-ai/TTS, but Unity support will be trickier there as it requires Python to run. I'll ping you in this issue as soon as we have the .NET CI up and running so you can test it and see if it works in Unity.

[nadir500]

Much appreciated m8. Also Apologies for the typo, I was referring to STT not TTS. Looking forward for the outcome of the project.

[botaurus]

@reuben hello I'm also trying to get STT working in Unity. I built the native_client/dotnet. I put all the "native_client\dotnet\STTConsole\bin\x64\Release" dlls and files into my Unity project (not sure if I need other files from the dotnet bin folders). The DLL's seem to be recongized.
My issue is that the libkenlm.so and libstt.so aren't being recognized. I tried .so files from the libstt.tflite.Windows.zip release, tried placing them in various folders in my Unity project but get "DllNotFoundException: libstt.so assembly: type: member:(null)"
Any ideas? Thank you

[botaurus]

I renamed the .so to .dll and it worked :/

[giorgosgs]

The TTS and STT projects are very impressive.
If these could be run and be used commercially in Unity Games with an easy way of implementing like a Unity Plugin and run in multiple platforms like consoles, desktops and mobile, then this project would upgrade to a whole new level!

[FrontierDK]

I wholeheartedly agree - which is why I don't get why development has stopped on the STT part. There is a new demand for easy voice-assisted use of web sites and the web sites who will us it, will have a great advantage over those that don't. And since cloud-hosted solutions (even Google etc.) isn't a viable solution for economical reasons, this project is king. I don't get why some say "use Whisper", since Coqui STT supports just about any language - I can't see anywhere that Whisper does.

[HarikalarKutusu]

Maybe this: Whisper is Transformers-based, and training and/or fine-tuning requires a higher level of hardware for the same task. It is also a new stack for us developers. It took about a month for me to get hold of these.
On the other hand, I can get good improvement wrt WER when I finetune the whisper multi-lingual model - that is if I use languages with Latin scripts - it performs badly for non-latin.