Offline/on-device speech recognition

[szewai]

There is privacy concern that audio data captured in SpeechRecognition API is sent to third-party servers for processing (a similar issue in SpeechSynthesis API: #98). Actually not all browsers will send the data to third-party servers, but there is no way for developer to tell with existing API. Developers may check the implementation of different browsers and enable the feature on browsers with offline support, but this solution is not sustainable as browsers may add support for offline mode later, and browsers may switch between modes based on other conditions (networking, language, etc).

To address this concern, we should add a new interface to indicate online/offline mode for recognition. We might also let developer set the mode, as browsers may support different modes.

[tomuench]

I have dropped a paper on the conference "the web conf 2023". Hopefully, we can discuss this in a greater group of people.

If we would extend our speech standard, it is a great opportunity to build web apps with speech recognition, that are ready for privacy policies and offline usage. As a developer we want to configure, if we use public, private or even services on the device. This local services could be compiled to WebAssembly and connected to the API over JavaScript.

[evanbliu]

Chrome is supportive of adding this proposed functionality to the Web Speech API. I'd like to propose adding two attributes to the SpeechRecognition interface:

localService attribute, of type [boolean]
Controls whether speech recognition happens on-device. When set to true, speech recognition may happen on-device. When set to false, speech recognition will not happen on-device. The default value is true.

allowCloudFallback attribute, of type [boolean]
Controls whether a server-based speech recognition engine may be used if on-device speech recognition is not available. When set to false, speech recognition will throw an exception if on-device speech recognition is not supported. The default value is true.