CPU optimizaton for Whisper.Net

[shishirsasal8]

I am using whisper.net in my Windows application. This application is for people working from home with low laptop configuration. Currently, whisper.net is taking 50% CPU when it is processing speech to text, is there a way I can optimize the CPU usage?

Thanks in Advance, please allow me to give more details if required,

[sandrohanea]

Hello @shishirsasal8 ,

You can either use lower models (e.g. Tiny, etc). but the quality will degrate or you can use less threads (see method WithThreads on the WhisperProcessorBuilder) => if using less threads, it will result in longer processing times, but the CPU usage will be lower.

Regarding optimizations, they are done on whisper.cpp (a lot of them, based on the runtime and the available resources).

You can also check vulkan and openvino builds which might use the integrated graphics for various processors on low laptop configurations (but you'll need to test individually).