Custom APIs

[raivisdejus]

This discussion thread is a place to collect custom API urls that can be used with Buzz. Enter API urls and keys in Help -> Settings.
API key has to be in ASCII encoding with no special language characters.

Speech recognition APIs

• Default OpenAI API (https://api.openai.com/v1) supports transcription of files up to 25MB in size. Price $0.006 / minute. More info
• https://api.lemonfox.ai/v1. Price $0.50 per 3 hours of speech. More info
• https://api.groq.com/openai/v1. Free transcription with quite generous limits. More into
• Customer API servers you can run yourself on your existing hardware or rented cloud servers from runpod.io, vast.ai, tensordock.com or other providers
  • tiny-openai-whisper-api is a minimalistic API
  • faster-whisper-server - Faster whisper API server

Translation APIs

Local translation with Ollama is also possible. For setup instructions see this.
Use this configuration

• OpenAI base url http://localhost:11434/v1
• OpenAI API key ollama

When enabling translation, specify model you have downloaded in Ollama from the list of models.

For translation with Groq.com

• OpenAI base url https://api.groq.com/openai/v1
• OpenAI API key, your Groq.com API key, that looks like gsk_AAbbCC......xxYYzz

When enabling translation, specify supported groq.com model from the list of models.

Please note that if API key validation fails transcription and translation with the API may still work, as some API may not support key validation.

[carlosgb55]

Thanks for the work you have done in this software!

I'm using whisper.cpp at medium to transcribe video files and output subtitles in English, but I intend in translate these subtitles to Portuguese, I create an openai api key just to discover that I have to pay to use. I'm trying to use ollama with llama2 model as it's been shown in https://ollama.com/blog/openai-compatibility.

I had openai api key in Preferences and on "OpenAI base url" I put http://localhost:11434/.

When I open the subtitle in buzz and click the translate button I type the model "llama2" and type the instructions as in the link https://chidiwilliams.github.io/buzz/docs/usage/translations, and nothing happens.

Any instructions on how to use buzz with ollama models?
I'm using buzz snap version 1.2.1 on ubuntu linux.

[raivisdejus]

@carlosgb55 Please use http://localhost:11434/v1 as the OpenAI base url.

If you translate previously transcribed recordings from the transcription viewer screen the first click on "Translate" button will open the transcription settings dialog where you can enter the instructions for the model. To start the translation you have to push the "Translate" button again.

Also you may consider using the groq.com if you do not have a lot of subtitles to translate their free trier may be enough for you

[raivisdejus]

The very latest development version will start translations when the settings window is closed.

[carlosgb55]

Thanks for the answer, I tried groq.com and the translation works, but I find a problem with the translations.

Buzz sends an instruction and a sentence from a subtitle, sending part of a sentence in one instruction and the other part in another instruction, make the result text and subtitle incoherent.

I used the web interface of groq and the translation was almost perfect, but the translation with buzz using the same instruction and the same AI model made the translation incoherent.

I guess, buzz should send the whole text or subtitle to the api and put the result in a file.

With my test on the groq interface I could even instruct the ai assistant to recognize that is a subtitle and translate each sentence with grammatically correctness and outputting the results with same time stamps. So, it translate as if it was a whole text, but put the output on the same timestamp as the original!

[raivisdejus]

Good idea. Created a task to track improvement of the translations at some later point #1026

[gugaucb]

When I tried to use the local Ollama to translate, I received this message: 404 page not found. Does Someone receive this error message?

I believe that the solution is in this issue:

ollama/ollama#5830

when I use the groq.com in MacOS, the app throws these errors:

---

Translated Report (Full Report Below)

Process: Buzz [47934]
Path: /Applications/Buzz.app/Contents/MacOS/Buzz
Identifier: com.chidiwilliams.buzz
Version: 1.1.0 (???)
Code Type: ARM-64 (Native)
Parent Process: launchd [1]
User ID: 501

Thread 0:: Dispatch queue: com.apple.main-thread
0 libsystem_kernel.dylib 0x19552de34 mach_msg2_trap + 8
1 libsystem_kernel.dylib 0x1955405d0 mach_msg2_internal + 80
2 libsystem_kernel.dylib 0x1955369d8 mach_msg_overwrite + 480
3 libsystem_kernel.dylib 0x19552e17c mach_msg + 24
4 CoreFoundation 0x195655edc __CFRunLoopServiceMachPort + 160
5 CoreFoundation 0x19565473c __CFRunLoopRun + 1212
6 CoreFoundation 0x195653bc4 CFRunLoopRunSpecific + 588
7 HIToolbox 0x1a0ac3f64 RunCurrentEventLoopInMode + 292
8 HIToolbox 0x1a0ac9d54 ReceiveNextEventCommon + 636
9 HIToolbox 0x1a0ac9eb8 _BlockUntilNextEventMatchingListInModeWithFilter + 76
10 AppKit 0x19917fa08 _DPSNextEvent + 660
11 AppKit 0x199abfe0c -[NSApplication(NSEventRouting) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 688
12 AppKit 0x199172ae0 -[NSApplication run] + 480
13 libqcocoa.dylib 0x17ba2bf50 0x17ba18000 + 81744
14 QtCore 0x12008a174 QEventLoop::exec(QFlagsQEventLoop::ProcessEventsFlag) + 540
15 QtCore 0x120080b14 QCoreApplication::exec() + 112
16 QtWidgets.abi3.so 0x1789d839c meth_QApplication_exec(_object*, _object*) + 96
17 Python 0x105fbbb98 cfunction_call + 96
18 Python 0x105f5733c _PyObject_MakeTpCall + 128
19 Python 0x10607f0bc _PyEval_EvalFrameDefault + 54032
20 Python 0x106070af8 PyEval_EvalCode + 276
21 Buzz 0x10210c538 0x102108000 + 17720
22 Buzz 0x10210cae0 0x102108000 + 19168
23 dyld 0x1951ec274 start + 2840

[.... Removed the rest ....]

[raivisdejus]

@gugaucb I get a feeling that maybe you did not specify model to use with Ollama. Let's test it step by step.

Let's say you have http://localhost:11434/v1 as OpenAI base url and ollama as OpenAi API key in Help->Preferences
First question is what happens when you click "Test" button for the API key? This will test if Buzz can talk to the AI server. It should work and say that the API key is ok.

Second test is to paste this in the terminal to send a test request to the Ollama

curl -D - http://localhost:11434/v1/chat/completions -d '{"model": "llama3:instruct", "messages": [{"role": "user", "content": "hello"}], "stream": false}'

Please note the llama3:instruct. This is the model I have downloaded on my system. You have to change it to a model that you have downloaded for the Ollama.

When you want to translate in the Buzz you also have to specify the model that is available in the Ollama. In my case I use llama3:instruct.

For Groq.com you have to specify one of their models from https://console.groq.com/docs/models