feat:Add rest call support similar to oepn-api style

[wuhongsheng]

No description provided.

[andimarafioti]

You're such a goat! Thank you 🙏
With which base_url were you testing this? for me to reproduce

[wuhongsheng]

You're such a goat! Thank you 🙏 With which base_url were you testing this? for me to reproduce

I tested it with deepseek url

from openai import OpenAI

client = OpenAI(api_key="", base_url="https://api.deepseek.com")

response = client.chat.completions.create(
model="deepseek-chat",
messages=[
{"role": "system", "content": "You are a helpful assistant"},
{"role": "user", "content": "Hello"},
],
stream=False
)

print(response.choices[0].message.content)

[SinlinLi]

a tip: this commit doesn't implement stream transmission.

[MattiPaivike]

Yes, for this to be useful it should stream the text from the API and create TTS for each chunked text stream. Too much lag if you have to wait for generation of the entire message.

There is an example of how streaming can be implemented in OpenAI Python library:
https://github.com/openai/openai-python?tab=readme-ov-file#streaming-responses

[eustlb]

Great work, thanks @wuhongsheng ! 🙏🤗
This will definitely meet a relevant use case.
Almost ready to merge, it just needs to handle the --open_api_stream flag. The suggestion given below to handle it has been tested using --whisper_mlx and --melo, works fine.

[wuhongsheng]

add sent_tokenize

[eustlb]

Hey @wuhongsheng, thanks for iterating. Nevertheless, you have not committed suggested changes to handle the --open_api_stream flag, can you do so ? After that, we'll be able to merge 👌

[eustlb]

LGTM! Thanks for the good work 🤗