Sending a message to the server and receiving a response in an external application?

[Regular-Baf]

I think the whole idea behind having the user's computer handle the compute for AI-powered applications is quite awesome, so I've decided to try and build a Minecraft mod around it. The application seems to host a server, but I was just wondering how to correctly send and receive ?

[louisgv]

@Regular-Baf
What language does the Minecraft mod use? The streaming server exposed by local.ai is a standard SSE completion endpoint, mirroring closely openai's API. LMK if you wanna jump on a call and we can jam on something this week.

[Regular-Baf]

I'm quite new to modding and programming in general so ChatGPT is doing the heavy lifting, but I'm writing the mod in Kotlin for Fabric. The code is all public here. I spent a few hours today and I've managed to get some basic functionality down, but the replies from the model are very hit and miss, likely a prompt template issue. I'd be down to discuss this project more, my Discord is Bafmaster#2997, or let me know if there is a better place to reach you :)

[louisgv]

@Regular-Baf friend request sent on Discord!

[FernandoFerrer]

Could you please provide an exaple call to the SSE completion endpoint? I am trying to call it using Python requests and sseclient but not able to make it work. Thanks!

[louisgv]

@FernandoFerrer it's a similar API to openai, here's the js call:

https://github.com/louisgv/local.ai/blob/main/apps/desktop/src/features/thread/use-active-thread.ts#L82-L114
https://github.com/louisgv/local.ai/blob/main/apps/desktop/src/features/thread/process-sse-stream.ts

It's a data stream /completion endpoint. Maybe you didn't include the /completion?

[FernandoFerrerTDK]

Thank you very much! Following your example I was able to make it work :)

[louisgv]

I think it would be awesome if we have a simple example python project or something like a streamlit playground for folks to quickly test out the API call.

@FernandoFerrerTDK @Regular-Baf Would you guys be down publish an example github repo project? We can then link them in our github and frontpage as well. I know that @Regular-Baf got a working Kotlin client, and I think having both as an example would be huge!

[Regular-Baf]

Maybe a wiki page on local.ai's own GitHub page would go a long way for helping with this? I'd absolutely be happy for my code to be used there as an example. This is the part of my mod which handles the API call.

[louisgv]

Just enabled wiki: https://github.com/louisgv/local.ai/wiki

[louisgv]

See https://github.com/louisgv/local.ai/wiki