OpenAI compatible minimal API using ORT-GenAI #995
bpietroiu
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I have created a minimal implementation, including a full port of a Jinja parser/interpreter
Docker and docker compose for deployment
OpenTelemetry for logging and monitoring
Dynamic Batching
Chat prompt template processing using Jinja JS to C# ported by me
can be found here https://github.com/bpietroiu/GenAI.Server
Beta Was this translation helpful? Give feedback.
All reactions