You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Since model server like HF TGI/vLLM/OpenLLM offer their models as OpenAI API REST endpoint, a generic OpenAI end point with a free field for name model could offer an integration point.
In the case of model served by HF TGI, the model name is fixed to "tgi" since the server "serve" only one model at time.
So, this generic option should be added on request into the Model configuration page more than once, to reach out several independent model servers.
The text was updated successfully, but these errors were encountered:
Since model server like HF TGI/vLLM/OpenLLM offer their models as OpenAI API REST endpoint, a generic OpenAI end point with a free field for name model could offer an integration point.
In the case of model served by HF TGI, the model name is fixed to "tgi" since the server "serve" only one model at time.
So, this generic option should be added on request into the Model configuration page more than once, to reach out several independent model servers.
The text was updated successfully, but these errors were encountered: