Multilingual-support microservice #259
Conversation
siddhivelankar23
requested review from
chensuyue,
ftian1,
mkbhanda,
preethivenkatesh,
chickenrae and
tomlenth
as code owners
| ```python | ||
| @traceable(run_type="multilingual-support") | ||
| @register_statistics(names=["opea_service@multilingual_support"]) | ||
| def multilingual(input: LLMParamsDoc) -> Union[LLMParamsDoc, ChatCompletionRequest, SearchedDoc]: |
There was a problem hiding this comment.
LLMParamsDoc only contains user's query, where do you plan to store the RAG's response?
There was a problem hiding this comment.
The input should be GeneratedDoc (to contain both input query and llm response) - I have updated the rfc accordingly
There was a problem hiding this comment.
Why not ensure the LLM generates output in the same language as the query during the first LLM inference, instead of introducing an additional microservice? Adding another service significantly increases deployment costs. It would be more efficient to select an LLM within the RAG pipeline that supports both input and output languages. Are there specific application scenarios that justify this approach?
There was a problem hiding this comment.
We tested a lot of models that support multiple languages but noticed significant tradeoff between accuracy of answers and multilingual support. Most of these modes would not respond accurately in all the cases in languages other than English and hence we set up this microservice for our purpose and now want to upstream it to OPEA.
No description provided.