-
Notifications
You must be signed in to change notification settings - Fork 57
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multilingual-support microservice #259
Open
siddhivelankar23
wants to merge
18
commits into
opea-project:main
Choose a base branch
from
siddhivelankar23:main
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
18 commits
Select commit
Hold shift + click to select a range
a881ab2
Add files via upload
siddhivelankar23 13ae998
Create multilingual-support.md
siddhivelankar23 730bda7
Update multilingual-support.md
siddhivelankar23 c641713
Update multilingual-support.md
siddhivelankar23 9475d79
Update multilingual-support.md
siddhivelankar23 d27f6de
Update multilingual-support.md
siddhivelankar23 15672eb
Merge branch 'opea-project:main' into main
siddhivelankar23 56968cc
Add files via upload
siddhivelankar23 89dfadb
Delete community/rfcs/assets/translation-enterprise-rag2.drawio.png
siddhivelankar23 64ffd74
Add files via upload
siddhivelankar23 e037683
Update multilingual-support.md
siddhivelankar23 f71f029
Merge branch 'opea-project:main' into main
siddhivelankar23 0bc1e39
Add files via upload
siddhivelankar23 3e89b4b
Delete multilingual-support-diagram.png
siddhivelankar23 32ee245
Add files via upload
siddhivelankar23 1600a33
Update multilingual-support.md
siddhivelankar23 5e081cb
Update multilingual-support.md
siddhivelankar23 cf8fa62
input is GeneratedDoc
siddhivelankar23 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
# Multilingual-support microservice | ||
|
||
## Author | ||
|
||
[Siddhi Velankar](https://github.com/siddhivelankar23) | ||
|
||
## Status | ||
|
||
Under review | ||
|
||
## Objective | ||
|
||
The objective of this RFC is to propose the creation of a separate multilingual-support microservice to handle translation that can be seamlessly used with other microservices. Currently, translation exists as a separate megaservice, which makes it difficult to integrate with other standalone microservices or applications. This proposal aims to simplify and standardize the process of multilingual support by creating a dedicated, scalable, and easily integrable microservice that can seamlessly interact with other services within the system architecture. The goal is to provide a flexible, centralized approach to translation and enhancing the user experience across various languages. | ||
|
||
## Motivation | ||
|
||
In the current system, translation is managed by a monolithic megaservice, which is challenging to integrate with other microservices. | ||
By introducing multilingual support as a standalone microservice, we aim to decouple the translation functionality from other services, making it easier to scale and manage. | ||
|
||
|
||
## Design Proposal | ||
|
||
![Multilingual support microservice](./assets/multilingual-support-diagram.png) | ||
|
||
|
||
The proposed architecture involves the creation of a new microservice called multilingual-support. | ||
This microservice does the following - | ||
1. Detects the language of the user's query as well as the response from the first llm microservice. | ||
2. Configures a translation prompt to convert the answer from the response language to the query language. | ||
3. This prompt is sent to the second llm microservice to generate the final answer. | ||
|
||
This ensures seamless, accurate communication across different languages in real time. | ||
|
||
|
||
Signature of multilingual-support microservice | ||
```python | ||
@traceable(run_type="multilingual-support") | ||
@register_statistics(names=["opea_service@multilingual_support"]) | ||
def multilingual(input: GeneratedDoc) -> Union[LLMParamsDoc, ChatCompletionRequest, SearchedDoc]: | ||
``` |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not ensure the LLM generates output in the same language as the query during the first LLM inference, instead of introducing an additional microservice? Adding another service significantly increases deployment costs. It would be more efficient to select an LLM within the RAG pipeline that supports both input and output languages. Are there specific application scenarios that justify this approach?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We tested a lot of models that support multiple languages but noticed significant tradeoff between accuracy of answers and multilingual support. Most of these modes would not respond accurately in all the cases in languages other than English and hence we set up this microservice for our purpose and now want to upstream it to OPEA.