[REQUEST] - Reasoning Settings, other languages?

[consecutiotemporum]

Reference Issues

No response

Summary

Is there any way to include other languages among those available for Reasoning Settings?
What does the current list of languages depend on?
Thank you.

Basic Example

--

Drawbacks

--

Additional information

No response

[taprosoft]

Adding more language is as simple as changing some config but you have to run kotaemon from source.

Go to kotaemon/libs/ktem/ktem/utils/lang.py

SUPPORTED_LANGUAGE_MAP = {"en": "English", "ja": "Japanese", "vi": "Vietnamese"}

Change the above list as you like.

Also, change this value in flowsettings.py

SETTINGS_REASONING{
    "lang": {
        "name": "Language",
        "value": "en",
        "choices": [("English", "en"), ("Japanese", "ja"), ("Vietnamese", "vi")],
        "component": "dropdown",
    },
}

Run the app again python app.py and that's it.

[consecutiotemporum]

Go to kotaemon/libs/ktem/ktem/utils/lang.py

Thanks!

(As for "user install", the correct folder is: kotaemon-app\install_dir\env\Lib\site-packages\ktem\utils\lang.py)

[drorata]

I'm trying to understand whether this will allow me to upload documents in other (natural) languages? Trying out the docker version and loading documents in German, I always got close to zero relevancy score.

[taprosoft]

@drorata should be no problems. Scores can be low but please check if LLM answer can satisfy your need. Make sure to check Use LLM relevant scoring in Retrieval settings to make it more reliable.