Multi language support #214

AlexHTW · 2023-04-11T23:27:02Z

This PR implements a language setting for the general bot messages with translations for german and russian, as suggested in #193.
Text fragments from bot messages are stored for different languages in the translations.json file using descriptive keys.
The bot messages in telegram_bot.py have been replaced by function calls pulling the desired language option of a text fragment. I decided to store text fragments without variables or formatting for better decoupling and easier addition of new languages. However, maybe it obscures the actual output in the code.
The language can be set in the .env variable LANGUAGE using the ISO 639-1 code (en, de, ru). With a possible addition of user settings it can hopefully be easily set per user in the future.

I think this feature will be a useful addition to many (myself included). However, it probably complicates maintainability and the addition of new bot messages. Looking forward to your thoughts on this @n3d1117.

@C0la149 Can you check out this PR, test it and maybe give feedback on the russian translations?

todo:

localize openai messages
create translations post and link in README
add turkish translations

"language" seems to be reserved on some systems for the actual system language

MATTAM540 · 2023-04-13T10:20:31Z

@AlexHTW I can help with the Turkish language

AlexHTW · 2023-04-13T11:20:38Z

I noticed that I didn't add the "Tokens used: xx (xx prompt, xx completion)" part to the translations, because it is the only general bot message coming from another class. Should I add it? Then I would need to add the translations.json import and localized_text function to openai_helper aswell or is there a better way?

AlexHTW · 2023-04-13T11:24:18Z

@AlexHTW I can help with the Turkish language

@muhammed540
Awesome!
If you want your contribution to be in the repository you'll have to send a PR to the branch of my fork and I will add it to this PR or you wait if this will be added to the main repo and then create a seperate PR.
If you don't want to deal with the PR I can provide you the list of text fragments to translate and then add them myself and credit you in the commit message or somewhere.

MATTAM540 · 2023-04-13T11:28:36Z

@AlexHTW Send me the texts and I will translate them. Then add it to your own files.

AlexHTW · 2023-04-13T11:30:04Z

@muhammed540 great, here are the texts

"Show help message",
"Reset the conversation. Optionally pass high-level instructions (e.g. /reset You are a helpful assistant)",
"Generate image from prompt (e.g. /image cat)",
"Get your current usage statistics",
"Resend the latest massage",
"Chat with the bot!",
"Sorry, you are not allowed to use this bot. You can check out the source code at https://github.com/n3d1117/chatgpt-telegram-bot",
"Sorry, you have reached your usage limit.",
"I'm a ChatGPT bot, talk to me!", 
"Send me a voice message or file and I'll transcribe it for you", 
"Open source at https://github.com/n3d1117/chatgpt-telegram-bot",
"Current conversation",
"chat messages in history",
"chat tokens in history",
"Usage today:",
"Usage this month:",
"chat tokens used.",
"images generated.",
"minutes and",
"seconds transcribed.",
"For a total amount of $",
"Your remaining budget for",
"this month",
"today",
"This month your OpenAI account was billed $",
"You have nothing to resend",
"Done!",
"Please provide a prompt! (e.g. /image cat)",
"Failed to generate image",
"Failed to download audio file",
"Make sure the file is not too large. (max 20MB)",
"Unsupported file type",
"Transcript",
"Answer",
"Failed to transcribe text",
"Failed to get response"

Edit:
In case we also add the Tokens used text (that is shown after each answer if SHOW_USAGE=true), please also translate those fragments:

"prompt",
"completion"

Thanks for your help!

n3d1117 · 2023-04-13T11:34:28Z

Great work @AlexHTW!

I noticed that I didn't add the "Tokens used: xx (xx prompt, xx completion)" part to the translations, because it is the only general bot message coming from another class. Should I add it? Then I would need to add the translations.json import and localized_text function to openai_helper aswell or is there a better way?

Sure! Maybe you could reuse the same localized_text function in telegram_bot.py? So that you would only need to add bot_language to openai_config and call it from the helper class by importing it (from bot.telegram_bot import localized_text).

Maybe these could be localized as well?

OpenAI Rate Limit exceeded
OpenAI Invalid request
An error has occurred
Please try again in a while.

Also, one last minor thing, maybe you could add a small section in the README explaining how to contribute with new translations! 🚀

AlexHTW · 2023-04-13T11:46:00Z

Hey @n3d1117 , thanks for the feedback!

Sure! Maybe you could reuse the same localized_text function in telegram_bot.py? So that you would only need to add bot_language to openai_config and call it from the helper class by importing it (from bot.telegram_bot import localized_text).

Yes, of course 🤦, thanks for the hint!

Maybe these could be localized as well?

* OpenAI Rate Limit exceeded

* OpenAI Invalid request

* An error has occurred

* Please try again in a while.

Ah, I didn't realize they are also send in the bot. Will add them, @muhammed540 Please also translate those 🇹🇷

Also, one last minor thing, maybe you could add a small section in the README explaining how to contribute with new translations!

Yes, of course, I am really looking forward to language additions from other people!
I was thinking about making a discussions post to explain how to contribute translations, which we can link from the README.

n3d1117 · 2023-04-13T11:47:11Z

I was thinking about making a discussions post to explain how to contribute translations, which we can link from the README.

Works for me! Thanks

MATTAM540 · 2023-04-13T12:40:52Z

@AlexHTW

"Show help message",
"Reset the conversation. Optionally pass high-level instructions (e.g. /reset You are a helpful assistant)",
"Generate image from prompt (e.g. /image cat)",
"Get your current usage statistics",
"Resend the latest massage",
"Chat with the bot!",
"Sorry, you are not allowed to use this bot. You can check out the source code at https://github.com/n3d1117/chatgpt-telegram-bot",
"Sorry, you have reached your usage limit.",
"I'm a ChatGPT bot, talk to me!", 
"Send me a voice message or file and I'll transcribe it for you", 
"Open source at https://github.com/n3d1117/chatgpt-telegram-bot",
"Current conversation",
"chat messages in history",
"chat tokens in history",
"Usage today:",
"Usage this month:",
"chat tokens used.",
"images generated.",
"minutes and",
"seconds transcribed.",
"For a total amount of $",
"Your remaining budget for",
"this month",
"today",
"This month your OpenAI account was billed $",
"You have nothing to resend",
"Done!",
"Please provide a prompt! (e.g. /image cat)",
"Failed to generate image",
"Failed to download audio file",
"Make sure the file is not too large. (max 20MB)",
"Unsupported file type",
"Transcript",
"Answer",
"Failed to transcribe text",
"Failed to get response"

"Yardım mesajını göster",
"Konuşmayı sıfırla. İsteğe bağlı olarak botun nasıl davranacağını belirleyin (Örneğin: /reset Sen yardımcı bir asistansın)
"Verilen komuta göre görüntü üret (Örneğin /image kedi)",
"Mevcut kullanım istatistiklerinizi alın",
"En son mesajı yeniden gönder",
"Bot ile sohbet edin!",
"Üzgünüz, bu botu kullanmanıza izin verilmiyor. Botun kaynak koduna göz atmak isterseniz: https://github.com/n3d1117/chatgpt-telegram-bot",
"Üzgünüz, kullanım limitinize ulaştınız.",
"Ben bir ChatGPT botuyum, konuş benimle!", 
"Bana bir sesli mesaj veya dosya gönderin, sizin için yazıya çevireyim", 
"Açık kaynak kodu: https://github.com/n3d1117/chatgpt-telegram-bot",
"Güncel sohbet",
"sohbet mesajı",
"sohbet jetonu",
"Bugünkü kullanım:",
"Bu ayki kullanım:",
"sohbet token'i kullanıldı.",
"görüntüler oluşturuldu.",
"dakika ve",
"saniye sesten yazıya çeviri yapıldı.",
"Bu kullanımların toplam maliyeti $",
"için kalan bütçeniz",
"bu ay",
"bugün",
"Bu ay OpenAI hesabınız $ harcadı",
"Yeniden gönderilecek bir şeyiniz yok",
"Tamamlandı!",
"Lütfen komut giriniz (Örneğin /image kedi)",
"Görüntü oluşturulamadı",
"Ses dosyası indirilemedi",
"Dosyanın çok büyük olmadığından emin olun. (maksimum 20MB)",
"Desteklenmeyen dosya türü",
"Yazıya çevirme",
"Cevap",
"Metnin yazıya çevirme işlemi yapılamadı",
"Yanıt alınamadı"

"prompt",
"completion"

"Komut",
"Tamamlama"

OpenAI Rate Limit exceeded
OpenAI Invalid request
An error has occurred
Please try again in a while.


OpenAI maksimum istek limiti aşıldı
OpenAI Geçersiz istek
Bir hata oluştu 
Lütfen birazdan tekrar deneyiniz.

AlexHTW · 2023-04-13T14:00:07Z

@muhammed540 Thank you very much! Is it possible to rephrase this text: "Bu ay OpenAI hesabınız $ harcadı" with the $-amount at the end? Because the dollar amount is inserted after the phrase in the code. Or does it sound okay if you say "Bu ay OpenAI hesabınız $ harcadı 10"?

MATTAM540 · 2023-04-13T14:07:18Z

@muhammed540 Thank you very much! Is it possible to rephrase this text: "Bu ay OpenAI hesabınız $ harcadı" with the $-amount at the end? Because the dollar amount is inserted after the phrase in the code. Or does it sound okay if you say "Bu ay OpenAI hesabınız $ harcadı 10"?

New sentence:
Bu ay OpenAI hesabınıza kesilen fatura tutarı: $

AlexHTW · 2023-04-13T14:22:05Z

@muhammed540 added your translation :) Please check it out, or proofread my screenshots

MATTAM540 · 2023-04-13T14:35:15Z

@AlexHTW
Sorry. These translations are wrong.

"sohbet jetonu" -> "sohbet token'i"

"Yazıya çevirme" -> "Yazıya çevirilmiş hali"

"Yeniden gönderilecek bir şeyiniz yok" -> "Yeniden gönderilecek bir şey yok"

"dakika ve" -> "dakika" (The word "and -> ve" is meaningless)

AlexHTW · 2023-04-13T14:42:06Z

@muhammed540

"Yazıya çevirme -> "Yazıya çevirilmiş hali"

does that also mean:
"Metnin yazıya çevirme işlemi yapılamadı" -> "Metnin yazıya çevirilmiş hali işlemi yapılamadı" ?

Here is the complete set of turkish translations, looking good now?

"tr": {
    "help_description":"Yardım mesajını göster",
    "reset_description":"Konuşmayı sıfırla. İsteğe bağlı olarak botun nasıl davranacağını belirleyin (Örneğin: /reset Sen yardımcı bir asistansın)",
    "image_description":"Verilen komuta göre görüntü üret (Örneğin /image kedi)",
    "stats_description":"Mevcut kullanım istatistiklerinizi alın",
    "resend_description":"En son mesajı yeniden gönder",
    "chat_description":"Bot ile sohbet edin!",
    "disallowed":"Üzgünüz, bu botu kullanmanıza izin verilmiyor. Botun kaynak koduna göz atmak isterseniz: https://github.com/n3d1117/chatgpt-telegram-bot",
    "budget_limit":"Üzgünüz, kullanım limitinize ulaştınız.",
    "help_text":["Ben bir ChatGPT botuyum, konuş benimle!", "Bana bir sesli mesaj veya dosya gönderin, sizin için yazıya çevireyim", "Açık kaynak kodu: https://github.com/n3d1117/chatgpt-telegram-bot"],
    "stats_conversation":["Güncel sohbet", "sohbet mesajı", "sohbet token'i"],
    "usage_today":"Bugünkü kullanım:",
    "usage_month":"Bu ayki kullanım:",
    "stats_tokens":"sohbet token'i kullanıldı.",
    "stats_images":"görüntüler oluşturuldu.",
    "stats_transcribe":["dakika", "saniye sesten yazıya çeviri yapıldı."],
    "stats_total":"💰 Bu kullanımların toplam maliyeti $",
    "stats_budget":"Için kalan bütçeniz",
    "monthly":"bu ay",
    "daily":"bugün",
    "all-time":"",
    "stats_openai":"Bu ay OpenAI hesabınıza kesilen fatura tutarı: $",
    "resend_failed":"Yeniden gönderilecek bir şey yok",
    "reset_done":"Tamamlandı!",
    "image_no_prompt":"Lütfen komut giriniz (Örneğin /image kedi)",
    "image_fail":"Görüntü oluşturulamadı",
    "media_download_fail":["Ses dosyası indirilemedi", "Dosyanın çok büyük olmadığından emin olun. (maksimum 20MB)"],
    "media_type_fail":"Desteklenmeyen dosya türü",
    "transcript":"Yazıya çevirilmiş hali",
    "answer":"Cevap",
    "transcribe_fail":"Metnin yazıya çevirme işlemi yapılamadı",
    "chat_fail":"Yanıt alınamadı",
    "prompt":"Komut",
    "completion":"Tamamlama",
    "openai_rate_limit":"OpenAI maksimum istek limiti aşıldı",
    "openai_invalid":"OpenAI Geçersiz istek",
    "error":"Bir hata oluştu",
    "try_again":"Lütfen birazdan tekrar deneyiniz"
}

MATTAM540 · 2023-04-13T14:45:28Z

@AlexHTW

does that also mean:
"Metnin yazıya çevirme işlemi yapılamadı" -> "Metnin yazıya çevirilmiş hali işlemi yapılamadı" ?

No, Keep it old. But I found another bug.

"Metnin yazıya çevirme işlemi yapılamadı" -> "Sesin yazıya çevirilme işlemi yapılamadı"

AlexHTW · 2023-04-13T22:38:57Z

Hey @n3d1117,
I added the discussions post and linked it in the README. I added translations information in several places, I leave it to you where and how you prefer it.
About the google sheet: maybe it would be better if it was yours. Alternatively I can give you full access, maybe leave a comment on the sheet.

AlexHTW and others added 5 commits April 12, 2023 00:52

added german and russian translations for bot msgs

0130faa

added language setting to env.example and readme

ab29487

fixed refactored keys for german translations

2598ac1

refactored language to bot_language

df1d405

"language" seems to be reserved on some systems for the actual system language

updated README and .env.example

7811374

AlexHTW added 2 commits April 13, 2023 16:10

added turkish translations, credit @muhammed540

4b65031

added translations for messages from openai_helper

acf500e

improved some turkish translations @muhammed540

810ad89

AlexHTW added 2 commits April 13, 2023 16:47

small fix turkish translation @muhammed540

f7fd850

small fix tukish translation n3d1117#3 @muhammed540

2011281

MATTAM540 approved these changes Apr 13, 2023

View reviewed changes

AlexHTW added 3 commits April 13, 2023 18:03

improved consistency in text localizations

47babe8

fixed punctuations in translations.json

1e0ddd7

added translations info to README

56afe44

n3d1117 merged commit 2ec6983 into n3d1117:main Apr 14, 2023

AlexHTW deleted the multi-language-support branch April 25, 2023 22:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Multi language support #214

Multi language support #214

AlexHTW commented Apr 11, 2023 •

edited

Loading

MATTAM540 commented Apr 13, 2023

AlexHTW commented Apr 13, 2023

AlexHTW commented Apr 13, 2023 •

edited

Loading

MATTAM540 commented Apr 13, 2023

AlexHTW commented Apr 13, 2023 •

edited

Loading

n3d1117 commented Apr 13, 2023

AlexHTW commented Apr 13, 2023

n3d1117 commented Apr 13, 2023

MATTAM540 commented Apr 13, 2023

AlexHTW commented Apr 13, 2023

MATTAM540 commented Apr 13, 2023

AlexHTW commented Apr 13, 2023

MATTAM540 commented Apr 13, 2023 •

edited

Loading

AlexHTW commented Apr 13, 2023 •

edited

Loading

MATTAM540 commented Apr 13, 2023 •

edited

Loading

AlexHTW commented Apr 13, 2023

Multi language support #214

Multi language support #214

Conversation

AlexHTW commented Apr 11, 2023 • edited Loading

MATTAM540 commented Apr 13, 2023

AlexHTW commented Apr 13, 2023

AlexHTW commented Apr 13, 2023 • edited Loading

MATTAM540 commented Apr 13, 2023

AlexHTW commented Apr 13, 2023 • edited Loading

n3d1117 commented Apr 13, 2023

AlexHTW commented Apr 13, 2023

n3d1117 commented Apr 13, 2023

MATTAM540 commented Apr 13, 2023

AlexHTW commented Apr 13, 2023

MATTAM540 commented Apr 13, 2023

AlexHTW commented Apr 13, 2023

MATTAM540 commented Apr 13, 2023 • edited Loading

AlexHTW commented Apr 13, 2023 • edited Loading

MATTAM540 commented Apr 13, 2023 • edited Loading

AlexHTW commented Apr 13, 2023

AlexHTW commented Apr 11, 2023 •

edited

Loading

AlexHTW commented Apr 13, 2023 •

edited

Loading

AlexHTW commented Apr 13, 2023 •

edited

Loading

MATTAM540 commented Apr 13, 2023 •

edited

Loading

AlexHTW commented Apr 13, 2023 •

edited

Loading

MATTAM540 commented Apr 13, 2023 •

edited

Loading