Inconsistent locale string between platforms

[KoBeWi]

Godot version:

3.2.3 rc1

Issue description:

So I had this piece of code in my game:

match OS.get_locale():
  "pl":
    text = "some pl string"
  _:
    text = "some en string"

It worked fine on Windows, but always returned the latter string on Android. As it turned out, on Android the returned locale is pl_PL. Now, this is platform-specific (OS and stuff), so let's say it's ok. But then there's TranslationServer.get_locale(), which has exactly the same problem and it's not ok. My bizarre finding is that when you use TranslationServer.set_locale(), the get_locale() method will return the string you just set (not sure about OS). This is evident by the fact that my code above worked when I replaced OS with TranslationServer and then changed language to en and back to pl (which overrode pl_PL with pl).

Sooo... TranslationServer should be platform-independent when being queried for locale. Or at least the documentation should mention the intended way to check current locale.

Steps to reproduce:

print(OS.get_locale())

Run it on different platforms and compare the results.

[akien-mga]

It's normal that the locale is different, even on the same OS you can have one using pl_PL and one using pl_GB, but you want both Polish speakers in Poland and in UK to use the Polish (pl) translation. So unless you provide separate translations for regional variants, the regional variant is truncated and the base language code is used. The OS locale is a property of the OS, so there's no reason for Godot to change that one.

What you need is #37246.

[akien-mga]

@bruvzg #40708 makes this clear in the documentation, but I guess there might still be a use case to provide users with the language code only if they're not interested in all the extras.

Currently they'd have to do OS.get_locale().left(3).replace("_", "") to get a 2 or 3 letter language code, maybe we can provide this as a convenience method?

Could be OS.get_locale(bool language_only = false) though I guess this would imply re-implementing this in each OS::get_locale() override.

[bruvzg]

we can provide this as a convenience method?

Maybe it's better to have a few locale matching functions: locale_match(a, b) -> distance or/and locale_best_match(target, [list]) -> best from the list. (With the different weights for different parts of locale).

[akien-mga]

Maybe it's better to have a few locale matching functions: locale_match(a, b) -> distance or/and locale_best_match(target, [list]) -> best from the list. (With the different weights for different parts of locale).

I feel that's a bit too complex to use for most users.

A case I was just told about was trying to index a dictionary (I suppose) using OS.get_locale(), and running into a crash when testing on Linux because it differed from what was expected while developing on Windows, e.g.:

• Developer users French locale on Windows and OS.get_locale() says fr (at least in 3.3), so they use this as key for French language.
• French Canadian user on Linux tries the game, OS.get_locale() returns fr_CA and developer didn't know they should sanitize the value, so crash when indexing dictionary with invalid key.

IMO something simple like OS.get_locale_language() that returns only the 2 or 3 letter language code from OS.get_locale() would be a good option. It should be referenced in the documentation of OS.get_locale() for the specific use case of retrieving only the "main" OS lang code when one doesn't aim at supporting various scripts or regional variants (which is the most common use case for many games supporting "generic" French, Spanish, Simplified Chinese, etc.).