Optimize storage for IcuLocaleData data

[marek-safar]

The massive string which is declared in

runtime/src/libraries/System.Private.CoreLib/src/System/Globalization/IcuLocaleData.cs

Line 32 in e2099f6

private const string c_localeNames =

is cost close to 10kb on the disc. It should be possible to optimize this to reduce the size as the data are in a very narrow range (low case ASCII only).

@eerhardt

[ghost]

Tagging subscribers to this area: @tarekgh, @safern, @krwq
See info in area-owners.md if you want to be subscribed.

Issue Details

---

The massive string which is declared in

runtime/src/libraries/System.Private.CoreLib/src/System/Globalization/IcuLocaleData.cs

Line 32 in e2099f6

private const string c_localeNames =

is cost close to 10kb on the disc. It should be possible to optimize this to reduce the size as the data are in a very narrow range (low case ASCII only).

@eerhardt

Author:	marek-safar
Assignees:	-
Labels:	area-System.Globalization, untriaged
Milestone:	-

[tarekgh]

[edited comment]

@marek-safar do you suggest we compress it and expand it at runtime? I think we can store it as ASCII encoding or store it as compressed binary and expand it at runtime.

[krwq]

@tarekgh I think suggestion might be to use UTF-8 or ASCII rather than UTF-16, that should remove almost half of the bytes

[marek-safar]

do you suggest we compress it and expand it at runtime

Yeah, story it more efficiently so we don't have a massive string in metadata and on a whole copy of it on the heap. Most apps will use a few lcid strings only anyway. There are numerous way how to do it one of them could be to replace s_lcids, s_localeNamesIndices arrays with something like

static ReadOnlySpan<byte>values => new byte[] {
   // "az" or "az-cyrl" or "az-cyrl-az" 
   (byte)'a', (byte)'z', (byte)'-', (byte)'c', (byte)'y', (byte)'r', (byte)'l', (byte)'-', (byte)'a', (byte)'z', 
};

// LCID values don't match the real ones in this example
static readonly int[] s_lcids = new int[]
{
   // 16bit LCID | index into string values | string value length
   0x10 | 0 << 12 | 7 << 4, // az-cyrl
   0xbb | 0 << 12 | 2 << 4, // az
};

There is a dependency from c_localeNames but that could be improved as well