translation cache

[Roger-luo]

We need a cache for switching languages and store the Translation for a generated i18n package

[Gnimuc]

How about directly working on the META level? I mean directly manipulate those IdDicts(they're mutable objects) instead of reimplementing some Base's doc macros. Here is an example:

Retrieve IdDict

julia> mathDoc = getfield(Base.Math, Base.Docs.META)
IdDict{Any,Any} with 72 entries:
  Base.log2                 => Base.Docs.MultiDoc(Type[Tuple{Any}], IdDict{Any,Any}(Tuple{Any}=>Base.Docs.DocStr(svec("    log2(x)\n\nCompute …
  Base.Math.cos_kernel      => Base.Docs.MultiDoc(Type[Tuple{Base.Math.DoubleFloat64}], IdDict{Any,Any}(Tuple{Base.Math.DoubleFloat64}=>Base.D…
  Base.Math.rem_pio2_kernel => Base.Docs.MultiDoc(Type[Tuple{Float64}], IdDict{Any,Any}(Tuple{Float64}=>Base.Docs.DocStr(svec("    rem_pio2_ke…
  Base.Math.cot             => Base.Docs.MultiDoc(Type[Union{Tuple{T}, Tuple{T}} where T<:Number], IdDict{Any,Any}(Union{Tuple{T}, Tuple{T}} w…
  Base.muladd               => Base.Docs.MultiDoc(Type[Tuple{Any,Any,Any}], IdDict{Any,Any}(Tuple{Any,Any,Any}=>Base.Docs.DocStr(svec("    mul…
  Base.atanh                => Base.Docs.MultiDoc(Type[Tuple{Number}], IdDict{Any,Any}(Tuple{Number}=>Base.Docs.DocStr(svec("    atanh(x)\n\nC…

julia> libcDoc = getfield(Base.Libc, Base.Docs.META)
IdDict{Any,Any} with 18 entries:
  Base.Libc.time          => Base.Docs.MultiDoc(Type[Tuple{Base.Libc.TmStruct}, Tuple{}], IdDict{Any,Any}(Tuple{}=>Base.Docs.DocStr(svec("    …
  Base.Libc.flush_cstdio  => Base.Docs.MultiDoc(Type[Tuple{}], IdDict{Any,Any}(Tuple{}=>Base.Docs.DocStr(svec("    flush_cstdio()\n\nFlushes t…
  Base.Libc.TmStruct      => Base.Docs.MultiDoc(Type[Union{}], IdDict{Any,Any}(Union{}=>Base.Docs.DocStr(svec("    TmStruct([seconds])\n\nConv…
  Base.Libc.strptime      => Base.Docs.MultiDoc(Type[Tuple{AbstractString}], IdDict{Any,Any}(Tuple{AbstractString}=>Base.Docs.DocStr(svec("   …
  Base.Libc.gethostname   => Base.Docs.MultiDoc(Type[Tuple{}], IdDict{Any,Any}(Tuple{}=>Base.Docs.DocStr(svec("    gethostname() -> AbstractSt…

Duplicate

julia> tmp = copy(libcDoc)
IdDict{Any,Any} with 18 entries:
  Base.Libc.time          => Base.Docs.MultiDoc(Type[Tuple{Base.Libc.TmStruct}, Tuple{}], IdDict{Any,Any}(Tuple{}=>Base.Docs.DocStr(svec("    …
  Base.Libc.flush_cstdio  => Base.Docs.MultiDoc(Type[Tuple{}], IdDict{Any,Any}(Tuple{}=>Base.Docs.DocStr(svec("    flush_cstdio()\n\nFlushes t…
  Base.Libc.TmStruct      => Base.Docs.MultiDoc(Type[Union{}], IdDict{Any,Any}(Union{}=>Base.Docs.DocStr(svec("    TmStruct([seconds])\n\nConv…
  Base.Libc.strptime      => Base.Docs.MultiDoc(Type[Tuple{AbstractString}], IdDict{Any,Any}(Tuple{AbstractString}=>Base.Docs.DocStr(svec("   …
  Base.Libc.gethostname   => Base.Docs.MultiDoc(Type[Tuple{}], IdDict{Any,Any}(Tuple{}=>Base.Docs.DocStr(svec("    gethostname() -> AbstractSt…
  Base.Libc.calloc        => Base.Docs.MultiDoc(Type[Tuple{Integer,Integer}], IdDict{Any,Any}(Tuple{Integer,Integer}=>Base.Docs.DocStr(svec(" …
  Base.Libc.malloc        => Base.Docs.MultiDoc(Type[Tuple{Integer}], IdDict{Any,Any}(Tuple{Integer}=>Base.Docs.DocStr(svec("    malloc(size::…

Edit

julia> empty!(libcDoc)
IdDict{Any,Any} with 0 entries

julia> merge!(libcDoc, mathDoc)
IdDict{Any,Any} with 72 entries:

julia> empty!(mathDoc)
IdDict{Any,Any} with 0 entries

julia> merge!(mathDoc, tmp)
IdDict{Any,Any} with 18 entries:

julia> getfield(Base.Libc, Base.Docs.META)
IdDict{Any,Any} with 72 entries:
  Base.log2                 => Base.Docs.MultiDoc(Type[Tuple{Any}], IdDict{Any,Any}(Tuple{Any}=>Base.Docs.DocStr(svec("    log2(x)\n\nCompute …
  Base.Math.cos_kernel      => Base.Docs.MultiDoc(Type[Tuple{Base.Math.DoubleFloat64}], IdDict{Any,Any}(Tuple{Base.Math.DoubleFloat64}=>Base.D…
  Base.Math.rem_pio2_kernel => Base.Docs.MultiDoc(Type[Tuple{Float64}], IdDict{Any,Any}(Tuple{Float64}=>Base.Docs.DocStr(svec("    rem_pio2_ke…
  Base.Math.cot             => Base.Docs.MultiDoc(Type[Union{Tuple{T}, Tuple{T}} where T<:Number], IdDict{Any,Any}(Union{Tuple{T}, Tuple{T}} w…
  Base.muladd               => Base.Docs.MultiDoc(Type[Tuple{Any,Any,Any}], IdDict{Any,Any}(Tuple{Any,Any,Any}=>Base.Docs.DocStr(svec("    mul…

julia> getfield(Base.Math, Base.Docs.META)
IdDict{Any,Any} with 18 entries:
  Base.Libc.srand         => Base.Docs.MultiDoc(Type[Union{Tuple{}, Tuple{Any}}], IdDict{Any,Any}(Union{Tuple{}, Tuple{Any}}=>Base.Docs.DocStr…
  Base.Libc.flush_cstdio  => Base.Docs.MultiDoc(Type[Tuple{}], IdDict{Any,Any}(Tuple{}=>Base.Docs.DocStr(svec("    flush_cstdio()\n\nFlushes t…
  Base.Libc.TmStruct      => Base.Docs.MultiDoc(Type[Union{}], IdDict{Any,Any}(Union{}=>Base.Docs.DocStr(svec("    TmStruct([seconds])\n\nConv…
  Base.Libc.strptime      => Base.Docs.MultiDoc(Type[Tuple{AbstractString}], IdDict{Any,Any}(Tuple{AbstractString}=>Base.Docs.DocStr(svec("   …
  Base.Libc.gethostname   => Base.Docs.MultiDoc(Type[Tuple{}], IdDict{Any,Any}(Tuple{}=>Base.Docs.DocStr(svec("    gethostname() -> AbstractSt…
  Base.Libc.calloc        => Base.Docs.MultiDoc(Type[Tuple{Integer,Integer}], IdDict{Any,Any}(Tuple{Integer,Integer}=>Base.Docs.DocStr(svec(" …

now, the META of Base.Libc and Base.Math has exchanged.

[Roger-luo]

You can get meta directly with Docs.meta. The doc system's meta is for storing current docstrings, but we will need to store the pair of Chinese<=>English, the original META IdDict is not able to store this pair, since you can not add a new member to a Julia type.

[Roger-luo]

This package now basically works on v0.7, but the @i18n macro will directly replace the doc string in this way instead of store them in a cache. you will have to restart Julia to get other languages.

What I have in mind is:

1. @i18n will store the doc string tag by string literals like zh"", @lang zh_TW "", fr"" to their related signature, e.g sin(::Number), then we have an IdDict (maybe Dict is enough, I still don't know why they use IdDict) stores pairs like binding=>translation
2. then check the current language in ENV, if there is none, then check the system locale. And select doc strings of this language from our own IdDict.
3. switch language: users can switch languages by set a global variable represents the current language (maybe just store it in ENV)

[Gnimuc]

I guess I misunderstood the issue😅. What I was talking about is how to cache META and exchange current docstrings online. In this way, we don't need to restart Julia to do the switch.

then we have an IdDict (maybe Dict is enough, I still don't know why they use IdDict) stores pairs like binding=>translation

It's impossible to store IdDict offline, since those keys aka object ids vary between different Julia sessions.

[Roger-luo]

Okay, we cannot simply exchange two IdDict, because localization can use different locales, this usually maps the original document string to multiple strings, the relationship will looks like the Translationtype here. Therefore, simply cache the original IdDict is not very clean, we should let users be able to switch different locales online.

[Gnimuc]

Now, my understanding is we have many locales and a single META(current docstrings); the original docstrings belongs to one of the locales(e.g. en); before modifying META, we need to dump the original docstrings out as locale, this could happen either online or offline; we need a function to loop through the META and change the docstrings to a specific language(e.g. zh_CN, en).

[Roger-luo]

Yup, that's it. Therefore we need to store all the locales in a data set, and select one locale to the META according to the ENV