Deferred translation class with optional context for option/effect/item-action names #26402

Qrox · 2018-10-25T17:27:32Z

Summary

SUMMARY: I18N "Deferred translation class with optional context for option/effect/item-action names"

Purpose of change

There are several places in both cpp and json code where a word with multiple meanings is used without translation context. This makes it hard to translate the given strings in some other languages. This PR adds the ability to specify a translation context in these cases.

Closes #21760.

Describe the solution

A translation class is created which supports an optional translation context and deserialization from JSON;
Code for options, effects, and item actions is adjusted to use the new translation class, and JSON strings that needed a context are updated to include one.
Keywords for the xgettext call in update_pot.sh is updated so it also searches for the constructor of the translation class;
The python scripts used for extracting json strings are updated so they work properly on Windows;
The python scripts are updated to extract json strings with context;
merge_po.sh is updated to disable fuzzy matching and remove obsolete strings; (I'm not sure about this one. The .po files in the repo does not have fuzzy matching nor obsolete strings, but maybe they are not removed in the script for a reason? @BrettDong should merge_po.sh be kept as is?)
Updated documentations to reflect the new translation class and JSON syntax;
Also fixed excess spaces in language names.

Additional context

I need comments on the changes to the shell and python scripts, as I'm not very familiar with the languages.

The following are some screenshots showing the changes with this PR. I've marked relevant text with English so it's easier to see what is changed.

Before:

Item action menu:

The translations for "play - handheld game machine" and "smoke - cigar" do not make sense here.

Options menu:

According to the translation, the option is "physically disabled".

Character info UI:

These translations are correct.

After (after generating, editing, and compiling .po files):

Item action menu:

Options menu:

Character info UI:

BevapDin · 2018-10-25T17:44:00Z

src/effect.cpp

-            new_etype.name.push_back( jsarr.next_string() );
+            translation name;
+            if( !jsarr.read_next( name ) ) {
+                debugmsg( "Error reading effect names" );


It would be better to call jsarr.throw_error to generate an exception along with a proper JSON context.

BevapDin · 2018-10-25T17:51:02Z

src/translations.h

@@ -19,6 +19,10 @@
 #define translate_marker_context(c, x) x
 #endif

+#include <string>
+#include "json.h"


Don't include "json.h" here, it's quite large and has more dependencies. You don't need to derive from JsonDeserializer, having a deserialze function with proper signature is enough.

BevapDin · 2018-10-25T18:33:30Z

Have you considered to adding a translation::operator std::string() const { return translated(); } function? This would implicitly convert translation objects to std::string (applying the translation).

This means you don't need to add translated() explicitly so often.

AMurkin · 2018-10-25T19:18:45Z

Will close #24102?

Qrox · 2018-10-26T01:16:49Z

@AMurkin Currently keybindings are not using the translation class, but that's definitely doable. On another hand this should close #21760 you referenced in that issue.

BrettDong · 2018-10-26T01:23:16Z

merge_po.sh is a pre-Transifex era thing. The script is actually not used anymore.

Qrox · 2018-10-26T03:16:32Z

@BrettDong I'll adapt the script as a tool for locally testing translations then.

Qrox · 2018-10-26T03:25:41Z

Have you considered to adding a translation::operator std::string() const { return translated(); } function?

I'm not sure about it. It could of course make the code less verbose, but I'm concerned that if someone wants to use the class in more places, and happens to forget to change the variable from std::string to translation, then the string may get translated prematurely. I guess it's better to keep it explicit.

…lation # Conflicts: # lang/extract_json_strings.py

Qrox · 2018-10-26T13:08:10Z

@AMurkin Actually the strings you mentioned in #24102 are fixable by simply using pgettext, so I'm not going to address it in this PR.

jbytheway · 2018-10-26T19:29:54Z

I see you've changed the Python scripts to use some Python 3-isms, rather than Python 2.

This is not really about this PR, but the #!/usr/bin/env python lines should really be changed to specify either python2 or python3 specifically, because just python can end up using either one and makes it difficult to have the script work everywhere (which leads to things like you having to make these changes).

Qrox · 2018-10-27T02:38:57Z

Thanks, I'll change them to python3 since extract_json_strings.py already uses python3.

Qrox added 16 commits October 25, 2018 16:25

Translation class

b14e967

Update json string extraction code

87950ac

Use translation class for effect names

cac5392

Use translation class for item actions

0ba22d8

Use the new optional context in effect.json & item_actions.json

6190ee4

Break on error when reading effect names

15743bf

Restore empty effect name = not shown behavior

9ba252e

Update EFFECTS_JSON.md

7773792

Modify python scripts for them to run correctly on Windows

d3aeed8

Disable fuzzy matching for msgmerge

fe7c890

Fix translation loading

c6c1826

Remove obsolete translations

be45e7f

Use translation class for options

731b6cd

Change smoke effect name context

a099f88

Add translation constructor to xgettext keywords list

dd9d206

Mark language strings as non-translated & remove excess spaces

32a43ed

Qrox mentioned this pull request Oct 25, 2018

Add an optional translation context for option items #26256

Closed

BevapDin reviewed Oct 25, 2018

View reviewed changes

Update TRANSLATING.md

390a25b

ZhilkinSerg added [C++] Changes (can be) made in C++. Previously named `Code` <Bugfix> This is a fix for a bug (or closes open issue) Code: Infrastructure / Style / Static Analysis Code internal infrastructure and style Translation I18n labels Oct 26, 2018

Qrox added 6 commits October 26, 2018 19:23

Merge remote-tracking branch 'CleverRaven/master' into deferred-trans…

a09d38f

…lation # Conflicts: # lang/extract_json_strings.py

Fix msgmerge parameter in merge_po.sh

ff2cbca

Fix comment in translations.h

548b2ce

Throw error when fail to parse effect names

4c81b16

Remove json.h dependency from translations.h & fix a comment

aae1447

Restore name string emptiness test in item_action.cpp

ac435f7

Qrox added 2 commits October 27, 2018 13:43

Use python3 for python scripts

ee0b251

More documentations on contexted JSON strings

c179055

Qrox changed the title ~~[WIP][CR] Deferred translation class with optional context for option/effect/item-action names~~ Deferred translation class with optional context for option/effect/item-action names Oct 28, 2018

Qrox added 2 commits November 9, 2018 12:55

Merge branch 'master' into deferred-translation

50e22ab

Update header includes

39ca634

Qrox force-pushed the deferred-translation branch from 5e77e4d to 39ca634 Compare November 9, 2018 05:58

kevingranade merged commit 46f969f into CleverRaven:master Nov 10, 2018

Qrox deleted the deferred-translation branch November 15, 2018 04:08

This was referenced Aug 3, 2019

[i18n] 'sling' has different translations in different contexts. #32863

Closed

Add some missing translations for gunmod and Mod #32868

Closed

Qrox mentioned this pull request Sep 30, 2019

"pike" is ambiguous #34263

Closed

olanti-p mentioned this pull request Nov 27, 2021

Prioritize active mod translation files over base game language file? cataclysmbnteam/Cataclysm-BN#1142

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deferred translation class with optional context for option/effect/item-action names #26402

Deferred translation class with optional context for option/effect/item-action names #26402

Qrox commented Oct 25, 2018 •

edited

Loading

BevapDin Oct 25, 2018

BevapDin Oct 25, 2018

BevapDin commented Oct 25, 2018

AMurkin commented Oct 25, 2018

Qrox commented Oct 26, 2018

BrettDong commented Oct 26, 2018

Qrox commented Oct 26, 2018

Qrox commented Oct 26, 2018 •

edited

Loading

Qrox commented Oct 26, 2018

jbytheway commented Oct 26, 2018

Qrox commented Oct 27, 2018

Deferred translation class with optional context for option/effect/item-action names #26402

Deferred translation class with optional context for option/effect/item-action names #26402

Conversation

Qrox commented Oct 25, 2018 • edited Loading

Summary

Purpose of change

Describe the solution

Additional context

BevapDin Oct 25, 2018

Choose a reason for hiding this comment

BevapDin Oct 25, 2018

Choose a reason for hiding this comment

BevapDin commented Oct 25, 2018

AMurkin commented Oct 25, 2018

Qrox commented Oct 26, 2018

BrettDong commented Oct 26, 2018

Qrox commented Oct 26, 2018

Qrox commented Oct 26, 2018 • edited Loading

Qrox commented Oct 26, 2018

jbytheway commented Oct 26, 2018

Qrox commented Oct 27, 2018

Qrox commented Oct 25, 2018 •

edited

Loading

Qrox commented Oct 26, 2018 •

edited

Loading