Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial support for ICU MessageFormat. #6582

Merged
merged 7 commits into from
Oct 27, 2021
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/admin/checks.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,10 @@ Here is a list of flags currently accepted:
Define font-size for rendering checks, see :ref:`fonts`.
``font-spacing:SPACING``
Define letter spacing for rendering checks, see :ref:`fonts`.
``icu-flags:FLAGS``
Define flags for customizing the behavior of the :ref:`check-icu-message-format` quality check.
``icu-tag-prefix:PREFIX``
Set a required prefix for XML tags for the :ref:`check-icu-message-format` quality check.
``placeholders:NAME:NAME2:...``
Placeholder strings expected in translation, see :ref:`check-placeholders`.
``replacements:FROM:TO:FROM2:TO2...``
Expand Down Expand Up @@ -89,6 +93,8 @@ Here is a list of flags currently accepted:
Enable the :ref:`check-es-format` quality check.
``i18next-interpolation``
Enable the :ref:`check-i18next-interpolation` quality check.
``icu-message-format``
Enable the :ref:`check-icu-message-format` quality check.
``java-format``
Enable the :ref:`check-java-format` quality check.
``java-messageformat``
Expand Down Expand Up @@ -145,6 +151,8 @@ Here is a list of flags currently accepted:
Skip the :ref:`check-es-format` quality check.
``ignore-i18next-interpolation``
Skip the :ref:`check-i18next-interpolation` quality check.
``ignore-icu-message-format``
Skip the :ref:`check-icu-message-format` quality check.
``ignore-java-format``
Skip the :ref:`check-java-format` quality check.
``ignore-java-messageformat``
Expand Down
67 changes: 66 additions & 1 deletion docs/user/checks.rst
Original file line number Diff line number Diff line change
Expand Up @@ -242,8 +242,73 @@ i18next interpolation
:ref:`check-formats`,
`i18next interpolation <https://www.i18next.com/translation-function/interpolation>`_

.. _check-java-format:

.. _check-icu-message-format:

ICU MessageFormat
*****************

.. versionadded:: 4.9

:Summary: There are inconsistencies or syntax errors within ICU MessageFormat placeholders.
:Scope: translated strings
:Check class: ``weblate.checks.icu.ICUMessageFormatCheck``
:Flag to enable: ``icu-message-format``
:Flag to ignore: ``ignore-icu-message-format``
:Interpolation example: ``There {number, plural, one {is one apple} other {are # apples}}.``

This check has support for both pure ICU MessageFormat messages as well as ICU with simple
XML tags. You can configure the behavior of this check by using ``icu-flags:*``, either by
opting into XML support or by disabling certain sub-checks. For example, the following flag
enables XML support while disabling validation of plural sub-messages:

.. code-block::text

icu-message-format, icu-flags:xml:-plural_selectors

+---------------------------+------------------------------------------------------------+
| ``xml`` | Enable support for simple XML tags. By default, XML tags |
| | are parsed loosely. Stray ``<`` characters are ignored |
| | if they are not reasonably part of a tag. |
+---------------------------+------------------------------------------------------------+
| ``strict-xml`` | Enable support for strict XML tags. All ``<`` characters |
| | must be escaped if they are not part of a tag. |
+---------------------------+------------------------------------------------------------+
| ``-highlight`` | Disable highlighting placeholders in the editor. |
+---------------------------+------------------------------------------------------------+
| ``-require_other`` | Disable requiring sub-messages to have an ``other`` |
| | selector. |
+---------------------------+------------------------------------------------------------+
| ``-submessage_selectors`` | Skip checking that sub-message selectors match the source. |
+---------------------------+------------------------------------------------------------+
| ``-types`` | Skip checking that placeholder types match the source. |
+---------------------------+------------------------------------------------------------+
| ``-extra`` | Skip checking that no placeholders are present that were |
| | not present in the source string. |
+---------------------------+------------------------------------------------------------+
| ``-missing`` | Skip checking that no placeholders are missing that were |
| | present in the source string. |
+---------------------------+------------------------------------------------------------+

Additionally, when ``strict-xml`` is not enabled but ``xml`` is enabled, you can use the
``icu-tag-prefix:PREFIX`` flag to require that all XML tags start with a specific string.
For example, the following flag will only allow XML tags to be matched if they start with
``<x:``:

.. code-block::text

icu-message-format, icu-flags:xml, icu-tag-prefix:"x:"

This would match ``<x:link>click here</x:link>`` but not ``<strong>this</strong>``.

.. seealso::

:ref:`check-formats`,
`ICU: Formatting Messages <https://unicode-org.github.io/icu/userguide/format_parse/messages/>`_,
`Format.JS: Message Syntax <https://formatjs.io/docs/core-concepts/icu-syntax/>`_


.. _check-java-format:

Java format
***********
Expand Down
1 change: 1 addition & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ Pillow>=6.0.0,<8.4.0
pyahocorasick>=1.4,<1.5
pycairo>=1.15.3
pygobject>=3.27.0
pyicumessageformat>=1.0.0
nijel marked this conversation as resolved.
Show resolved Hide resolved
pyparsing>=2.4.0,<2.5.0
python-dateutil>=2.8.1
python-redis-lock>=3.6.0,<3.8.0
Expand Down
1 change: 1 addition & 0 deletions scripts/yarn-update
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ cp node_modules/prismjs/components/prism-core.js ../../weblate/static/vendor/pri
cp node_modules/prismjs/components/prism-markup.js ../../weblate/static/vendor/prism/
cp node_modules/prismjs/components/prism-rest.js ../../weblate/static/vendor/prism/
cp node_modules/prismjs/components/prism-markdown.js ../../weblate/static/vendor/prism/
cp node_modules/prismjs/components/prism-icu-message-format.js ../../weblate/static/vendor/prism/

# Clipboard
cp node_modules/clipboard/dist/clipboard.js ../../weblate/static/vendor/
Expand Down
4 changes: 4 additions & 0 deletions weblate/checks/flags.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,10 @@
TYPED_FLAGS_ARGS["font-weight"] = single_value_flag(get_font_weight)
TYPED_FLAGS["font-spacing"] = gettext_lazy("Font spacing")
TYPED_FLAGS_ARGS["font-spacing"] = single_value_flag(int)
TYPED_FLAGS["icu-flags"] = gettext_lazy("ICU MessageFormat Flags")
TYPED_FLAGS_ARGS["icu-flags"] = multi_value_flag(str)
TYPED_FLAGS["icu-tag-prefix"] = gettext_lazy("ICU MessageFormat Tag Prefix")
TYPED_FLAGS_ARGS["icu-tag-prefix"] = single_value_flag(str)
TYPED_FLAGS["priority"] = gettext_lazy("Priority")
TYPED_FLAGS_ARGS["priority"] = single_value_flag(int)
TYPED_FLAGS["max-length"] = gettext_lazy("Maximum length of translation")
Expand Down
Loading