Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update selection-declaration design doc based on mtg / issue discussion #867

Merged
merged 1 commit into from
Aug 26, 2024

Conversation

echeran
Copy link
Collaborator

@echeran echeran commented Aug 22, 2024

Updates design doc prompted by meeting discussions as well as #824 comments.

Comment on lines +143 to +149
A `:gender` function can return the person's gender,
but a `:personName` person name formatter function formats the name.
```
.match {$person :gender}
male {{Bienvenido {$person :personName}}}
female {{Bienvenida {$person :personName}}}
other {{Le damos la bienvenida {$person :personName}}}
Copy link
Member

@aphillips aphillips Aug 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be clearer if we show separate annotation?

.input {$person :personName}
.match {$person :gender}
male {{Does {$person} print 'male' or the person's name?}}
* {{This example is the same as use case 6}}

... or is that not your intention?

Copy link
Collaborator

@mihnita mihnita Aug 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't find this to be clear.
It is in fact what we have today.

And depending on the mental model of the reader, it is unclear if the {$person} inside the message is the :gender one or the :personName one.

It is in fact what started the issue, that I can do stuff like this:

.input {$count :integer}
.match {$count :number minFractionalDigits=2}
 ...

For clarity I can imagine two options:

  1. One has a declare a .local, which gives us a different name:
.local $theName = {$person :personName}
.match {$person :gender}
male {{Does {$theName} print 'male' or the person's name?}}
* {{This example is the same as use case 6}}

OR

  1. The .match cannot have any function / parameters
.input {$person :personName}
.match {$person}
male {{Does {$person} print 'male' or the person's name?}}
* {{This example is the same as use case 6}}

Case 2 would not be useful in this case, with gender, but would be when we want consistency, in plural

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


We already decided for immutability in .local and .input
WDYT about thinking of .match also as some kind of immutable?

These would be the rules:

if .match on $foo has any function / attributes / options on it
    if there was a .local $foo = {...} or .input {$foo ...} before
        ERROR
    else
        $foo is "bound" to the function / attributes / options in .match for the rest of the message
        // This gives us consistency between the :number selection / formatting
else
    if there was a .local $foo = {...} or .input {$foo ...} before, with a function on it
        use that exact definition ("binding") to do the selection // What we do today
    else
        ERROR, because the items on which we select must always have a function
        // So that we know what we select on, what keys might be valid, etc.
        // This is something we agreed a long time ago, useful for both linting, L10N tools, and human translators

Copy link
Collaborator

@mihnita mihnita Aug 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exercising the idea above:


.input {$person :name}
.match {$person :gender}

=> ERROR.
$person is "bound" to two different types / transforms / functions
Or .match tries to change an immutable .input, if you want to think of it in terms of immutability


.local $personName = {$person :name}
.match {$person :gender}
   .... {Here we can use both {$personName} and {$person} (second one kind of useless, but clear) }

=> CORRECT
We have two "variables", personName and person, no conflicts, everything is clear


.match {$count :number minFractionalDigits=2}
   .... {Here we can use {$count} and it means it is formatted as described in .match, good}

CORRECT


.input {$count :number minFractionalDigits=2}
.match {$count}
   .... {Here we can use {$count} and it means it is formatted as described in .match, good}

CORRECT


.input {$count :number minFractionalDigits=2}
.match {$count :number}
   .... {Here we can use {$count} and it means it is formatted as described in .match, good}

ERROR
Even if the function name is the same, it is still confusing.
Is .match {$count :number} inheriting the options from .input or not?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My intention is to highlight that selection and formatting may require different functions. My example accomplishes that straightforwardly without getting into tangential questions about function composition, tradeoffs of concision vs. simplicity, etc.

I was initially confused by your example, even though it's more concise, because it wasn't making sense to me. If .input is binding the value of the expression {$person :personName} back to $person, then we no longer have the full person object accessible to provide to the :gender function in the subsequent line performing selection: .match {$person :gender}. Am I reading that right? If so, that of course highlights the outstanding question of how we should define function composition, but that's a separate topic.

Also, the example might engender the reader to want to declare a .local person-name {$person :personName} for all the repetition in the patterns, but then if they do, they would realize that such refactoring is independent of selection. So the idea of requiring selection to take place only on variables (the design doc alternative named "Match on variables instead of expressions"), which in turn would require a .local declaration for the selector expression {$person :gender}, would not buy us anything in terms of consistency within the message because there was never a linkage between the selector expression function and pattern placeholder formatting function.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mihnita Having read your logic and explanation, it makes sense and I personally wouldn't mind this sort of conditional mutability, but as I've stated elsewhere I just see this as overly complicated and most developers are not going to invest the time to fully grasp what will happen under these different conditions, which ultimately just trades slightly increasing flexibility and conciseness for much more confusion and errors.

I think something more rigid like "selection never mutates" is incredibly clear, and barely harms the experience, if at all. You can repeat your expressions or use an input/ local.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this discussion is about text that @echeran is helpfully suggesting in the design document... a document that, I'll point out, talks about all of the issues on this thread and whose point is to illustrate choices we might make to resolve the problems it raises. @echeran has answered my question, so I'm happy to merge this so that we can have the actual discussion 😄

@aphillips aphillips added design Design principles, decisions formatting labels Aug 23, 2024
@macchiati
Copy link
Member

I can think of one slightly modified option

Much of objection to the 'mutating by match' option seems to be caused by the somewhat unusual case with two selectors on the same variable.

.match {$num ..x..}{$num ..y..}
a b {{which {$num} is it?}}

I think think of two options to address that.

Option 1. Forbid two of the same variables in the same .match.

That is, the above example would need to be reworked to:

.local $num2 {$num}
.match {$num ..x..}{$num2 ..y..}
a b {{It is the second, {$num2}.}}

or the equivalent

.local $num2 {$num ..y..}
.match {$num ..x..}{$num2}
a b {{It is the second, {$num2}.}}

Option 2. A slight amendment to "Provide a #-like Feature"

If you have two identical variables in the match statement, use _N to distinguish.

.match {$num ..x..}{$num ..y..}
a b {{It is the second, {$num_2}.}}

@echeran
Copy link
Collaborator Author

echeran commented Aug 23, 2024

Responding to what @macchiati said:

Much of objection to the 'mutating by match' option seems to be caused by the somewhat unusual case with two selectors on the same variable.

.match {$num ..x..}{$num ..y..} a b {{which {$num} is it?}}

I disagree; I think the crux of the matter is actually about how we treat the formatting/transformation for selection as compared to the formatting for display of placeholders in a pattern. Are these things the same or not?

See my comment in the parent PR in which I describe the undesirable complexities introduced by the various proposed options as a consequence of their attempts to create concision in support of the cases when -- or require declarations that always assert & enforce -- the idea that formatting for selection and formatting for display are the same. I assert that they're not the same, and this PR is providing another counter example to the design doc, and calling out the strong concerns of complexity mentioned by others in issues and the meeting.

@macchiati
Copy link
Member

I don't think anyone is saying that selection should be literally based on the formatted string (that seems to be what you are saying by "formatting for selection and formatting for display are the same"). What I'm saying is that selection should be consonant with the formatted string — for functions that are both selectors and formatters.

What constitutes "consonant with" will vary according to the function. For :number, that means that the digits and decimal position within them are mapped to either a plural category or an ASCII literal (compact numbers and scientific notation are slightly different; I won't go into that here).

@echeran
Copy link
Collaborator Author

echeran commented Aug 24, 2024

re: @macchiati

I don't think anyone is saying that selection should be literally based on the formatted string (that seems to be what you are saying by "formatting for selection and formatting for display are the same").

That's not what I am saying overall, although yes that particular sentence was unclear and misleading because I was too terse. The point I was making in the parent PR comment is that the design of the alternatives in the design doc lean in that direction -- either they are designed for or are inspired to better support cases where the function is the same and the options are similar. Hopefully I spilled enough words for clarity there.

What I'm saying is that selection should be consonant with the formatted string — for functions that are both selectors and formatters. What constitutes "consonant with" will vary according to the function.

Right, and I agree that there is a level of nuance to the situation that needs to be considered as we decide things. My concern is how much complexity the design doc's alternatives would add to the mental model needed to make sense of a message, and that's separate from the question of how well those designs would support cases where the selection function & display formatting function differ.

@aphillips aphillips merged commit 92d8940 into unicode-org:main Aug 26, 2024
1 check passed
@echeran echeran deleted the selection-declaration-edits branch August 26, 2024 16:50
aphillips added a commit that referenced this pull request Oct 26, 2024
* Create notes-2024-08-19.md

* Accept attributes design & remove spec note (#845)

* Accept attributes design & remove spec note

* Disallow duplicate attribute names (closes #756)

* Add link to contextual options PR

* Add more prose to tag example text

Co-authored-by: Addison Phillips <[email protected]>

* Mention attribute validity condition in the **_valid_** definition

---------

Co-authored-by: Addison Phillips <[email protected]>

* Update selection-declaration design doc based on mtg / issue discussion (#867)

* Add tests for pattern selection (#863)

* Add tests for pattern selection

* Add missing errors

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>

* Add Duplicate Variant to table in test/README.md (#861)

* Add new selection-declaration alternative: Require annotation of selector variables in placeholders (#860)

* Add new selection-declaration alternative: Require annotation of selector variables in placeholders

* Improve examples

* Switch example order

* Update the stability policy (#834)

* Update the stability policy

Based on discussion in the 2024-07-22 call and in PR #829, update the stability policy.

* A deeper, more thorough rewrite

- Standardizes the phrasing completely.
- Moves all potential future changes (which are not, after all, stability policies) to an "important" block
- Removes duplication
- Separates functions, options, and option values into separate guarantees
- Clarifies the note about formatting changing over time

* Update spec/README.md

Co-authored-by: Tim Chevalier <[email protected]>

* Update spec/README.md

Co-authored-by: Eemeli Aro <[email protected]>

* remove well-formed

* Update spec/README.md

---------

Co-authored-by: Tim Chevalier <[email protected]>
Co-authored-by: Eemeli Aro <[email protected]>

* Refine error handling text (#816)

* Refine error handling text

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Update fallback text

* Turn bullet point list into paragraphs

* Be more mighty

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>

* Create notes-2024-08-26.md

* Select "Match on variables instead of expressions" for selection-declarations (#824)

* Select "Match on variables instead of expressions" for selection-declarations

* Add hybrid option to selection-declaration.md (#870)

* Add hybrid option to selection-declaration.md

* Update selection-declaration.md

fixed glitch in original edit

* Update selection-declaration.md

* Apply suggestions from code review

Fixing typos

Co-authored-by: Addison Phillips <[email protected]>

* Update selection-declaration.md

* Update exploration/selection-declaration.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update exploration/selection-declaration.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update exploration/selection-declaration.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>
Co-authored-by: Eemeli Aro <[email protected]>

* Update selection-declaration.md

---------

Co-authored-by: Mark Davis <[email protected]>
Co-authored-by: Addison Phillips <[email protected]>

* Fix "Allow immutable input declarative selectors" example (#874)

* Update README.md (#875)

* Update README.md

* Update README.md

* [DESIGN] Update bidi design document to show proposed design (#871)

* [DESIGN] Update bidi design document to show proposed design

The design I actually think we should adopt is the "hybrid approaches" one. This is a necessary first step on the highway to UAX31 compliance and I think is responsibly contained/managed. It is a hybrid approach, in that it permits testable strict implementations to be created (particularly for message serialization).

This PR consists of moving text around. I added one "pro" to one option also.

* Address comments

* Miscellaneous test fixes (#862)

* Add missing expected bad-selector errors

* Fix expected parts for unsupported-statement test

* Add a few new tests for leading-whitespace and duplicate-variant

* Add tests for escaped-char changes made in #743

* Fix tests for attributes with variable values

* Update contributing and joining info (#876)

* Update contributing and joining info

* Update README.md

* Update CONTRIBUTING.md

* Restore CLA copy

* Clarify error & fallback handling (#879)

* Clarify error & fallback handling

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Select last rather than first attribute

* Drop mention of "starting with Pattern Selection"

* Attributes can't change the formatted output

* Use "nor" instead of "or" regarding attribute restrictions

---------

Co-authored-by: Addison Phillips <[email protected]>

* Clarify rule selection (#878)

* Clarify rule selection

Fixes #868 

This adds normative SHOULD language to using CLDR plural and ordinal data, which was intended originally.

- clarifies that keyword selection follows exact match
- clarifies the purpose of rule-based selection
- makes non-CLDR-based implementation permitted

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Eemeli Aro <[email protected]>

* [DESIGN] Maintaining the Standard, Optional and Unicode Namespace Function Sets (#634)

* Design doc to capture registry maintenance

* Update maintaining-registry.md

* Update exploration/maintaining-registry.md

Co-authored-by: Tim Chevalier <[email protected]>

* Update exploration/maintaining-registry.md

Co-authored-by: Tim Chevalier <[email protected]>

* Add user stories, small updates to RGI

* Update exploration/maintaining-registry.md

* Adding additional detail

* Remove machine readable registry; update prose

* Update maintaining-registry.md

* Further development work

* Update to change format and naming

Per the 2024-08-19 call, we decided to switch towards a specification-per-function model, with statuses. This commit includes the initial set of changes to try and implement this.

* Address some comments.

---------

Co-authored-by: Tim Chevalier <[email protected]>

* Create notes-2024-09-09.md

* Fix a typo in an example (#880)

The upcoming work to implement resolved value might make this patch unnecessary or obsolete, but fixing the typo (missing `{`/`}` around the variable in the pattern) just in case

* Remove forward-compatibility promise and all reserved & private syntax (#883)

* Remove forwards compatibility from stability guarantee

* Drop reserved statements and expressions

* Drop private-use annotations

* Update tests

* Clarify that deprecation is not removal

* Match on variables instead of expressions (#877)

* Match on variables instead of expressions

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Apply suggestions from code review

* Add missing test changes noticed during implementation

* Empty commit to re-trigger CLA check

---------

Co-authored-by: Addison Phillips <[email protected]>

* Create notes-2024-09-10.md

* Add bidi support and address UAX31/UTS55 requirements (#884)

* Add bidi support and address UAX31/UTS55 requirements

Adds the bidi strong marks ALM, RLM, and LRM plus the bidi isolate controls LRI, RLI, FSI, and PDI to the syntax.

Formally defines optional vs. non-optional whitespace.

Non-optional whitespace must include at least one whitespace character. Optional whitespace may contain only bidi marks (which are invisible)

* Update syntax.md including text from previous PR

* Repair the guidance on strongly directional marks

Include ALM and better specify how to use the marks.

* Fix formatting of the "important"

* Add bidi characters to description of whitespace.

* Permit bidi in a few more places

Add optional whitespace at the start of `variant`

Add optional whitespace around `quoted-pattern`

These changes result in allowing bidi around keys and quoted patterns as intended.

* Update syntax.md ABNF

* Update formatting.md

- Add a note about the difference between formatting and message syntax.
- Clarify the sentence about message directionality.

* Address comment about name/identifier

* Address comments related to bidi in `name`

* Fix variable's location

* Address comment about the list of LRI/PDI targets

* One character typo :-P

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Address comments about rule R3a-1

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Address comment about U+061C

* Change [o]wsp => `o` or `s`

* Match syntax spec to abnf

* Remove *

* Update syntax.md

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/message.abnf

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/message.abnf

Co-authored-by: Eemeli Aro <[email protected]>

* Update syntax.md

* Update spec/message.abnf

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Eemeli Aro <[email protected]>

* Specify `bad-option` for bad digit size option values (#882)

* Specify `bad-option` for bad digit size option values

Fixes #739

* adopt 'non-negative integer'

* Create notes-2024-09-16.md

* Address name and literal equality (#885)

* Address name and literal equality

This change defines equality as discussed in the 2024-09-09 teleconference in the following ways:

- It defines _name_ equality as being under NFC
- It defines _literal_ equality as explicitly **not** under NFC
- It moves _name_ before _identifier_ in that section of text to avoid a forward definition.

Note that this deviates from discussion in 2024-09-09's call in that we didn't discuss literals at length. It also doesn't discuss non-name/non-literal values, which I'll point out are limited to ASCII sequences such as keywords.

* Typo fix

* Add a note about not requiring implementations to actually normalize

* Implement changes dicussed in 2024-09-16 call.

- Make _key_ require NFC for uniqueness/comparison
- Add a note about NFC
- Make _literal_ **_not_** define equality
- Make text in _name_ identical to that in _key_ for consistency

* Update formatting.md to include keys in NFC

* Address comments

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Eemeli Aro <[email protected]>

* Update list of normative changes during the LDML45 period (#890)

* Fix typos in data-model-errors tests (#892)

Fix #886

* Update note on exact numeric match for v46 (#891)

Addresses #887 

Non-normative changes to the notes specifically part of LDML46

* Fix attribute value to be literal (#894)

Fixes #893

* Create notes-2024-09-30.md

* Add Resolved Values and Function Handler sections to formatting (#728)

* Add Resolved Values section to formatting

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Tim Chevalier <[email protected]>

* Linkify "resolved value"

* Add some examples & explicitly allow wrapping input values

* No throw, only emit

Co-authored-by: Tim Chevalier <[email protected]>

* Add section on Function Handlers, defining the term

* Apply suggestions from code review

* Rephrase initial resolved value definition

* Update spec/formatting.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update resolved value definition again

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Tim Chevalier <[email protected]>
Co-authored-by: Addison Phillips <[email protected]>

* Define function composition for :number and :integer values (#823)

* Define function composition for :number and :integer values

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Add operand option priority example

* Add apostrophes'

Co-authored-by: Tim Chevalier <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>
Co-authored-by: Tim Chevalier <[email protected]>

* Create notes-2024-10-07.md

* Apply NFC normalization during :string key comparison (#905)

* Apply NFC normalization during :string key comparison

* Add link to UAX#15

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>

* Add tests for changes due to bidi/whitespace (#902)

* Add tests for changes due to bidi/whitespace

* Correct output

* Make erroneous test a syntax error

* Define function composition for date/time values (#814)

* Define function composition for date/time values

* Apply suggestions from code review

Co-authored-by: Stanisław Małolepszy <[email protected]>

* Drop the "only"

* Update spec/registry.md

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Make :date and :time composition implementation-defined

---------

Co-authored-by: Stanisław Małolepszy <[email protected]>
Co-authored-by: Addison Phillips <[email protected]>

* DESIGN: Add alternative designs to the design doc on function composition (#806)

* DESIGN: Add a sequel to the design doc on function composition

This document sketches out some alternatives for the machinery
provided to enable function composition.

The goal is to provide an exhaustive list of alternatives.

* Remove 'part 2' document and move contents to the end of part 1

* Revise introduction to reflect the changed goal

* Edited for conciseness

* Further edits for conciseness

* Give a name to InputType and use it

* Refer to motivating examples

* Update function-composition-part-1.md status

Per 2024-10-14 telecon

* Create notes-2024-10-14.md

* Add test for :integer and :number composition (#907)

* Fix `:integer` option `useGrouping` values (#912)

I noticed that `:integer` does not include the "never" value for the option `useGrouping`. This is a bug.

* Drop syntax note on additional bidi changes (#910)

Drop syntax note on addition bidi changes

* Add tests for changes due to #885 (name/literal equality) (#904)

* Add tests for changes due to #885 (name/literal equality)

* Update test/tests/functions/string.json

Co-authored-by: Eemeli Aro <[email protected]>

* Update test/tests/syntax.json

Co-authored-by: Eemeli Aro <[email protected]>

* Update test/tests/functions/string.json

Co-authored-by: Eemeli Aro <[email protected]>

* Added tests for reordering and special case mapping

* Add another selection test

---------

Co-authored-by: Eemeli Aro <[email protected]>

* Add u: options namespace (#846)

* Move spec/registry.md -> spec/registry/default.md

* Add Unicode Registry definition

* Refer to BCP47, add note about only requiring normal tags

* Call it a namespace

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Fix test file reference

Co-authored-by: Tim Chevalier <[email protected]>

* Apply suggestions from code review

* Update spec/u-namespace.md

Co-authored-by: Eemeli Aro <[email protected]>

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Add mention of functions to namespace description

---------

Co-authored-by: Addison Phillips <[email protected]>
Co-authored-by: Tim Chevalier <[email protected]>

* Define function composition for :string values (#798)

* Define function composition for :string values

* Update spec/registry.md as suggested by @stasm in #814

* Drop the "only"

* Update text following code review comments

---------

Co-authored-by: Addison Phillips <[email protected]>

* Drop data model request for feedback on "name" (#909)

* Allow surrogates in content, issue #895 (#906)

* Allow surrogates in content, issue #895

* Grammar and typos, linkify terms, make into a note, and fix 2119 keywords

Thanks Addison!

Co-authored-by: Addison Phillips <[email protected]>

* Not using "localizable elements"

Co-authored-by: Addison Phillips <[email protected]>

* Keep syntax.md in sync with message.abnf

* Added note about surrogates to quoted literals

* Moved the note about surrogates from Security Considerations to The Message

* Update spec/syntax.md

* Update spec/syntax.md

* Italicize  in a couple of places

* Implemeted more (all?) feedback from review

---------

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Eemeli Aro <[email protected]>
Co-authored-by: Elango Cheran <[email protected]>
Co-authored-by: Tim Chevalier <[email protected]>
Co-authored-by: Mark Davis <[email protected]>
Co-authored-by: Danny Gleckler <[email protected]>
Co-authored-by: Steven R. Loomis <[email protected]>
Co-authored-by: Stanisław Małolepszy <[email protected]>
Co-authored-by: Eemeli Aro <[email protected]>
Co-authored-by: Mihai Nita <[email protected]>
aphillips added a commit that referenced this pull request Nov 4, 2024
* [DESIGN] Number selection design refinements

This is to build up and capture technical considerations for how to address the issues raised by @eemeli's PR #842.

* Update examples to match changes to syntax

Also responds to the long discussion with @eemeli about significant digits by removing from the example.

* Address 2024-09-16 call comments

This changes the status to "Re-Opened" and adds a link to the PR. Expect to merge this imminently, although discussion on number selection remains.

* Update exploration/number-selection.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update from main (#914)

* Create notes-2024-08-19.md

* Accept attributes design & remove spec note (#845)

* Accept attributes design & remove spec note

* Disallow duplicate attribute names (closes #756)

* Add link to contextual options PR

* Add more prose to tag example text

Co-authored-by: Addison Phillips <[email protected]>

* Mention attribute validity condition in the **_valid_** definition

---------

Co-authored-by: Addison Phillips <[email protected]>

* Update selection-declaration design doc based on mtg / issue discussion (#867)

* Add tests for pattern selection (#863)

* Add tests for pattern selection

* Add missing errors

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>

* Add Duplicate Variant to table in test/README.md (#861)

* Add new selection-declaration alternative: Require annotation of selector variables in placeholders (#860)

* Add new selection-declaration alternative: Require annotation of selector variables in placeholders

* Improve examples

* Switch example order

* Update the stability policy (#834)

* Update the stability policy

Based on discussion in the 2024-07-22 call and in PR #829, update the stability policy.

* A deeper, more thorough rewrite

- Standardizes the phrasing completely.
- Moves all potential future changes (which are not, after all, stability policies) to an "important" block
- Removes duplication
- Separates functions, options, and option values into separate guarantees
- Clarifies the note about formatting changing over time

* Update spec/README.md

Co-authored-by: Tim Chevalier <[email protected]>

* Update spec/README.md

Co-authored-by: Eemeli Aro <[email protected]>

* remove well-formed

* Update spec/README.md

---------

Co-authored-by: Tim Chevalier <[email protected]>
Co-authored-by: Eemeli Aro <[email protected]>

* Refine error handling text (#816)

* Refine error handling text

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Update fallback text

* Turn bullet point list into paragraphs

* Be more mighty

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>

* Create notes-2024-08-26.md

* Select "Match on variables instead of expressions" for selection-declarations (#824)

* Select "Match on variables instead of expressions" for selection-declarations

* Add hybrid option to selection-declaration.md (#870)

* Add hybrid option to selection-declaration.md

* Update selection-declaration.md

fixed glitch in original edit

* Update selection-declaration.md

* Apply suggestions from code review

Fixing typos

Co-authored-by: Addison Phillips <[email protected]>

* Update selection-declaration.md

* Update exploration/selection-declaration.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update exploration/selection-declaration.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update exploration/selection-declaration.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>
Co-authored-by: Eemeli Aro <[email protected]>

* Update selection-declaration.md

---------

Co-authored-by: Mark Davis <[email protected]>
Co-authored-by: Addison Phillips <[email protected]>

* Fix "Allow immutable input declarative selectors" example (#874)

* Update README.md (#875)

* Update README.md

* Update README.md

* [DESIGN] Update bidi design document to show proposed design (#871)

* [DESIGN] Update bidi design document to show proposed design

The design I actually think we should adopt is the "hybrid approaches" one. This is a necessary first step on the highway to UAX31 compliance and I think is responsibly contained/managed. It is a hybrid approach, in that it permits testable strict implementations to be created (particularly for message serialization).

This PR consists of moving text around. I added one "pro" to one option also.

* Address comments

* Miscellaneous test fixes (#862)

* Add missing expected bad-selector errors

* Fix expected parts for unsupported-statement test

* Add a few new tests for leading-whitespace and duplicate-variant

* Add tests for escaped-char changes made in #743

* Fix tests for attributes with variable values

* Update contributing and joining info (#876)

* Update contributing and joining info

* Update README.md

* Update CONTRIBUTING.md

* Restore CLA copy

* Clarify error & fallback handling (#879)

* Clarify error & fallback handling

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Select last rather than first attribute

* Drop mention of "starting with Pattern Selection"

* Attributes can't change the formatted output

* Use "nor" instead of "or" regarding attribute restrictions

---------

Co-authored-by: Addison Phillips <[email protected]>

* Clarify rule selection (#878)

* Clarify rule selection

Fixes #868 

This adds normative SHOULD language to using CLDR plural and ordinal data, which was intended originally.

- clarifies that keyword selection follows exact match
- clarifies the purpose of rule-based selection
- makes non-CLDR-based implementation permitted

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Eemeli Aro <[email protected]>

* [DESIGN] Maintaining the Standard, Optional and Unicode Namespace Function Sets (#634)

* Design doc to capture registry maintenance

* Update maintaining-registry.md

* Update exploration/maintaining-registry.md

Co-authored-by: Tim Chevalier <[email protected]>

* Update exploration/maintaining-registry.md

Co-authored-by: Tim Chevalier <[email protected]>

* Add user stories, small updates to RGI

* Update exploration/maintaining-registry.md

* Adding additional detail

* Remove machine readable registry; update prose

* Update maintaining-registry.md

* Further development work

* Update to change format and naming

Per the 2024-08-19 call, we decided to switch towards a specification-per-function model, with statuses. This commit includes the initial set of changes to try and implement this.

* Address some comments.

---------

Co-authored-by: Tim Chevalier <[email protected]>

* Create notes-2024-09-09.md

* Fix a typo in an example (#880)

The upcoming work to implement resolved value might make this patch unnecessary or obsolete, but fixing the typo (missing `{`/`}` around the variable in the pattern) just in case

* Remove forward-compatibility promise and all reserved & private syntax (#883)

* Remove forwards compatibility from stability guarantee

* Drop reserved statements and expressions

* Drop private-use annotations

* Update tests

* Clarify that deprecation is not removal

* Match on variables instead of expressions (#877)

* Match on variables instead of expressions

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Apply suggestions from code review

* Add missing test changes noticed during implementation

* Empty commit to re-trigger CLA check

---------

Co-authored-by: Addison Phillips <[email protected]>

* Create notes-2024-09-10.md

* Add bidi support and address UAX31/UTS55 requirements (#884)

* Add bidi support and address UAX31/UTS55 requirements

Adds the bidi strong marks ALM, RLM, and LRM plus the bidi isolate controls LRI, RLI, FSI, and PDI to the syntax.

Formally defines optional vs. non-optional whitespace.

Non-optional whitespace must include at least one whitespace character. Optional whitespace may contain only bidi marks (which are invisible)

* Update syntax.md including text from previous PR

* Repair the guidance on strongly directional marks

Include ALM and better specify how to use the marks.

* Fix formatting of the "important"

* Add bidi characters to description of whitespace.

* Permit bidi in a few more places

Add optional whitespace at the start of `variant`

Add optional whitespace around `quoted-pattern`

These changes result in allowing bidi around keys and quoted patterns as intended.

* Update syntax.md ABNF

* Update formatting.md

- Add a note about the difference between formatting and message syntax.
- Clarify the sentence about message directionality.

* Address comment about name/identifier

* Address comments related to bidi in `name`

* Fix variable's location

* Address comment about the list of LRI/PDI targets

* One character typo :-P

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Address comments about rule R3a-1

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Address comment about U+061C

* Change [o]wsp => `o` or `s`

* Match syntax spec to abnf

* Remove *

* Update syntax.md

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/message.abnf

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/message.abnf

Co-authored-by: Eemeli Aro <[email protected]>

* Update syntax.md

* Update spec/message.abnf

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Eemeli Aro <[email protected]>

* Specify `bad-option` for bad digit size option values (#882)

* Specify `bad-option` for bad digit size option values

Fixes #739

* adopt 'non-negative integer'

* Create notes-2024-09-16.md

* Address name and literal equality (#885)

* Address name and literal equality

This change defines equality as discussed in the 2024-09-09 teleconference in the following ways:

- It defines _name_ equality as being under NFC
- It defines _literal_ equality as explicitly **not** under NFC
- It moves _name_ before _identifier_ in that section of text to avoid a forward definition.

Note that this deviates from discussion in 2024-09-09's call in that we didn't discuss literals at length. It also doesn't discuss non-name/non-literal values, which I'll point out are limited to ASCII sequences such as keywords.

* Typo fix

* Add a note about not requiring implementations to actually normalize

* Implement changes dicussed in 2024-09-16 call.

- Make _key_ require NFC for uniqueness/comparison
- Add a note about NFC
- Make _literal_ **_not_** define equality
- Make text in _name_ identical to that in _key_ for consistency

* Update formatting.md to include keys in NFC

* Address comments

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Eemeli Aro <[email protected]>

* Update list of normative changes during the LDML45 period (#890)

* Fix typos in data-model-errors tests (#892)

Fix #886

* Update note on exact numeric match for v46 (#891)

Addresses #887 

Non-normative changes to the notes specifically part of LDML46

* Fix attribute value to be literal (#894)

Fixes #893

* Create notes-2024-09-30.md

* Add Resolved Values and Function Handler sections to formatting (#728)

* Add Resolved Values section to formatting

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Tim Chevalier <[email protected]>

* Linkify "resolved value"

* Add some examples & explicitly allow wrapping input values

* No throw, only emit

Co-authored-by: Tim Chevalier <[email protected]>

* Add section on Function Handlers, defining the term

* Apply suggestions from code review

* Rephrase initial resolved value definition

* Update spec/formatting.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update resolved value definition again

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Tim Chevalier <[email protected]>
Co-authored-by: Addison Phillips <[email protected]>

* Define function composition for :number and :integer values (#823)

* Define function composition for :number and :integer values

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Add operand option priority example

* Add apostrophes'

Co-authored-by: Tim Chevalier <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>
Co-authored-by: Tim Chevalier <[email protected]>

* Create notes-2024-10-07.md

* Apply NFC normalization during :string key comparison (#905)

* Apply NFC normalization during :string key comparison

* Add link to UAX#15

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>

* Add tests for changes due to bidi/whitespace (#902)

* Add tests for changes due to bidi/whitespace

* Correct output

* Make erroneous test a syntax error

* Define function composition for date/time values (#814)

* Define function composition for date/time values

* Apply suggestions from code review

Co-authored-by: Stanisław Małolepszy <[email protected]>

* Drop the "only"

* Update spec/registry.md

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Make :date and :time composition implementation-defined

---------

Co-authored-by: Stanisław Małolepszy <[email protected]>
Co-authored-by: Addison Phillips <[email protected]>

* DESIGN: Add alternative designs to the design doc on function composition (#806)

* DESIGN: Add a sequel to the design doc on function composition

This document sketches out some alternatives for the machinery
provided to enable function composition.

The goal is to provide an exhaustive list of alternatives.

* Remove 'part 2' document and move contents to the end of part 1

* Revise introduction to reflect the changed goal

* Edited for conciseness

* Further edits for conciseness

* Give a name to InputType and use it

* Refer to motivating examples

* Update function-composition-part-1.md status

Per 2024-10-14 telecon

* Create notes-2024-10-14.md

* Add test for :integer and :number composition (#907)

* Fix `:integer` option `useGrouping` values (#912)

I noticed that `:integer` does not include the "never" value for the option `useGrouping`. This is a bug.

* Drop syntax note on additional bidi changes (#910)

Drop syntax note on addition bidi changes

* Add tests for changes due to #885 (name/literal equality) (#904)

* Add tests for changes due to #885 (name/literal equality)

* Update test/tests/functions/string.json

Co-authored-by: Eemeli Aro <[email protected]>

* Update test/tests/syntax.json

Co-authored-by: Eemeli Aro <[email protected]>

* Update test/tests/functions/string.json

Co-authored-by: Eemeli Aro <[email protected]>

* Added tests for reordering and special case mapping

* Add another selection test

---------

Co-authored-by: Eemeli Aro <[email protected]>

* Add u: options namespace (#846)

* Move spec/registry.md -> spec/registry/default.md

* Add Unicode Registry definition

* Refer to BCP47, add note about only requiring normal tags

* Call it a namespace

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Fix test file reference

Co-authored-by: Tim Chevalier <[email protected]>

* Apply suggestions from code review

* Update spec/u-namespace.md

Co-authored-by: Eemeli Aro <[email protected]>

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Add mention of functions to namespace description

---------

Co-authored-by: Addison Phillips <[email protected]>
Co-authored-by: Tim Chevalier <[email protected]>

* Define function composition for :string values (#798)

* Define function composition for :string values

* Update spec/registry.md as suggested by @stasm in #814

* Drop the "only"

* Update text following code review comments

---------

Co-authored-by: Addison Phillips <[email protected]>

* Drop data model request for feedback on "name" (#909)

* Allow surrogates in content, issue #895 (#906)

* Allow surrogates in content, issue #895

* Grammar and typos, linkify terms, make into a note, and fix 2119 keywords

Thanks Addison!

Co-authored-by: Addison Phillips <[email protected]>

* Not using "localizable elements"

Co-authored-by: Addison Phillips <[email protected]>

* Keep syntax.md in sync with message.abnf

* Added note about surrogates to quoted literals

* Moved the note about surrogates from Security Considerations to The Message

* Update spec/syntax.md

* Update spec/syntax.md

* Italicize  in a couple of places

* Implemeted more (all?) feedback from review

---------

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Eemeli Aro <[email protected]>
Co-authored-by: Elango Cheran <[email protected]>
Co-authored-by: Tim Chevalier <[email protected]>
Co-authored-by: Mark Davis <[email protected]>
Co-authored-by: Danny Gleckler <[email protected]>
Co-authored-by: Steven R. Loomis <[email protected]>
Co-authored-by: Stanisław Małolepszy <[email protected]>
Co-authored-by: Eemeli Aro <[email protected]>
Co-authored-by: Mihai Nita <[email protected]>

* Add serialization proposal

* Revert "Add serialization proposal"

This reverts commit 17af553.

* Revert "Update from main (#914)"

This reverts commit da9377b.

* Add serialization proposal

---------

Co-authored-by: Eemeli Aro <[email protected]>
Co-authored-by: Elango Cheran <[email protected]>
Co-authored-by: Tim Chevalier <[email protected]>
Co-authored-by: Mark Davis <[email protected]>
Co-authored-by: Danny Gleckler <[email protected]>
Co-authored-by: Steven R. Loomis <[email protected]>
Co-authored-by: Stanisław Małolepszy <[email protected]>
Co-authored-by: Eemeli Aro <[email protected]>
Co-authored-by: Mihai Nita <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
design Design principles, decisions formatting
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants