Markup: can we use argument-less functions to represent standalone display elements?

[mihnita]

No description provided.

[mihnita]

Comments migrated from the slides

{b}bold{/b}

---

Slides comment, Richard Gibson (@gibson042), 7:52 AM Apr 21

This overloading of the same syntax for singleton expressions and paired markup seems awkward... I guess it's like expressions are a special degenerate form of markup that cannot have nested content?

---

Slides comment, Eemeli Aro (@eemeli), 8:07 AM Apr 21

That's one way of looking at it, sure. Our thinking is more driven by considering within message contents for the {...} to wrap "special" stuff, which in this proposal can take one of three forms.

[eemeli]

TL;DR: Argument-less functions and standalone markup elements could be synonymous if the function registry allows for pass-through "formatting" of type+name+options entries as parts.

---

Having pondered this question recently, I think it may be useful to map out the space in which we need to answer this question.

When we parse the syntax of either argument-less function expressions or standalone markup elements into a data model, prior to resolving any registered functions or considering the values of formatting parameters, we may consider the result to be a structure with:

• The type of this node.
• A string identifier for the function or markup element name.
• A map of options, with string keys and values that are either literals or variable references.

During message resolution and formatting, we need to perform three distinct steps that are relevant:

1. When the message has multiple variants, we need to select one of them.

   • As far as I am aware, this is the primary (but not only) use case for argument-less function expressions, of being able to use a function like :platform that can access system-level information, allowing for a selector to pick e.g. between variants that use "continue" and "next", or "options" and "preferences", to align with external style guides.
   • Selecting between variants based on a standalone markup element does not really make sense, and I am not aware of anyone advocating for this.

2. We need to resolve the values of each part of the selected variant.

   • For both argument-less function expressions or standalone markup elements, this starts with resolving the values of its options.
   • When resolving a function expression, we need to call the function defined for it in the registry, and get some sort of formattable output.
   • When resolving a markup element, we also need to consult the function registry to validate the element, and e.g. filter or validate its resolved option values, even if not actually calling any custom function at this time.

3. Once we have these resolved parts, we need to format them to the requested format.

   • For function expressions, the resolved value that we got by calling the registry function is either already in the shape we are formatting to, or provides a way to format it accordingly (depending on implementation details).
   • We have not really spent much time talking about the formatting of markup elements. My current view on this is that markup elements should not necessarily be formatted as such within MF2, but that they should be emitted as a single "formatted part" together with their resolved options values.

Given all that, to me the important next question to ask is:

Is the use of argument-less function expressions in message contents rare enough that we can make it a little clumsy?

If these should be considered relatively common, then we ought to handle argument-less functions the same way the we handle function expressions that do have an argument.

On the other hand, if this is a sufficiently rare use case, I think we could optimise the syntax and its processing towards there being less difference between formatting functions and markup. To start, let's consider a slightly different syntax for markup elements, where rather than {b}bold{/b} we would use {+b}bold{-b} to mark the start and end elements. The + and - sigils are conceptually closer to our : function indicator, allowing for this to feel more like one whole set, rather than two separate worlds.

Now, what could something like {:foo opt=42} resolve and format to? Should our default assumption here be that it formats to a string "foo-of-42", or to something possibly closer to an element <foo opt="42" />? Because if we could presume the latter while allowing for the former, then I don't think we actually need to have any fundamental difference between "formatting functions" and "markup elements", outside of what's defined in the registry.

Overall, I think this view has quite a bit of similarity with ideas that @mihnita has been presenting (of there being little difference between formatting functions and markup elements) and with some thoughts that I recall @zbraniecki making, of asking whether a registered function can return an element.

To sum up, I would solve this by allowing for four different sorts of placeholders:

• {$arg :foo opt=42}
• {:foo opt=42}
• {+foo opt=42} (not valid as selector)
• {-foo} (no options, not valid as selector)

I would not mandate in the spec any particular meaning to any of these, leaving that to the function registry. It would need to answer the following questions:

• Can foo be used as a selector?
• Does foo require an argument?
• Can foo be used with +/- to start/end a span?
• When resolving and formatting, is a custom function called, or are the placeholder type, name and resolved options passed through to the output?

[markusicu]

I don't like giving certain markup special status in the syntax, and I get lost a bit in @eemeli's intro above, but his conclusion makes sense to me :-)

I think it's fine to have a function without a value (argument or literal).

I know that various kinds of markup, or style spans, are common, and have XLIFF support for open/close annotations. Eemeli's plus/minus prefixes seem ok for that, including the restrictions like "not valid as selector", and "no options" for the closing one.

I would permit arguments and literals for at least an "opening" placeholder. It could allow something like {(a) +html href=(some url)}link text{-html} -- using parentheses as literal delimiters as an example. If we were to use the same thing for making the link bold, then we would get a nested pair of placeholders with the html function. This may not be the best way to represent this, but syntactically it should be allowed, and the registry should say which function does what.

[markusicu]

PS: It might be more obvious if a function name always has the same prefix, such as : in the current discussion. The open/close indicators could be additional prefixes, such as {+:function optkey=optval}...{-:function}.

[eemeli]

I would permit arguments and literals for at least an "opening" placeholder. It could allow something like {(a) +html href=(some url)}link text{-html} -- using parentheses as literal delimiters as an example. If we were to use the same thing for making the link bold, then we would get a nested pair of placeholders with the html function. This may not be the best way to represent this, but syntactically it should be allowed, and the registry should say which function does what.

I would prefer a namespaced function name instead:

{+html.a href=(some url)}link text{-html.a}

This approach has the same expressive power, but allows for clearer nesting. For example:

{+html.a href=(some url)}link {+html.b}bold{-html.b} text{-html.a}

It would also be more straightforward for the registry to define different valid options baskets for +html.a and +html.b, compared to a single +html function having its valid options depend on the value of its argument.

Given the above preference, I remain unconvinced that there's a real-world use case for any +foo function to be able to take a positional argument.

[macchiati]

I like the namespaced function

…

On Tue, May 31, 2022, 02:48 Eemeli Aro ***@***.***> wrote: I would permit arguments and literals for at least an "opening" placeholder. It could allow something like {(a) +html href=(some url)}link text{-html} -- using parentheses as literal delimiters as an example. If we were to use the same thing for making the link bold, then we would get a nested pair of placeholders with the html function. This may not be the best way to represent this, but syntactically it should be allowed, and the registry should say which function does what. I would prefer a namespaced function name instead: {+html.a href=(some url)}link text{-html.a} This approach has the same expressive power, but allows for clearer nesting. For example: {+html.a href=(some url)}link {+html.b}bold{-html.b} text{-html.a} It would also be more straightforward for the registry to define different valid options baskets for +html.a and +html.b, compared to a single +html function having its valid options depend on the value of its argument. Given the above preference, I remain unconvinced that there's a real-world use case for any +foo function to be able to take a positional argument. — Reply to this email directly, view it on GitHub <#240 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACJLEMAEKZ2YLSYBVGIVVOLVMXN55ANCNFSM5VWCS36A> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[markusicu]

Good points about different sets of options for different markup functions, rather than using a value literal to distinguish.

In the discussion yesterday, several people liked allowing dots in function names for the purpose of namespacing custom functions (functions not defined in the CLDR MF2 registry) in Java package style, such as :com.google.fancyNumber.

For various HTML elements, we could use an underscore to visually segment a flat namespace, such as :html_a.

If we want to standardize them, we should probably try to define a "link" function rather than tying it to HTML. Links are useful in lots of environments. Same for some common styles. For things that are truly specific to HTML, we can use an html_... prefix.

@eemeli:

Given the above preference, I remain unconvinced that there's a real-world use case for any +foo function to be able to take a positional argument.

I would allow it syntactically, and note in the function registry whether a particular function takes an argument or not.

[mihnita]

I think we still need to determine if there is any conceptual difference between {html.a} and {(a) :html}
Is it a function (you choose to call it namespace).

Because the engine should be aware of it, should be able to split html.a vs tts.a
And call the html processor vs calling the tts processor (vs registered custom "processors")

If we treat it as a simple separator that is not part of the "engine" concern (like using "html_...")
I don't see how we can delegate it to various rendering engine / processor.

This is what the mf2 engine (and icu implementation) would do format a literal value:
{(20220427T1442) :datetime skeleton=yMMMdjm}

Find the function registered under the "datetime" name, and invoke it with the sting value "20220427T1442" and a map of options that contains skeleton=yMMMdjm

This is what we need to do for markup (ignore the syntax for a second):
{(a) :html href=url_to_jump_to}

100% similar: find the "function" that handles "html" and invoke it with the string "a" and a map of options that contains href=url_to_jump_to
And we invoke a different function for {(a) :tts foo=bar} (the one registered under "tts")

If we say {html.a} (or html_a or {mihai.a} / mihai_a) the (mf2 / icu) engine would need to split the mihai.a (mihai_a) into the "mihai" part, to find out who will process the "a" part.

In other words: who does the "dispatch" between the tts / html / mihai namespaces?
We can't expect developers to "register" tens or hundreds of "custom functions" for each combination namespace + tag (html_a, html_br, html_h1 and so on)

[mihnita]

TL;DR: Argument-less functions and standalone markup elements could be synonymous if the function registry allows for pass-through "formatting" of type+name+options entries as parts. After reading the whole thread several times, I still don't quite get it. Standalone and argument-less seem orthogonal.For example <img src="..." alt="..."> is standalone, is "markup element", but has arguments.

To sum up, I would solve this by allowing for four different sorts of placeholders:> {$arg :foo opt=42}  {:foo opt=42}> {+foo opt=42} (not valid as selector)> {-foo} (no options, not valid as selector)

We need more than that.What about {+bar :foo opt=42} and {-bar :foo} ?The :foo acts like a namespace. It tells us if these elements should be processed by a html engine, a markdown engine, a tts engine, etc.I also don't see a standalone tag here.

I would prefer a namespaced function name instead:> {+html.a href=(some url)}link text{-html.a}

Isn't this just syntactic sugar to :html as a function?The tag is "a", the function is :html.So why not {"+a" :html href=(some url)} ?Why invent another syntax? M. On Tuesday, May 31, 2022, 06:36:05 AM PDT, Mark Davis ***@***.***> wrote: I like the namespaced function

On Tue, May 31, 2022, 02:48 Eemeli Aro ***@***.***> wrote: I would permit arguments and literals for at least an "opening" placeholder. It could allow something like {(a) +html href=(some url)}link text{-html} -- using parentheses as literal delimiters as an example. If we were to use the same thing for making the link bold, then we would get a nested pair of placeholders with the html function. This may not be the best way to represent this, but syntactically it should be allowed, and the registry should say which function does what. I would prefer a namespaced function name instead: {+html.a href=(some url)}link text{-html.a} This approach has the same expressive power, but allows for clearer nesting. For example: {+html.a href=(some url)}link {+html.b}bold{-html.b} text{-html.a} It would also be more straightforward for the registry to define different valid options baskets for +html.a and +html.b, compared to a single +html function having its valid options depend on the value of its argument. Given the above preference, I remain unconvinced that there's a real-world use case for any +foo function to be able to take a positional argument. — Reply to this email directly, view it on GitHub <#240 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACJLEMAEKZ2YLSYBVGIVVOLVMXN55ANCNFSM5VWCS36A> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[eemeli]

Re-styled and re-indented for clarity; the email style processing seems a bit broken here.

I'll reply below regarding standalone elements; for the other matters this issue is getting rather off-topic. We should probably continue under #238?

From @mihnita:

From @eemeli:
TL;DR: Argument-less functions and standalone markup elements could be synonymous if the function registry allows for pass-through "formatting" of type+name+options entries as parts.

After reading the whole thread several times, I still don't quite get it. Standalone and argument-less seem orthogonal.For example <img src="..." alt="..."> is standalone, is "markup element", but has arguments.

To clarify, I would consider the MF2 equivalent of element attributes to be named options, rather than positional arguments. With that model, the example you give is indeed "argument-less" as <img> does not have any positional arguments.

To sum up, I would solve this by allowing for four different sorts of placeholders:

• {$arg :foo opt=42}
• {:foo opt=42}
• {+foo opt=42} (not valid as selector)
• {-foo} (no options, not valid as selector)

We need more than that.What about {+bar :foo opt=42} and {-bar :foo} ?The :foo acts like a namespace. It tells us if these elements should be processed by a html engine, a markdown engine, a tts engine, etc.I also don't see a standalone tag here.

A standalone tag representation would here be provided by {:foo opt=42}.

I would prefer a namespaced function name instead:

{+html.a href=(some url)}link text{-html.a}

Isn't this just syntactic sugar to :html as a function?The tag is "a", the function is :html.So why not {"+a" :html href=(some url)} ?Why invent another syntax?
M.

[aphillips]

This issue should be addressed by the design document on spannables. Can we close this in favor of considering our design choices there?

[eemeli]

@aphillips I've linked #541 to close this when it's merged.