Syntax for Rich Text & Dynamic Templates

[jeromesimeon]

This issue is related to #346 but with a specific focus on syntax for discussion. Previous related issue #171 is closed and somewhat outdated, so I think a new issue is useful.

Requirements and proposals

The requirements for rich text and dynamic templates syntax are:

1. Handle rich-text annotations, based as much as possible on some existing spec Syntax for Rich Text & Dynamic Templates #392 (comment)
2. Handle arbitrary switching between Ergo expressions and text Syntax for Rich Text & Dynamic Templates #392 (comment)
3. Handle variables, expressions, lists, tables, based as much as possible on some existing template mechanism Syntax for Rich Text & Dynamic Templates #392 (comment)
4. Support both parse and generateText methods in Cicero Syntax for Rich Text & Dynamic Templates #392 (comment)

Some eye can be kept on backward compatibility, although this is not a strong requirement and consistency of design may take precedence.

Due diligence

• Directly related issues Syntax for Rich Text & Dynamic Templates #392 (comment)
• Breaking changes Syntax for Rich Text & Dynamic Templates #392 (comment)

[jeromesimeon]

For 1. Handle rich-text annotations, based as much as possible on some existing spec

The main candidate is markdown based on common mark. which is already used in the Markdown Editor used by Cicero UI.

Some open questions:

• Support for tables?
• Should XML-like markup be allowed or not?
• Should code blocks be allowed or not?
• How does it interact with the dynamic template support?

[jeromesimeon]

For 2. Handle arbitrary switching between Ergo expressions and texts

To use expressions inside text, quote the expressions using {{% ... %}}.
To use text inside expressions, anti-quote the text using `...`.

[mttrbrts]

I'm intrigued by the ` notation, for completeness and consistency with the conceptual use in JS, here's a third option:

This is text ` let x = 1; {{ This is text again with value ` x `}} ` 

[jeromesimeon]

I'm intrigued by the ` notation, for completeness and consistency with the conceptual use in JS, here's a third option:

This is text ` let x = 1; {{ This is text again with value ` x `}} ` 

The correspondence with code blocks is intriguing...

[jeromesimeon]

For 3. Handle variables, expressions, lists, tables, based as much as possible on some existing template mechanism

The proposal bellow is inspired by handlebars.

Note: In the range between very little language-binding (mustache) and deep integration with a language (JSX), I find handlebars the most relevant/close to what we are trying to achieve

Variables ✅

Variables are quoted with {{...}}.

{{firstName}}                       // Source variable (used for parsing and rendering)
{{deliveryDate as "MMMM, DD YYYY"}} // Source variable with date formatting

Ergo Expressions ✅

Expressions are quoted with {{% ... %}}

Many special purpose things in handlebars can be written as Ergo expressions:

{{% author.name %}}                    // Nested paths (as paths)✅
{{% roundn(rate,precision) %}}         // Helpers (as function calls)✅
{{% roundn(rate,2) %}}                 // Literals (also as function calls)✅
{{% foreach p in products
    return `Product: {{ p.name }}` %}} // Iterators✅
{{% if state = "US"
    then address.state
    else address.country %}}           // Conditionals✅
{{% if forceMajeure
    then `This is a force majeure` %}} // More conditionals ❌

Other things you can do with Ergo expressions:

{{% firstName ++ lastName %}}                // String concatenation✅
{{% enforce rate > 0.0 %}}                   // Enforce conditions ❌
etc.

Block Expressions

Block expressions of kind kind with scope scope and parameters params are quoted with {{#kind scope params}} ... {{/kind}}.

Conditionals Blocks ✅

{{#if forceMajeure}}
This is a force majeure
{{/if}}

{{#if isActive}}
  I'm active
{{else}}
  I'm not active
{{/if}}

Unordered Lists ✅

{{#ulist products}}Product {[name}} with code {[id}}{{/ulist}}

Those are rendered as markdown. From:

{ products: [
    { name: "Sponge", id: 1312 },
    { name: "Bleach", id: 341 }
  ] }

to

- Product "Sponge" with code 1312
- Product "Bleach" with code 341

Ordered Lists ✅

{{#olist products}}Product {{name}} with code {{id}}{{/olist}}

Those are rendered as markdown. From:

{ products: [
    { name: "Sponge", id: 1312 },
    { name: "Bleach", id: 341 }
  ] }

to

1. Product "Sponge" with code 1312
1. Product "Bleach" with code 341

Joined Lists ✅

You create a list with a delimiter

{{#join products ","}}{{name}}{{/join}}

To render from:

{ products: [
    { name: "Sponge", id: 1312 },
    { name: "Bleach", id: 341 }
  ] }

to render:

"Sponge","Bleach"

Clauses

Clauses can be inlined: ✅

Payment
-------
{{#clause payment}}
As consideration in full for the rights granted herein, Licensee shall pay Licensor a one-time fee in the amount of {{amountText}} ({{amount}}) upon execution of this Agreement, payable as follows: {{paymentProcedure}}.
{{/clause}}

Or can be imported: ❌

Payment
-------
{{>payment ap://[email protected]}}

Tables ❌

Note: This is a very tentative part of the proposal

{{#table products}}
  {{#cell "Name"}}{{name}}{{/cell}}
  {{#cell "Code"}}{{id}}{{/cell}}
{{/table}}

Those are rendered as markdown. From:

{ products: [
    { name: "Sponge", id: 1312 },
    { name: "Bleach", id: 341 }
  ] }

to

| Name | Code |
|---|---|
| "Sponge" | 1312 |
| "Bleach" | 341 |

Other things that could be included

Iterators Blocks ❌

Eventhough you could use an Ergo expression, you could also support:

{{#foreach products}}
- Product: {{name}}
{{/foreach}}

{{#foreach(",") products}}Product: {{name}}{{/foreach}}

With ✅

With which can be used to set a context:

{{#with author}}
{{firstName}} {{lastName}}
{{/with}}

instead of

{{% author.firstName %}} {{% author.lastName %}}

Comments ❌

{{!-- This is a comment --}}

Whitespace control ❌

{{~name}}  // Remove whitespace on left
{{name~}}  // Remove whitespace on right
{{~name~}} // Remove whitespace on both

[jeromesimeon]

For 4. Support both parse and generateText methods in Cicero

The idea is to identify the source variables in the text, and generate a parser when possible. What is handled by the parser is:

{{variableName}}  // Source variables

And those can also occur inside the following blocks:

{{#list}}...{{/list}}             // List blocks may contain variables
{{#if}}...{{/if}}                 // List blocks may contain variables
{{#with fieldName}}...{{/with}}   // With blocks may contain variables

Note that those are important for two reasons: whether the template can be used on input sample text (cicero parse) and for rendering in Cicero UI (parsable variable can be edited, while other expressions/variable are only rendered but not edited).

[jeromesimeon]

Directly relevant Issues that are addressed by the proposal:
#296
#346
#79
#20
#17
#4

Directly relevant issues that are not addressed by the proposal:
#19 (Suggestion: support, what would the syntax be?)
#16 (Suggestion: do not support)

[jeromesimeon]

Breaking changes from our current syntax:

• Variables should be changed from [{variableName}] to {{variableName}}
• Formatted variables should be changed from [{variableName as "FORMAT"}] to {{variableName as "FORMAT"}}
• Boolean variables should be changed from [{"This is a force majeure":?forceMajeure}] to {{#if forceMajeure}}This is a force majeure{{/if}}
• Nested clauses should be changed from [{#payment}]As consideration in full for the rights granted herein...[{/payment}] to {{#clause payment}}As consideration in full for the rights granted herein...{{/clause}}
• Lexer reserves {{ instead of reserving [{

[jeromesimeon]

This proposal has been implemented as part of release 0.20. Specific features that did not make the 0.20 but could be of interest have been opened as new issues. I'm closing this.