Proposal for expansion of expression language

[jaredoconnell]

Changes introduced with this PR

This PR proposes three main changes to the expression language.

See the files changed for the proposal itself.

Voting Period is 7 21 days

Voting ends Friday March 24th April 7th.

---

By contributing to this repository, I agree to the contribution guidelines.

[ghost]

I would like to add one more element: every expression language component should be built in such a way that the expression language can create a resulting schema without executing the expression itself if a schema for the referenced value is available. This will help with later adding static workflow analysis.

[jaredoconnell]

I would like to add one more element: every expression language component should be built in such a way that the expression language can create a resulting schema without executing the expression itself if a schema for the referenced value is available. This will help with later adding static workflow analysis.

Especially since you're already implementing it, and it is required regardless as an implementation detail, I'll leave that out of the proposal. The proposed changes in this proposal change how the users create and interact with workflows.

[ghost]

Yes please!

[dustinblack]

I assume this is a breaking change, particularly:

• Make subexpressions default within map access since it is unintuitive as-is, removing the old () syntax. Moving from map[(sub-expression)] to map[sub-expression]

The new functions and operators I suppose could add functionality in a non-breaking way.

IIUC this would cause a workflow built before the change to be incompatible with the engine after the change, and new workflows will not be backward-compatible. This will require some housekeeping across the workflows we already have published, and a good set of release notes. Documentation should be a concurrent requirement with the new engine release, when ready.

[jaredoconnell]

I assume this is a breaking change, particularly:

• Make subexpressions default within map access since it is unintuitive as-is, removing the old () syntax. Moving from map[(sub-expression)] to map[sub-expression]

The new functions and operators I suppose could add functionality in a non-breaking way.

IIUC this would cause a workflow built before the change to be incompatible with the engine after the change, and new workflows will not be backward-compatible. This will require some housekeeping across the workflows we already have published, and a good set of release notes. Documentation should be a concurrent requirement with the new engine release, when ready.

While the () removal is likely a breaking change, no one has used it as far as I am aware. It's an advanced use case that has not been taken advantage of.

The main documentation change is that this deviates from JSONPATH.

[mfleader]

I like the proposal. While domain specific programming languages (DSL) generally need to be targeted at non-software developers, many of the most used DSLs, like SQL, OS shells, and even Make, have all incorporated some combination of functionality from general purpose programming languages (GPL) like functions, binary comparison operators, and/or mathematical expressions, so it seems reasonable that these features would facilitate Arcaflow's expression language to meet the complexity of our end users' needs.

[dustinblack]

I'm taking the liberty to extend the voting date on this to April 7 since the process is still new for us and many of us were distracted by an internal event.