Arithmetic Expressions

[springcomp]

I think the spec should support simple arithmetic operations in the way of the +, -, * and / operators.

A few things to keep in mind:

• What happens in case of illegal or overflowing operations. Does that return null or does that raise an error ?
• Parentheses would need to be involved to override precedence.
• The geek in me would love to support proper symbols for arithmetic operators in addition to their ASCII approximations:
  • – (U+2212 MINUS SIGN)
  • ÷ (U+00F7 DIVISION SIGN)
  • × (U+00D7 MULTIPLY SIGN)

[springcomp]

If we support arithmetic operators, do we need to support JSON numbers ?
Currently, JMESPath only supports integers. But we could use json-value to support operations:

foo.bar + `3.25`

Do we want to be able to write:

foo.bar + 3.25

Without back ticks?

[innovate-invent]

Currently, JMESPath only supports integers

Can you point me at where this is stated?

[springcomp]

Oh, I mean the grammar only supports specifying numbers as \-?[0-9]+ and does not support the full JSON fractional numbers.
However, JMESPath does indeed support processing JSON numbers when coming from the source JSON document.

[Thaina]

In my opinion. I would love to see a query language that don't need a back tick for constant JSON

But that would not be jmespath anymore. It would be an explicitly new language that support JSON

Ideally I would love to have a language that completely a superset of JSON. I have tried to design it in the past but have no time to continuted. But the goal is to allow writing plain JSON into the parser. Extend it with other symbol to do query and math operation

Many idea was similar as the below comment

• +
  • Number + Number : Add number
  • String + String : concat string
  • Array + Array : concat array
  • Object + Object : set merge key. Override by latter
• -
  • Number - Number : Subtract number
  • Array - Array : remove element
  • Object - Array : set remove key
• *
  • Number * Number : Multiply number
  • any * Number : Repeat any into array in number amount
  • Collection * Collection : cartesian product of each respective item
• /
  • Number / Number : Divide number
  • Collection / Collection : set intersect of the same kind of collection
• %
  • Number % Number : Mod number
  • Collection % Collection : zip of each respective item into array

ps Collection means both array and object. And respective item is just element of array but keyvaluepair of object

[innovate-invent]

I would love to see a query language that don't need a back tick for constant JSON

This is actually being proposed: #34

[innovate-invent]

If we are adding these operators to the grammar we may as well define them for non-numbers as well:

Arrays:
cartesian product: ["a", "b", "b"] * [1, 2] = [["a", 1], ["a", 2], ["b", 1], ["b", 2], ["b", 1], ["b", 2]] chaining products of lists extends the nested lists rather than creating recursive two element lists
set intersect: ["a", "b", "b"] / [1, "b"] = ["b"]
set difference: ["a", 1, "b", "b"] - ["b", 2] = ["a", 1]

Objects (think of the key as if it were the subscript of the vector, or the object as a set of tuples):
?: {"a":1, "b":2} * {"a":3, "b":4, "c":5} = {"a":[1, 3], "b":[2, 4], "c":[null, 5]}
set union: {"a":1, "b":2, "d":6} + {"a":1, "b":4, "c":5} = {"a":1, "b":4, "c":5, "d":6}
set intersect: {"a":1, "b":2} / {"a":1, "b":4, "c":5} = {"a":1}
set difference: {"a":1, "b":2} - {"a":1, "b":4, "c":5} = {"b":2}
(Not sure if I like these last two)

A modulo operator or function should also be included for numbers.
I would also raise an error for any undefined operations, and follow up with the try() function.

[innovate-invent]

We could additionally support removing keys via {"a":1, "b":2, "d":6} - ["d"] = {"a":1, "b":2}

[springcomp]

Awesome.

[springcomp]

This recent issue also touches on this subject.

[innovate-invent]

We may also want unary +/-:

+`-1` => 1
-`-1` => -1
-`1` => -1

[springcomp]

We may also want unary +/-:

Thanks for this suggestion. That is really useful.
I included this in the specification and uploaded this to the live preview examples.

Funnily enough, this makes expressions like there valid 🤣:

• a +++++ b
• a −+− b

[chipzoller]

FWIW you may consider taking these from us at Kyverno as we've implemented all the basic ones: https://kyverno.io/docs/writing-policies/jmespath/#custom-filters

[springcomp]

Indeed, @chipzoller thanks for the link.

We consider using syntax as opposed to functions.
But by all means please let you voice heard.

I understand as a library author, you had no choice but to create additionnal functions.
Would you be happy with this being implemented at the core language level?
That‘s what we are trying to propose here.

[springcomp]

As for other filters – not related to arithmetic operations, please, @chipzoller , feel free to raise new discussions if you feel some would be better included in the specification itself. Please, bear in mind we consider actually specifying the concept of third-party functions, so implementing custom filter will still be possible in the future.

[chipzoller]

Yeah, I understand, just sharing how we've done it in case it helps. We want to contribute back to the JMESPath community as much as possible, but Kyverno is a Golang app and so what we've done we have done for our needs only. We can either figure out a way to push those into the core, or other contributors to this community can do that legwork.

[springcomp]

Arithmetic Expressions

JEP	16
Author	Maxime Labelle
Created	01-August-2022
SemVer	MINOR
Status	Draft
[Discussion #25]	#25

Abstract

This JEP proposes a new arithmetic-expression rule in the grammar to support simple arithmetic operations.

Motivation

JMESPath supports querying JSON documents that may return numbers at various stages, but lacks the capability to operate on them using simple arithmetic operations. The only support for arithmetic is currently brought by the sum() function.

Specification

To support arithmetic operations, we need to introduce the following operators:

• + addition operator
• - subtraction operator
• * multiplication operator
• / division operator
• % modulo operator
• // integer division operator

Proper mathematical operators also exist in Unicode and SHOULD be supported:

• – (U+2212 MINUS SIGN)
• ÷ (U+00F7 DIVISION SIGN)
• × (U+00D7 MULTIPLY SIGN)

Syntax changes

In addition to the following grammar changes, we introduce the usual operator precedence, from lowest to highest:

• - subtraction operator and + addition operator
• / division, * multiplication, % modulo and // integer division operators

In the absence of parentheses, operators of the same level of precedence are evaluated from left to right.

Arithmetic operators have higher precedence than comparison operators and lower precedence than the . "dot" sub-expression token separator.

expression =/ arithmetic-expression

arithmetic-expression =/ "+" expression ; + %x43
arithmetic-expression =/ ( "-" / "–" ) expression ; - %x45 – %x2212

arithmetic-expression = expression "%" expression ; % %x37
arithmetic-expression =/ expression ( "*" / "×" ) expression ; * %x42 ×  %xD7
arithmetic-expression =/ expression "+" expression ; + %x43
arithmetic-expression =/ expression ( "-" / "–" ) expression ; - %x45 – %x2212
arithmetic-expression =/ expression ( "/" / "÷" ) expression ; / %x47 ÷ %xF7
arithmetic-expression = expression "//" expression ; // %47 %47

Examples

Expression	Result
`1` + `2`	3.0
`1` – `2`	-1.0
`2` × `4`	8.0
`2` ÷ `3`	0.666666666666667
`10` % `3`	1.0
`10` // `3`	3.0
-`1` − +`2`	-3.0

Since arithmetic-expression is not valid on the right-hand-side of a sub-expression, a pipe-expression can be used instead:

Given	Expression	Result
{ "a": { "b": 1 }, "c": { "d": 2 } }	{ ab: a.b, cd: c.d } | ab + cd	3

Compliance Tests

An arithmetic.json file will be added to the compliance test suite.
The test suite will add the following new error type:

• not-a-number

This error type would be raised at run time when dividing by zero or when overflow occurs, for instance.

History

Date	Update
30-July-2022	Arithmetic operator tokens must have higher precedence than the . sub-expression separator token.
01-August-2022	Specified integer division.
03-August-2022	Revert incorrect high precedence of operator tokens. Removed paren-expression from rhs of a sub-expression.
04-August-2022	Removed arithmetic-expression from rhs of a sub-expression as it is useless. Detailed relative precedence. Added some examples.
16-August-2022	Specified unary + and − arithmetic operations and added an example.

[innovate-invent]

should we support integer division? //

[springcomp]

Why not 🤔. I was not aware of this symbol. Is this standardized and widely recognized ?

[springcomp]

For information, the arithmetic-expression syntax is demonstrated here.

[innovate-invent]

It is at least available in python: https://python-reference.readthedocs.io/en/latest/docs/operators/floor_division.html

…

On Fri, Jul 29, 2022, 1:07 AM Maxime Labelle ***@***.***> wrote: Why not 🤔. I was not aware of this symbol. Is this standardized and widely recognized ? — Reply to this email directly, view it on GitHub <#25 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANP25ABCUOLKCDUHWQ2VHTVWOGLXANCNFSM5P3RDYFQ> . You are receiving this because you commented.Message ID: <jmespath-community/jmespath.spec/repo-discussions/25/comments/3278286@ github.com>

[springcomp]

Seems you are right and there is a slight consensus over using // for integer division.
Although there does not seem to be a formal definition available.

[hell-racer]

Hi!

Arithmetic operator tokens must have higher precedence than the . sub-expression separator token.

I'm not sure I understand this... Given the input { "a": { "b": 1 }, "c": { "d": 2 } }, how do I sum these two values? Is it a.b + c.d? If that's correct, then the + has lower precedence than ., right?

[hell-racer]

Just another thought. Consider generating a new hash:
input: { "a": { "b": 1 }, "c": { "d": 2 } }
expr: { e: a.b, f: c.d }
We somehow have access to the same context (which is root in this case) while constructing each property. Why can't we have access to the same context while evaluating a.b + c.d?
Update: I think now I understand why :). Because at the start of evaluation of a.b + c.d we don't know that we have to keep current context. And when constructing a hash, we know that, because we already encountered {.

[hell-racer]

I think on my project most use cases would be like these:
input: { "a": { "b": 1 }, "c": { "d": 2, "e": 3 } }
expr1: a.b + c.d
expr2: c | d + e (would be totally fine if I had to use c | (d + e) in this case)

[hell-racer]

Probably we're better off just with more functions like sum()?

[springcomp]

Both variants of your referred to expr2 above are legal and valid as this JEP is currently specified. They will produce the correct result.

However, the expr1 would require altering the grammar in a different way. I did not manage to produce a valid grammar to support this example – it’s not just as simple as changing a precedence level – and when I do, I will probably have introduced a lot of regressions.

As for more functions like sum() that would be a good idea.

In fact, we are thinking about a reduce operation that would encompass those use cases.

Just like we have map(), I think we need a way to collapse a JSON array to a single value. It could be the sum, the product, whether any items fulfills a certain criteria, etc.

We just have not landed on a consensual syntax yet 😁.

[springcomp]

@hell-racer

I'm not sure I understand this... Given the input { "a": { "b": 1 }, "c": { "d": 2 } }, how do I sum these two values? Is it a.b + c.d? If that's correct, then the + has lower precedence than ., right?

I don't know what I was thinking.
But your preferred syntax absolutely works with the latest version of the proposal.

Arithmetic operators do have lower precedence than the sub-expression separator.
The expression a.b + c.d is indeed totally valid.

You can try it here for yourself.

[Thaina]

expression =/ arithmetic-expression

arithmetic-expression = expression "%" expression ; % %x37
arithmetic-expression =/ expression ( "*" / "×" ) expression ; * %x42 ×  %xD7
arithmetic-expression =/ expression "+" expression ; + %x43
arithmetic-expression =/ expression ( "-" / "–" ) expression ; - %x45 – %x2212
arithmetic-expression =/ expression ( "/" / "÷" ) expression ; / %x47 ÷ %xF7
arithmetic-expression = expression "//" expression ; // %47 %47

I am concern that this seem to not work with PEMDAS. * / and % should have higher priority

[springcomp]

@Thaina did you try it?

I think it works as intended.

I will double-check the operator precedence order.