Consider implementing a guid() function.

[springcomp]

A guid() function that outputs a string representing a globally unique identifier.

string guid([string $format_specifier])

Returns an RFC4122-compliant string representation of a globally unique identifier.

The format specifier is optional. If it is an empty string or the default value "d", the function returns a string with the following format:

"b14b6399-3a6d-4fa9-a5cb-5fba2331818e"

The accepted format specifiers are listed in the following table:

Specifier	Format of the return value
`"n"`	32 lowercase digits. "00000000000000000000000000000000"
`"d"`	32 lowercase digits, separated by hyphens. "00000000-0000-0000-0000-000000000000"
`"u"`	URN representation. "urn:uuid:00000000-0000-0000-0000-000000000000"

Those familiar with Microsoft would also be familiar with the two following format specifiers:

Specifier	Format of the return value
`"b"`	32 lowercase digits, separated by hyphens and enclosed in braces. "{00000000-0000-0000-0000-000000000000}"
`"p"`	32 lowercase digits, separated by hyphens and enclosed in parentheses. "(00000000-0000-0000-0000-000000000000)"

Expression	Result
guid()	"b14b6399-3a6d-4fa9-a5cb-5fba2331818e"
guid(`""`)	"b14b6399-3a6d-4fa9-a5cb-5fba2331818e"
guid(`"n"`)	"b14b63993a6d4fa9a5cb5fba2331818e"
guid(`"d"`)	b14b6399-3a6d-4fa9-a5cb-5fba2331818e"
guid(`"u"`)	"urn:uuid:b14b6399-3a6d-4fa9-a5cb-5fba2331818e"
guid(`"b"`)	"{b14b6399-3a6d-4fa9-a5cb-5fba2331818e}"
guid(`"p"`)	"(b14b6399-3a6d-4fa9-a5cb-5fba2331818e)"

Alternate name

This function can also be named uuid().

[jdevillard]

Can we also specify uppercase ?

[springcomp]

We could, but this is not RFC4122-compliant. As a matter of fact, the "b" and "p" formats are unique to Windows.
That’s why I specified them separately.

[innovate-invent]

My major concern with this is that we are getting into generating data rather than just transforming it. Is this out of scope for JMESPath?

This would also remove the idempotent nature of queries, pulling in other implications.

[innovate-invent]

Unless I am mistaken, Liquid is a templating syntax that we wouldn't want to compete with.
JMESPath (for me at least) has always been a library for traversing and transforming data-structures, for the purpose of outputting a different data-structure. The closest competitor would be something like jq, but jq is a complete command line solution rather than a library. It includes functionality specific to that application. I feel JMESPath should try to remain application agnostic so no application importing it is cornered into having to offer functionality that doesn't fit its use case.

I should also mention that I am pushing this conversation because it keeps coming up in other threads that we need a solid definition of scope for features moving forward.

[springcomp]

I agree that we need a definition of scope.
That may be worth a discussion open to the community.

To understand where @jdevillard and myself are coming from, let me explain how we came to be involved in writing an implementation of JMESPath in .Net.

@jdevillard and I have a long history in the integration space – think EAI, ESB, Middleware applications. As such, we have long been looking for "mapping tools" and, for history reasons, we started with XML-based tools.

For the last ten years, @jdevillard has been my partner in a upstart that we founded to design and implement the first iPaaS software in the french market. The company folded in late 2019 but @jdevillard and a consultancy company acquired the IP and are still maintaining and promoting the product to this day. I left in late 2019 as most other employees.

Anyway, our journey together lead us to work on more modern and mainstream protocols and we thus shifted to JSON. For this reason, we have long sought to find JSON-based mapping tools. JMESPath was promising, but not feature complete. However, with a bit of wrapping magic, we could probably come up with something that is spiritually similar to what XSL is to XML.

The work we are currently doing with the promotion and advancement of the spec is actually exceeding our expectations. Your invaluable inputs are great and I could not be more thrilled to have you onboard this project.

At the same time, the many improvements that we can almost see coming to fruition soon make us think that maybe, JMESPath is more and more capable of becoming what XSL is to XML. Whereas it was – is currently – somewhat limited and looks similar to what XPath is to XSL.

If that is the scope we should be limiting ourselves to fulfilling, great. Let's decide as a community.

Hope this clarifies where we are standing on this.

[innovate-invent]

That is an interesting thought, XSL has always scared me away as a syntax so I don't know much about it. What is the technical gap between XPath and XSL, or what would be needed within JMESPath to close this gap?

[springcomp]

At its heart, an XSL stylesheet is an XML document that drives transformations from an input XML document to an output XML document.
An XSL document describes the shape of the output XML document in terms of structure and derives its content from the source XML document.
In order to extract content from the source document, XSL uses XPath syntax.

Consider the following XML document

<greetings>
	<greet>
		<who>World!</who>
		<how>Hello</how>
	</greet>
	
	<greet>
		<who>Dude!</who>
		<how>Goodbye</how>
	</greet>
</greetings>

This would be similar to a JSON document that looks like so:

{
	"greetings": [
		{ "who": "World!", "how": "Hello" },
		{ "who": "Dude!", "how": "Goodbye" }
	]
}

We could have a very simple XSL transformation that concatenates greetings:

<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  
  <xsl:template match="/">
  
    <Salutations>
      <xsl:for-each select="./greetings/greet">
        <Salutation>
        <xsl:value-of select="./how/text()" />, <xsl:value-of select="./who/text()" />
        </Salutation>
      </xsl:for-each>
    </Salutations>
  
  </xsl:template>

</xsl:stylesheet>

Here, all XML tags with an <xsl: /> prefix are XSL syntax. XSL has a logic of transformation templates that can embed some reusable logic but here we only have one template. In terms of XSL, all it does is loop with a <xsl:for-each> and retrieve some values with <xsl:value-of>.

The regular XML tags like <Salutations /> and <Salutation /> are just transmitted verbatim to the output. Their structure actually define the shape of the resulting XML document.

Finally, the /, ./greetings/greet, ./how/text() and ./who/text() are XPath expressions, used to refer to or retrieve content from portions of the source XML document.

This would produce the resulting XML:

<Salutations>
  <Salutation>Hello, World!</Salutation>
  <Salutation>Goodbye, Dude!</Salutation>
</Salutations>

XSL is a wrapper for both structured output – akin to what JMESPath’s multi-select-hash and multi-select-list constructs would be – and XPath expressions that are anchored to the current context to refer to contents from the source document.

The first match happens on / which is the root of the source document.
Anchored to the root, the <xsl:for-each /> XSL construct uses the ./greetings/greet XPath expression to refer to the XML tags from the same name in the source document.

Then, anchored to the <greet /> element at each iteration, the <xsl:value-of /> XSL construct uses the ./how/text() and ./who/text() XPath expressions to contatenate the values from the corresponding tags in the source document.

In that respect, XPath shares some characteristics with JSONPath – to refer to portions in an XML document. It would also share some characteristics with the future of JMESPath, in that it has many utility functions for dealing with string manipulations, artithmetic operations, etc.

For reference, to achieve the same result coming from the source JSON document shown above is not currently possible with JMESPath as far as I know. However, a sufficiently advanced spec would allow us to support this scenario using an expression that could look like:

{salutations: greetings[].concat(how, `", "`, who)}

To produce the resulting document:

{
  "salutations": [
	  "Hello, World!",
	  "Goodbye, Dude!"
  ]
}

Of course, I do not think the goal of JMESPath is to replicate XSL.
Indeed, XSL has many features – such as the ability to isolate "transformation logic" into templates and reuse those templates at arbitrarily nested levels of the source document hierarchy.

But at least we would have a solid contender in the arena where "JSON mapping tools" compete.

To answer your question specifically, XSL and XPath are distinct but related technologies. Together, they are used to perform arbitrarily complex mapping scenarios.

I think what is distinctive of JMESPath from any other JSON mapping tools are the multi-select-list and multi-select-hash constructs, that allow us to define arbitrary shape for the resulting JSON output.

One critical missing part is the ability to refer to any arbitrary lexical scope – which will be solved by the let() function.

Together with the work with envision being part of the next version of the specification, I do not think that there is much missing to make JMESPath very similar to what XSL is to XML.

Maybe one will always want more and more specialized functions. Not to mention the fact that XSLT 2.0 is actually Turing Complete 🤯 which I guess is a non-goal for us!

[innovate-invent]

Thanks for the extensive writeup. It sounds like we are well on our way to getting it close to what you need. I would recommend the join() function for your specific example. I think an important distinction between XSL and JMESPath is that the output of XSL can be a free form text document while JMESPath outputs a data-structure. You could potentially use JMESPath in a templating engine, but that is an application of the library.

I agree that we wouldn't want JMESPath to be Turing complete, the importing application should be responsible for most of the logic while JMESPath remains a mechanism for the end user to express to the application how to ingest arbitrary data-structures, converting them to a structure the application expects. For your use case that would simply be wrapping JMESPath in a larger templating syntax like Liquid.

One feature that I have been tossing around in my head has been to allow expression-type values to be stored using let() so that they can be reused anywhere they are accepted. This would get close to user defined functions without being burdensome on the syntax. We could even go further to specify that any reference of a expression-type value outside of function arguments is replaced by the result of its evaluation within its current context.

As for guid() and get-date(), it would be problematic for me if the spec started requiring access to system resources that may or may not exist. Embedded or IoT devices communicate with JSON and applications of JMESPath within these systems don't necessarily have a real time clock or network device (MAC address) that would be required to implement version 1 or 2 of RFC4122. I think these scenarios need to be seriously considered when determining the scope of JMESPath.