Partial determinism of the descendant selector (..) #103
Comments
|
My preference is pre-order traversal (node then descendants). |
|
Except that I don't know what's "SHOULD" about that, I agree with these partial ordering guidelines. If we want to introduce different compliance levels, that is fine, but it SHOULD :-) not be coupled to using the term SHOULD vs. MUST. |
|
We need to be careful here that we don't optimize for tests. It seems to be a focus of this discussion that we want to make verification of output in the test suite easier. Rather, we need to focus on supportability and interoperability. If that aligns with running the test, that's good. But don't let making running the tests easier at the cost of making things significantly more difficult for implementors. It's non-trivial, but also not very complex, to verify unordered sets of nodes in output. |
|
More significant than complications for the JSONPath test suite is the
amount of non-determinism that tests of code which uses JSONPath will have
to accommodate.
…On Fri, 23 Jul 2021 at 22:58, Greg Dennis ***@***.***> wrote:
We need to be careful here that we don't optimize for tests. It seems to
be a focus of this discussion that we want to make verification of output
in the test suite easier.
Rather, we need to focus on supportability and interoperability. If that
aligns with running the test, that's good. But don't let making running the
tests easier at the cost of making things significantly more difficult for
implementors.
It's non-trivial, but also not very complex, to verify unordered sets of
nodes in output.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
<#103 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAXF2NMGQ3XYRKQODEWNGDTZHQY3ANCNFSM46XPPL6Q>
.
|
|
I recommend checking out the language around interoperability in 8259. Something like that might work here. |
|
I'm having a hard time judging the consensus, or lack of it, here. Regardless of the detailed language, please could interested parties indicate which of these options they favour: A. Don't tie down the non-determinism of /cc @goessner |
|
I recommend checking out the language around interoperability in 8259. Something like that might work here.
I’m not sure you actually mean that, but for me that would be about the worst result we could arrive at.
8259 was worked on in a politicized context, which simply didn’t allow us to clean it up. There barely was the courage to finally say that interchanging JSON implies UTF-8, which was at least some progress.
But there was no way to clarify the data model issues, and this is why JSON in practice essentially has been reduced to I-JSON, which is not bad at all, but really different from what JSON originally promised to be.
I’d prefer if we don’t need an I-JSONPath document to clean up the interoperability issues left in JSONPath-base.
Given that we have JSON and I-JSON, it may be useful to acknowledge (or violently reject! — we get to choose) the emergence of multiple levels/classes of interoperability with JSONPath, too.
Grüße, Carsten
|
I would vote for this option, but in the same time we have slices with negative steps, which naturally do sort of "backwards" iteration in Python. Here is an example - while there's no clear consensus, we see that all implementations with "python-compatible" slices return values in reverse order, no exceptions. If we determine order of array items visiting (thus determining the order of selected nodes in output), shouldn't we also take into consideration the effect of selectors, and if yes - then should the order of union items affect the order of visit? We can consider part of selectors as "order neutral" and others are "order affecting", with affecting selectors overriding natural order. At first glance I see slices and unions as order-affecting selectors. |
|
@remorhaz I don't follow. The ordering of |
This is not absolutely clear for me. Let us have the following structure: [
[
[1, 2, 3, 4],
[5, 6, 7, 8]
]
]And we apply
Then we apply
The resulting node list is:
I didn't notice any problems following both "natural order" while applying |
|
The idea of determinism in arrays itself seems important to me. For example, let some service provides array of some data in some garanteed order that is important for end user, and and another service wants to use JSONPath to filter them on some fancy condition. If JSONPath provides determined order of results, then it's at least possible to solve the problem with a single query. If no - we'll have no easy way to do it. We will need to extract from JSON original order somehow (with alternative tool or manual parsing), and then use it to rebuild original order of filtered items. It will be much more convenient to receive results in a determined order. |
|
The main aspect of JSON that leads to non-determinism is the fact that no
specification makes any guarantee that the order of Members in an Object
carries any meaning. RFC8259, last para of Section 4
<https://datatracker.ietf.org/doc/html/rfc8259#section-4>, explicitly
cautions against relying on this.
On the other hand, order in arrays is explicitly significant and I would be
shocked if application of array-based selectors did not produce identical
results in all implementations.
Or am I missing something obvious?
…On Thu, Sep 9, 2021 at 5:56 AM Edward Surov ***@***.***> wrote:
The idea of determinism in arrays itself seems important to me. For
example, let some service provides array of some data in some garanteed
order that is important for end user, and and another service wants to use
JSONPath to filter them on some fancy condition.
If JSONPath provides determined order of results, then it's at least
possible to solve the problem with a single query. If no - we'll have no
easy way to do it. We will need to extract from JSON original order somehow
(with alternative tool or manual parsing), and then use it to rebuild
original order of filtered items. It will be much more convenient to
receive results in a determined order.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#103 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAEJE4BWYATVFBIEWAYLWDUBCVG5ANCNFSM46XPPL6Q>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
|
Yes, I think we agree that array order should be preserved and that object order is not guaranteed, but I'd like this issue to focus specifically on the ordering of |
|
Added text indicating node-first / children-second ordering. |
|
... and preserving array order.
…On Thu, 11 Nov 2021, 05:03 Greg Dennis, ***@***.***> wrote:
Added text indicating node-first / children-second ordering.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
<#103 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAXF2K7BWKGFWQAZT3KVP3ULNFCLANCNFSM46XPPL6Q>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
|
Sure. That too. But I don't think that was ever in question. |
At today's meeting, I agreed to submit a PR to float the idea of partial determinism for the descendants operator, even though the consensus seems to be against any determinism for that operator. Given the descendants operator PR #102 isn't yet merged, I'll start with an issue to capture discussion and avoid rebasing effort.
We probably don't want to force any determinism on implementations, so the idea is to provide a recommendation for implementations which want to provide partial determinism and use "SHOULD" language so the recommendation is optional. Then implementations which opt in will agree on the ordering recommentations.
The benefits of partial determinism are:
In PR #102 there are no constraints on the non-determinism of the descendants operator:
We agreed in issue #60 to allow non-determinism in the order in which objects are iterated and the same should be true when the descendants operator encounters an object. But there are other details of the ordering which could be tied down:
(Post-order traversal, in which nodes are visited strictly after their descendants, would be an equally valid alternative.)
One of the reasons given at today's meeting for allowing full non-determinism was for parallel processing, so it would be interesting to discuss what constraints the above rules produce on parallel processing. The order of collection of the results seems to be more significant than whether or not parallel processing is used.
The text was updated successfully, but these errors were encountered: