Compare Query and Filter approaches

[mmccool]

We need to compare, contrast, and ultimately select a query/filter language for discovery.

Some candidates we have seen:

• SPARQL (https://www.w3.org/TR/sparql11-query/; https://github.com/thingweb/thingweb-directory)
• GRAPHQL
• TD Templates (need link)
• Simple Keyword search (need link to definition; Scripting API?)
• XML Path (predecessor of JSONPath, W3C standard; https://www.w3.org/TR/xpath-31/)
• JSON Path (see LinkSmart; https://goessner.net/articles/JsonPath/)
• CSS Selectors (but: part of TD vs. selecting TDs from a set; tied to document object model)

Consider two purposes:

• Selecting part of a TD (only makes sense if we support partial/part of TDs)
• Selecting some TDs from a larger set (eg TDs that match some criteria, like containing some particular semantic tag)

Need to evaluate against use cases, eg semantic query, ability to do comparison operations, ability to take apart structure, etc.

Possible requirements to consider:

• Does the query need to fit into an HTML query string?
• Can the query be in a body (eg for SPARQL or templates?)
• How much structure does the query NEED to understand?
• Do we want the ability to limit matches to parts of the structure? (opposite of above)
• Does it need to be an independent standard? (eg problem with JSONPath...)
• Do we need the ability to do relative comparisons on values, as opposed to just equalities?
• Do we need regular expressions?
• Does it work with the Scripting API? Does it need to be extended?
• Do we want to support more than one?
• Do we support partial TDs as return values? (JSONPath does this)
• How easy is it to implement?
• Is it "too powerful", eg does it pose a DOS risk from someone posting pathological queries with exponential runtimes? (this is a possibility with SPARQL and actually with some forms of regular expression as well)

[sebastiankb]

I wonder if we should also consider JSON pointers: https://tools.ietf.org/html/rfc6901

[mmccool]

Re JSON pointers, we should understand at least how they fit in. But my understanding is that they let you point at part of a JSON document, but aren't in themselves a search or query mechanism.

[relu91]

We used to use a little bit of SPARQL for discovering TDs. Giving full support to SPARQL syntax/protocol is quite challenging and I think the approach used here by @vcharpenay is the most appropriate.

The problem is how to treat application/sparql results and present them as a list of valid TDs. Using only GropuGraphPatterns is a nice workaround to filter TDs but you lack ways to use the CONSTRUCT primitive and enrich TDs with external semantic information, which might be a powerful feature too.

Furthermore, the SPARQL protocol does not provide a way to specify only GropuGraphPatterns. Consequently, a new HTTP protocol should be defined to specify the exchange of these "not-full-specified" SELECT queries.

[mmccool]

@relu91 we'll have to follow up on your comment. Can you make the next Discovery call? Ultimately we need to compare options and nail one down for a standardized directory service API definition.

[wiresio]

Yet another one: http://jsoniq.org/

[relu91]

@relu91 we'll have to follow up on your comment. Can you make the next Discovery call? Ultimately we need to compare options and nail one down for a standardized directory service API definition.

Yes, I will! see you the next Monday.

[egekorkan]

So from my understanding, the term to be used should be a JSON query language. This StackOverflow thread has a nice list: https://stackoverflow.com/questions/777455/is-there-a-query-language-for-json

It is important to note that XPath 3.1 supports JSON querying and it is standardized:

This version of XPath supports JSON as well as XML, adding maps and arrays to the data model and supporting them with new expressions in the language and new functions in [XQuery and XPath Functions and Operators 3.1].

Another little point to note would be that TDs can be serialized in JSON but not necessarily. Since we don't see many uses of other serialization formats, we can ignore this but it should be maybe at least noted.

[egekorkan]

Interesting find regarding this topic.
There is the Java implementation of the Scripting API actually uses JSON Path as a protocol binding! See here. It sort of makes sense that if one decides to store a JSON object in a resource but the client is interested only in part of it. This can be also interesting regarding the hypermedia proposal at https://github.com/w3c/wot-thing-description/tree/master/proposals/hypermedia-control

[mmccool]

Right now we only have normative support for SPARQL and it is optional, which frankly is not ideal for interoperability. It would still be useful to have a mandatory query language of some kind. I will mark this as Deferred since I think that some of the above options are worth looking at again for Discovery 2.0.