Skip to content

Commit

Permalink
fixed problems in the query parser
Browse files Browse the repository at this point in the history 
added more tests
added more documentation
  • Loading branch information
@poef
poef committed Mar 1, 2020
1 parent 6a0df4c commit 1d7dd4f
Show file tree
Hide file tree
Showing 3 changed files with 177 additions and 20 deletions.
124 changes: 120 additions & 4 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
arc\store: fast schema-free JSON store
======================================
# arc\store: fast schema-free JSON store

[![Scrutinizer Code Quality](https://scrutinizer-ci.com/g/Ariadne-CMS/arc-store/badges/quality-score.png?b=master)](https://scrutinizer-ci.com/g/Ariadne-CMS/arc-base/?branch=master)
[![Code Coverage](https://scrutinizer-ci.com/g/Ariadne-CMS/arc-store/badges/coverage.png?b=master)](https://scrutinizer-ci.com/g/Ariadne-CMS/arc-store/)
Expand All @@ -13,8 +12,7 @@ arc\store is part of [ARC - a component library](http://www.github.com/Ariadne-C
ARC is a spinoff from the Ariadne Web Application Platform and Content Management System
[http://www.ariadne-cms.org/](http://www.ariadne-cms.org/).

Installation
------------
## Installation

You can install the full set of ARC components using composer:

Expand All @@ -27,3 +25,121 @@ Or you can start a new project with arc/arc like this:
Or just use this package:

composer require arc/store

## Usage

```php
$store = \arc\store::connect('pgsql:host=localhost;port=5432;dbname=arcstore;user=arcstore;password=arcstore');
$store->initialize();
if ($store->save(\arc\prototype::create(["foo" => "bar"]), "/foo/")) {
$objects = $store->ls('/');
var_dump($objects);
}
```

This will show an array with one object, with parent '/', name 'foo', and a single property 'foo' => 'bar'.

## What is ARC\Store?

ARC\Store is a minimal implementation of the structured object store implemented in Ariadne-CMS. It stores free form object data in a tree structure, similar to a filesystem. It provides seperate query and save and delete methods. The query has its own format and parser. The data is stored in PostgreSQL using JSONB data blobs, which are fully indexed.

This solution gives you flexible and fast storage of any kind of data, while keeping many advantages of using a proven technology like PostgreSQL. Although this implementation doesn't have it, it would be easy to add transactions with commit/rollback to gain atomic updates, even for batch operations.

Because of its tree structure, ARC\Store integrates well with other ARC Components, like ARC\Grants and ARC\Config.

## methods

### \arc\store::connect
(\arc\store\PSQLStore) \arc\store::connect( (string) $dsn, (callable) $resultHandler=null)

This method creates a new PSQLStore instance and connects it to a PostgreSQL database. Optionally you can pass your own resultHandler function. The PSQLStore class contains two static functions predefined for this:
- \arc\store\PSQLStore::defaultResultHandler
- \arc\store\PSQLStore::generatorResultHandler
The result handler is called with a compiled SQL query where clause and arguments and must execute this and return the results.

### \arc\store::disconnect
(void) \arc\store::disconnect()

Removes the last store connection from the context stack (\arc\context).

### \arc\store::cd
(\arc\store\PSQLStore) \arc\store::cd($path)

Returns a new store istance, with its default path set to $path. This call will always succeed, even if $path doesn't exist in the object store. It does not update the path of the store instance in the context stack (\arc\context). To do that, you must push the new store onto the context stack:

```php
\arc\context::push([
'arcStore' => \arc\store::cd('/foo/')
]);
```

### \arc\store::find
(mixed) \arc\store::find((string) $query, (string) $path)

Compiles the query to SQL, calls the resultHandler with it and returs the results. The query syntax is read only, it can only read data, never update or delete it.
You can also call this method on a store istance, like this:

```php
$store = \arc\context::cd('/');
$objects = $store->find("foo='bar'");
```

The query format supports the following operators:

- `<` less than
- `>` more than
- `=` equals
- `<=` less than or equal
- `>=` more than or equal
- `<>`, `!=` not equal
- `~=` similar to, supports `%` and `?` wildcards
- `!~` not similar to, supports `%` and `?` wildcards
- `?` object contains the key (property)

You can combine multiple query parts using `and` and `or`. You can use parenthesis to group them. And you can negate a part by prefixing it with `not`. Strings must be enclosed in single quotes. A single quote inside the string should be escaped with a `\`.

You can query any part of the object, but there are a few meta data properties you can search for:
- `nodes.path` matches the full path of the object in the tree
- `nodes.parent` matches the full path of the objects parent in the tree
- `nodes.mtime` matches the datetime when the object was last changed
- `nodes.ctime` matches the datetime when the object was created

Example queries:

```php
$results = $store->find("nodes.path ~= '/foo/%'");
$results = $store->find("foo.bar>3");
$results = $store->find("foo.bar>3 and foo.bar<6");
$results = $store->find("foo.bar<2 or foo.bar>8");
$results = $store->find("type='order' and ( total<10 or total>1000 )");
```

### \arc\store::parents
(mixed) \arc\store::parents((string) $path)

Returns a list of parent objects, starting with the root and ending with the direct parent.

### \arc\store::ls
(mixed) \arc\store::ls((string) $path)

Returns a list of direct children of the given path.

### \arc\store::get
(object) \arc\store::get((string) $path)

Returns the object with the give path, or null.

### \arc\store::exists
(bool) \arc\store::exists((string) $path)

Returns true if an object with the given path exists.

### \arc\store\PSQLStore->save
(bool) $store->save( (object) $data, (string) $path = '')

Saves the object data at the given path. Returns true on success or false on failure.

### \arc\store\PSQLStore->delete
(bool) $store->delete((string) $path = '')

Deletes the object with the given path and all its children. It will never remove the root object. If you don't pass an argument, it will use the current path set in the store instance. Returns true on success or false on failure.
28 changes: 14 additions & 14 deletions src/store/PSQLQueryParser.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,18 @@ final class PSQLQueryParser {
* yields the tokens in the search query expression
* @param string $query
* @return \Generator
* @throws \Exception
* @throws \LogicException
*/

private function tokens($query)
{
$token = <<<'REGEX'
/^\s*
(
(?<compare>
<= | >= | <> | < | > | = | != | ~= | !~ | \?
)
|
(?<operator>
and | or
)
Expand All @@ -38,10 +42,6 @@ private function tokens($query)
not
)
|
(?<compare>
< | > | = | <= | >= | <> | != | like | not like | \?
)
|
(?<name>
[a-z]+[a-z0-9_-]*
(?: \. [a-z]+[a-z0-9_-]* )*
Expand Down Expand Up @@ -82,36 +82,36 @@ function($key) {
}
} while($result);
if ( trim($query) ) {
throw new \Exception('Could not parse '.$query);
throw new \LogicException('Could not parse '.$query);
}
}

/**
* @param string $query
* @return string postgresql 'where' part of the sql query
* @throws \Exception when a parse error occurs
* @throws \LogicException when a parse error occurs
*/
public function parse($query)
{
$indent = 0;
$part = '';
$sql = '';
$position = 0;
$expect = 'name|parenthesis';
$expect = 'name|parenthesis_open|not';

foreach( $this->tokens($query) as $token ) {
$type = key($token);
list($token, $offset)=$token[$type];
if ( !preg_match("/^$expect$/",$type) ) {
throw new \Exception('Parse error at '.$position.': expected '.$expect.', got '.$type.': '
throw new \LogicException('Parse error at '.$position.': expected '.$expect.', got '.$type.': '
.(substr($query,0, $position)." --> ".substr($query,$position)) );
}
switch($type) {
case 'number':
case 'string':
$sql .= $part.$token;
$part = '';
$expect = ['operator','parenthesis_close'];
$expect = 'operator|parenthesis_close';
break;
case 'name':
switch ($token) {
Expand Down Expand Up @@ -142,10 +142,10 @@ public function parse($query)
$part.=$token;
str_replace($part, '#>>', '#>');
break;
case 'like':
case '~=':
$part.=' like ';
break;
case 'not like':
case '!~':
$part.=' not like ';
break;
}
Expand Down Expand Up @@ -177,10 +177,10 @@ public function parse($query)
$position += $offset + strlen($token);
}
if ( $indent!=0 ) {
throw new \Exception('unbalanced parenthesis');
throw new \LogicException('unbalanced parenthesis');
} else if ( trim($part) ) {
$position -= strlen($token);
throw new \Exception('parse error at '.$position.': '.(substr($query,0, $position)." --> ".substr($query,$position)));
throw new \LogicException('parse error at '.$position.': '.(substr($query,0, $position)." --> ".substr($query,$position)));
} else {
return $sql;
}
Expand Down
45 changes: 43 additions & 2 deletions tests/store.Test.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,47 @@ function testStoreQuery()
$this->assertEquals("nodes.path='/'", $result);
$result = $qp->parse("foo.bar='baz'");
$this->assertEquals("nodes.data #>> '{foo,bar}'='baz'", $result);
$result = $qp->parse("foo.bar !~ 'b%z'");
$this->assertEquals("nodes.data #>> '{foo,bar}' not like 'b%z'", $result);
$result = $qp->parse("foo.bar ~= 'b%z'");
$this->assertEquals("nodes.data #>> '{foo,bar}' like 'b%z'", $result);
$result = $qp->parse("foo ? 'bar'");
$this->assertEquals("nodes.data #>> '{foo}'?'bar'", $result);
$result = $qp->parse("foo.bar>3");
$this->assertEquals("nodes.data #>> '{foo,bar}'>3",$result);
$result = $qp->parse("foo.bar <> 'bar\\'bar'");
$this->assertEquals("nodes.data #>> '{foo,bar}'<>'bar\\'bar'",$result);
$result = $qp->parse("foo.bar != 'bar\\'bar'");
$this->assertEquals("nodes.data #>> '{foo,bar}'!='bar\\'bar'",$result);
$result = $qp->parse("foo.bar !~ 'b%z' and bar.foo = 3");
$this->assertEquals("nodes.data #>> '{foo,bar}' not like 'b%z' and nodes.data #>> '{bar,foo}'=3", $result);
$result = $qp->parse("(foo.bar !~ 'b%z' and bar.foo = 3)");
$this->assertEquals("(nodes.data #>> '{foo,bar}' not like 'b%z' and nodes.data #>> '{bar,foo}'=3)", $result);
$result = $qp->parse("(foo.bar !~ 'b%z' and bar.foo = 3) or nodes.path='/'");
$this->assertEquals("(nodes.data #>> '{foo,bar}' not like 'b%z' and nodes.data #>> '{bar,foo}'=3) or nodes.path='/'", $result);
$result = $qp->parse("not(foo.bar = 'bar')");
$this->assertEquals("not(nodes.data #>> '{foo,bar}'='bar')", $result);
}

function testStoreParseError()
{
$qp = new \arc\store\PSQLQueryParser();
$this->expectException(\LogicException::class);
$result = $qp->parse("just_a_name_with_1_number");
}

function testStoreParseParenthesisError()
{
$qp = new \arc\store\PSQLQueryParser();
$this->expectException(\LogicException::class);
$result = $qp->parse("(parenthesis = 'unbalanced'");
}

function testStoreParseStringError()
{
$qp = new \arc\store\PSQLQueryParser();
$this->expectException(\LogicException::class);
$result = $qp->parse("foo = 'bar");
}


Expand Down Expand Up @@ -62,10 +103,10 @@ function testStoreLs()

function testStoreFind()
{
$result = $this->store->find("nodes.path like '/%'");
$result = $this->store->find("nodes.path ~= '/%'");
$this->assertContainsOnly('stdClass',$result);
$this->assertCount(2, $result);
$result = $this->store->find("foo.bar like 'Ba%'");
$result = $this->store->find("foo.bar ~= 'Ba%'");
$this->assertCount(1, $result);
}

Expand Down

0 comments on commit 1d7dd4f

Please sign in to comment.