Skip to content

Commit

Permalink
Update to a more jq-like syntax and behavior
Browse files Browse the repository at this point in the history
Signed-off-by: Ashley Jeffs <[email protected]>
  • Loading branch information
Jeffail committed Jul 29, 2020
1 parent e117e40 commit b9941ab
Show file tree
Hide file tree
Showing 3 changed files with 19 additions and 148 deletions.
80 changes: 19 additions & 61 deletions rfcs/2020-07-21-2744-vicscript.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,16 +23,16 @@ Joining events are within scope, but aggregation of those events is not. For exa

## Guide Level Proposal

The `vicscript` transform allows you to construct a new event by defining a sequence of mapping statements from a source event. Each mapping statement describes a read-only query operation on the right hand side, and on the left hand side a destination for the query result to be mapped within the resulting event:
The `vicscript` transform allows you to mutate events by defining a sequence of mapping statements. Each mapping statement describes a read-only query operation on the right hand side, and on the left hand side a destination for the query result to be mapped within the resulting event:

```toml
[transforms.mapper]
inputs = [ "foo" ]
type = "vicscript"
mapping = """
foo = "hello"
bar = bar + 10
baz_is_large = baz > 10
.foo = "hello"
.bar = .bar + 10
.baz_is_large = .baz > 10
"""
```

Expand All @@ -41,77 +41,45 @@ baz_is_large = baz > 10
The most common query type is a dot separated path, describing how to reach a target field within the input event:

```coffee
foo = bar.baz.0.buz
.foo = .bar.baz.0.buz
```

Path sections that include whitespace or other special characters (including `.`) can be quoted:

```coffee
foo = bar."baz.buz".bev
.foo = .bar."baz.buz".bev

# Use slash escapes within quotes
bar = buz."fub\"\\fob".fab
.bar = .buz."fub\"\\fob".fab
```

And supports coalescing fields within a path with a pipe operator `|`, where if the first value is null or missing then the next value is selected, and so on:

```coffee
foo = bar.(baz | buz).bev
```

You can assign a value to the root of your event by writing to the `root` keyword:

```coffee
root = nested.object
.foo = .bar.(baz | buz).bev
```

Vicscript supports float, int, boolean, string, null, array and object literals:

```coffee
first = 7
second = false
third = "string"
fourth = null
```

As well as a timestamp type:

```coffee
# NOTE: The full API for the timestamp type is out of the scope of this RFC. The
# following example is for demonstrative purposes.

created_at = now().unix()
# Sets `created_at` to an integer representing the current unix timestamp
# (in seconds).
.first = 7
.second = false
.third = "string"
.fourth = null
```

Boolean operators and arithmetic galore:

```coffee
is_big = number > 100
multiplied = number * 7
.is_big = .number > 100
.multiplied = .number * 7
```

Perform assignments conditionally with an `if` statement:

```coffee
id = id
sorted_foo = if foo.type() == "array" { foo.sort() }
```

```text
In: {"id":"first","foo":"not an array"}
Out: {"id":"first"}'
In: {"id":"second","foo":["c","a","d","b"]}
Out: {"id":"second","sorted_foo":["a","b","c","d"]}
```

The resulting event will only contain fields that are explicitly mapped, in order to begin your map with a full copy of the input event structure you can start by assigning `this` to `root`:

```coffee
root = this
sorted_foo = if foo.type() == "array" { foo.sort() }
.id = .id
.sorted_foo = if .foo.type() == "array" { .foo.sort() }
```

```text
Expand All @@ -122,22 +90,12 @@ In: {"id":"second","foo":["c","a","d","b"]}
Out: {"id":"second","foo":["c","a","d","b"],"sorted_foo":["a","b","c","d"]}
```

Use a wealth of methods on values in order to perform common mutations on them, since the underlying value is immutable these operations are safe to use in any combination:
Remove fields by assigning them the `delete` keyword:

```coffee
sorted = foo.sort()
uppercase = bar.uppercase()
```

```
In: {"foo":["c","a","d","b"],"bar":"hello world"}
Out: {"sorted":["a","b","c","d"],"uppercase":"HELLO WORLD"}
```text
.foo = delete
```

## Examples

I've written an implementation of [an embedded CSV parser](2020-07-21-2744-vicscript/example1.coffee) using the above proposal in order to demonstrate what it looks like. This example is currently runnable with Benthos, and I've used the `.coffee` suffix because the syntax highlighting for CoffeeScript basically gives us everything we need already.

## Prior Art

### JQ
Expand Down
47 changes: 0 additions & 47 deletions rfcs/2020-07-21-2744-vicscript/example1.coffee

This file was deleted.

40 changes: 0 additions & 40 deletions rfcs/2020-07-21-2744-vicscript/example2.coffee

This file was deleted.

0 comments on commit b9941ab

Please sign in to comment.