Skip to content
This repository has been archived by the owner on May 25, 2022. It is now read-only.

Remove '$' from field syntax #364

Merged
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions docs/operators/add.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ Add a value to the body using an expression
```yaml
- type: add
field: key2
value: EXPR($.key1 + "_suffix")
value: EXPR(body.key1 + "_suffix")
```

<table>
Expand Down Expand Up @@ -150,7 +150,7 @@ Add a value to attributes

```yaml
- type: add
field: $attributes.key2
field: attributes.key2
value: val2
```

Expand Down Expand Up @@ -193,7 +193,7 @@ Add a value to resource

```yaml
- type: add
field: $resource.key2
field: resource.key2
value: val2
```

Expand Down Expand Up @@ -235,8 +235,8 @@ Add a value to resource using an expression

```yaml
- type: add
field: $resource.key2
value: EXPR($.key1 + "_suffix")
field: resource.key2
value: EXPR(body.key1 + "_suffix")
```

<table>
Expand Down
6 changes: 3 additions & 3 deletions docs/operators/copy.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Copy a value from the body to resource
```yaml
- type: copy
from: key
to: $resource.newkey
to: resource.newkey
```

<table>
Expand Down Expand Up @@ -64,7 +64,7 @@ Copy a value from the body to attributes
```yaml
- type: copy
from: key2
to: $attributes.newkey
to: attributes.newkey
```

<table>
Expand Down Expand Up @@ -108,7 +108,7 @@ Copy a value from the body to attributes
Copy a value from attributes to the body
```yaml
- type: copy
from: $attributes.key
from: attributes.key
to: newkey
```

Expand Down
4 changes: 2 additions & 2 deletions docs/operators/csv_parser.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ The `csv_parser` operator parses the string-type field selected by `parse_from`
| `header_attribute` | required when `header` not set | An attribute name to read the header field from, to support dynamic field names |
| `delimiter` | `,` | A character that will be used as a delimiter. Values `\r` and `\n` cannot be used as a delimiter. |
| `lazy_quotes` | `false` | If true, a quote may appear in an unquoted field and a non-doubled quote may appear in a quoted field. |
| `parse_from` | $body | The [field](/docs/types/field.md) from which the value will be parsed. |
| `parse_to` | $body | The [field](/docs/types/field.md) to which the value will be parsed. |
| `parse_from` | `body` | The [field](/docs/types/field.md) from which the value will be parsed. |
| `parse_to` | `body` | The [field](/docs/types/field.md) to which the value will be parsed. |
| `preserve_to` | | Preserves the unparsed value at the specified [field](/docs/types/field.md). |
| `on_error` | `send` | The behavior of the operator if it encounters an error. See [on_error](/docs/types/on_error.md). |
| `timestamp` | `nil` | An optional [timestamp](/docs/types/timestamp.md) block which will parse a timestamp field before passing the entry to the output operator. |
Expand Down
6 changes: 3 additions & 3 deletions docs/operators/filter.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,22 +17,22 @@ The `filter` operator filters incoming entries that match an expression.

```yaml
- type: filter
expr: '$body.message matches "^LOG: .* END$"'
expr: 'body.message matches "^LOG: .* END$"'
output: my_output
```

#### Filter entries based on a label value

```yaml
- type: filter
expr: '$attributes.env == "production"'
expr: 'attributes.env == "production"'
output: my_output
```

#### Filter entries based on an environment variable

```yaml
- type: filter
expr: '$body.message == env("MY_ENV_VARIABLE")'
expr: 'body.message == env("MY_ENV_VARIABLE")'
output: my_output
```
4 changes: 2 additions & 2 deletions docs/operators/json_parser.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ The `json_parser` operator parses the string-type field selected by `parse_from`
| --- | --- | --- |
| `id` | `json_parser` | A unique identifier for the operator. |
| `output` | Next in pipeline | The connected operator(s) that will receive all outbound entries. |
| `parse_from` | `$body` | The [field](/docs/types/field.md) from which the value will be parsed. |
| `parse_to` | `$body` | The [field](/docs/types/field.md) to which the value will be parsed. |
| `parse_from` | `body` | The [field](/docs/types/field.md) from which the value will be parsed. |
| `parse_to` | `body` | The [field](/docs/types/field.md) to which the value will be parsed. |
| `preserve_to` | | Preserves the unparsed value at the specified [field](/docs/types/field.md). |
| `on_error` | `send` | The behavior of the operator if it encounters an error. See [on_error](/docs/types/on_error.md). |
| `if` | | An [expression](/docs/types/expression.md) that, when set, will be evaluated to determine whether this operator should be used for the given entry. This allows you to do easy conditional parsing without branching logic with routers. |
Expand Down
2 changes: 1 addition & 1 deletion docs/operators/metadata.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ Configuration:
- type: metadata
output: metadata_receiver
attributes:
environment: 'EXPR( $.environment == "production" ? "prod" : "dev" )'
environment: 'EXPR( body.environment == "production" ? "prod" : "dev" )'
```

<table>
Expand Down
8 changes: 4 additions & 4 deletions docs/operators/move.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ Move a value from the body to attributes
```yaml
- type: move
from: ip
to: $attributes.ip
to: attributes.ip
```

<table>
Expand Down Expand Up @@ -145,7 +145,7 @@ Replace the body with an individual value nested within the body
```yaml
- type: move
from: log
to: $body
to: body
```

<table>
Expand Down Expand Up @@ -184,7 +184,7 @@ Remove a layer from the body
```yaml
- type: move
from: wrapper
to: $body
to: body
```

<table>
Expand Down Expand Up @@ -232,7 +232,7 @@ Merge a layer to the body
```yaml
- type: move
from: object
to: $body
to: body
```

<table>
Expand Down
4 changes: 2 additions & 2 deletions docs/operators/recombine.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ Configuration:
- type: recombine
combine_field: message
combine_with: ""
is_last_entry: "$body.logtag == 'F'"
is_last_entry: "body.logtag == 'F'"
overwrite_with: "newest"
```

Expand Down Expand Up @@ -101,7 +101,7 @@ This can be expressed with the following configuration:
```yaml
- type: recombine
combine_field: message
is_first_entry: $body.message matches "^[^\s]"
is_first_entry: body.message matches "^[^\s]"
```

Given the following input file:
Expand Down
8 changes: 4 additions & 4 deletions docs/operators/regex_parser.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ This operator makes use of [Go regular expression](https://github.com/google/re2
| `id` | `regex_parser` | A unique identifier for the operator. |
| `output` | Next in pipeline | The connected operator(s) that will receive all outbound entries. |
| `regex` | required | A [Go regular expression](https://github.com/google/re2/wiki/Syntax). The named capture groups will be extracted as fields in the parsed body. |
| `parse_from` | `$body` | The [field](/docs/types/field.md) from which the value will be parsed. |
| `parse_to` | `$body` | The [field](/docs/types/field.md) to which the value will be parsed. |
| `parse_from` | `body` | The [field](/docs/types/field.md) from which the value will be parsed. |
| `parse_to` | `body` | The [field](/docs/types/field.md) to which the value will be parsed. |
| `preserve_to` | | Preserves the unparsed value at the specified [field](/docs/types/field.md). |
| `on_error` | `send` | The behavior of the operator if it encounters an error. See [on_error](/docs/types/on_error.md). |
| `if` | | An [expression](/docs/types/expression.md) that, when set, will be evaluated to determine whether this operator should be used for the given entry. This allows you to do easy conditional parsing without branching logic with routers. |
Expand Down Expand Up @@ -162,7 +162,7 @@ Configuration:
- type: regex_parser
regex: '^Host=(?<host>)$'
parse_from: message
if: '$body.type == "hostname"'
if: 'body.type == "hostname"'
```

<table>
Expand Down Expand Up @@ -229,7 +229,7 @@ Configuration:
- type: regex_parser
regex: '^Host=(?<host>)$'
parse_from: message
if: '$body.type == "hostname"'
if: 'body.type == "hostname"'
```

<table>
Expand Down
10 changes: 5 additions & 5 deletions docs/operators/remove.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ The `remove` operator removes a field from a record.
| --- | --- | --- |
| `id` | `remove` | A unique identifier for the operator. |
| `output` | Next in pipeline | The connected operator(s) that will receive all outbound entries. |
| `field` | required | The [field](/docs/types/field.md) to remove. if '$attributes' or '$resource' is specified, all fields of that type will be removed. |
| `field` | required | The [field](/docs/types/field.md) to remove. if 'attributes' or 'resource' is specified, all fields of that type will be removed. |
| `on_error` | `send` | The behavior of the operator if it encounters an error. See [on_error](/docs/types/on_error.md). |
| `if` | | An [expression](/docs/types/expression.md) that, when set, will be evaluated to determine whether this operator should be used for the given entry. This allows you to do easy conditional parsing without branching logic with routers. |

Expand Down Expand Up @@ -100,7 +100,7 @@ Remove an object from the body
Remove a value from attributes
```yaml
- type: remove
field: $attributes.otherkey
field: attributes.otherkey
```

<table>
Expand Down Expand Up @@ -142,7 +142,7 @@ Remove a value from attributes
Remove a value from resource
```yaml
- type: remove
field: $resource.otherkey
field: resource.otherkey
```

<table>
Expand Down Expand Up @@ -184,7 +184,7 @@ Remove a value from resource
Remove all resource fields
```yaml
- type: remove
field: $resource
field: resource
```

<table>
Expand Down Expand Up @@ -227,7 +227,7 @@ Remove all resource fields
Remove all attributes
```yaml
- type: remove
field: $attributes
field: attributes
```

<table>
Expand Down
12 changes: 6 additions & 6 deletions docs/operators/retain.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,8 +115,8 @@ Retain fields from resource
```yaml
- type: retain
fields:
- $resource.key1
- $resource.key2
- resource.key1
- resource.key2
```

<table>
Expand Down Expand Up @@ -165,8 +165,8 @@ Retain fields from attributes
```yaml
- type: retain
fields:
- $attributes.key1
- $attributes.key2
- attributes.key1
- attributes.key2
```

<table>
Expand Down Expand Up @@ -214,8 +214,8 @@ Retain fields from all sources
```yaml
- type: retain
fields:
- $resource.key1
- $attributes.key3
- resource.key1
- attributes.key3
- key5
```

Expand Down
8 changes: 4 additions & 4 deletions docs/operators/router.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,9 @@ An entry that does not match any of the routes is dropped and not processed furt
- type: router
routes:
- output: my_json_parser
expr: '$.format == "json"'
expr: 'body.format == "json"'
- output: my_syslog_parser
expr: '$.format == "syslog"'
expr: 'body.format == "syslog"'
```

#### Drop entries based on content
Expand All @@ -44,7 +44,7 @@ An entry that does not match any of the routes is dropped and not processed furt
- type: router
routes:
- output: my_output
expr: '$.message matches "^LOG: .* END$"'
expr: 'body.message matches "^LOG: .* END$"'
```

#### Route with a default
Expand All @@ -53,6 +53,6 @@ An entry that does not match any of the routes is dropped and not processed furt
- type: router
routes:
- output: my_json_parser
expr: '$.format == "json"'
expr: 'body.format == "json"'
default: catchall
```
4 changes: 2 additions & 2 deletions docs/operators/syslog_parser.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ The `syslog_parser` operator parses the string-type field selected by `parse_fro
| --- | --- | --- |
| `id` | `syslog_parser` | A unique identifier for the operator. |
| `output` | Next in pipeline | The connected operator(s) that will receive all outbound entries. |
| `parse_from` | `$body` | The [field](/docs/types/field.md) from which the value will be parsed. |
| `parse_to` | `$body` | The [field](/docs/types/field.md) to which the value will be parsed. |
| `parse_from` | `body` | The [field](/docs/types/field.md) from which the value will be parsed. |
| `parse_to` | `body` | The [field](/docs/types/field.md) to which the value will be parsed. |
| `preserve_to` | | Preserves the unparsed value at the specified [field](/docs/types/field.md). |
| `on_error` | `send` | The behavior of the operator if it encounters an error. See [on_error](/docs/types/on_error.md). |
| `protocol` | required | The protocol to parse the syslog messages as. Options are `rfc3164` and `rfc5424`. |
Expand Down
4 changes: 2 additions & 2 deletions docs/operators/uri_parser.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ The `uri_parser` operator parses the string-type field selected by `parse_from`
| --- | --- | --- |
| `id` | `uri_parser` | A unique identifier for the operator. |
| `output` | Next in pipeline | The connected operator(s) that will receive all outbound entries. |
| `parse_from` | `$body` | The [field](/docs/types/field.md) from which the value will be parsed. |
| `parse_to` | `$body` | The [field](/docs/types/field.md) to which the value will be parsed. |
| `parse_from` | `body` | The [field](/docs/types/field.md) from which the value will be parsed. |
| `parse_to` | `body` | The [field](/docs/types/field.md) to which the value will be parsed. |
| `preserve_to` | | Preserves the unparsed value at the specified [field](/docs/types/field.md). |
| `on_error` | `send` | The behavior of the operator if it encounters an error. See [on_error](/docs/types/on_error.md). |
| `if` | | An [expression](/docs/types/expression.md) that, when set, will be evaluated to determine whether this operator should be used for the given entry. This allows you to do easy conditional parsing without branching logic with routers. |
Expand Down
4 changes: 2 additions & 2 deletions docs/pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,9 +144,9 @@ pipeline:
# Route based on log type
- type: router
routes:
- expr: '$body startsWith "ERROR"'
- expr: 'body startsWith "ERROR"'
output: error_parser
- expr: '$body startsWith "INFO"'
- expr: 'body startsWith "INFO"'
output: info_parser

# Parse logs with format one
Expand Down
8 changes: 4 additions & 4 deletions docs/types/expression.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ being processed.
For reference documentation of the expression language, see [here](https://github.com/antonmedv/expr/blob/master/docs/Language-Definition.md).

Available to the expressions are a few special variables:
- `$body` contains the entry's body
- `$attributes` contains the entry's attributes
- `$resource` contains the entry's resource
- `$timestamp` contains the entry's timestamp
- `body` contains the entry's body
- `attributes` contains the entry's attributes
- `resource` contains the entry's resource
- `timestamp` contains the entry's timestamp
- `env()` is a function that allows you to read environment variables

## Examples
Expand Down
Loading