Skip to content

Commit

Permalink
Add syslog input plugin (influxdata#4181)
Browse files Browse the repository at this point in the history
  • Loading branch information
leodido authored and otherpirate committed Mar 15, 2019
1 parent f392489 commit 32edb48
Show file tree
Hide file tree
Showing 7 changed files with 1,520 additions and 0 deletions.
1 change: 1 addition & 0 deletions Godeps
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ github.com/go-redis/redis 73b70592cdaa9e6abdfcfbf97b4a90d80728c836
github.com/go-sql-driver/mysql 2e00b5cd70399450106cec6431c2e2ce3cae5034
github.com/hailocab/go-hostpool e80d13ce29ede4452c43dea11e79b9bc8a15b478
github.com/hashicorp/consul 5174058f0d2bda63fa5198ab96c33d9a909c58ed
github.com/influxdata/go-syslog 84f3b60009444d298f97454feb1f20cf91d1fa6e
github.com/influxdata/tail c43482518d410361b6c383d7aebce33d0471d7bc
github.com/influxdata/toml 5d1d907f22ead1cd47adde17ceec5bda9cacaf8f
github.com/influxdata/wlog 7c63b0a71ef8300adc255344d275e10e5c3a71ec
Expand Down
1 change: 1 addition & 0 deletions plugins/inputs/all/all.go
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@ import (
_ "github.com/influxdata/telegraf/plugins/inputs/solr"
_ "github.com/influxdata/telegraf/plugins/inputs/sqlserver"
_ "github.com/influxdata/telegraf/plugins/inputs/statsd"
_ "github.com/influxdata/telegraf/plugins/inputs/syslog"
_ "github.com/influxdata/telegraf/plugins/inputs/sysstat"
_ "github.com/influxdata/telegraf/plugins/inputs/system"
_ "github.com/influxdata/telegraf/plugins/inputs/tail"
Expand Down
119 changes: 119 additions & 0 deletions plugins/inputs/syslog/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# syslog input plugin

Collects syslog messages as per RFC5425 or RFC5426.

It can act as a syslog transport receiver over TLS (or TCP) - ie., RFC5425 - or over UDP - ie., RFC5426.

This plugin listens for syslog messages following RFC5424 format. When received it parses them extracting metrics.

### Configuration

```toml
[[inputs.syslog]]
## Specify an ip or hostname with port - eg., tcp://localhost:6514, tcp://10.0.0.1:6514
## Protocol, address and port to host the syslog receiver.
## If no host is specified, then localhost is used.
## If no port is specified, 6514 is used (RFC5425#section-4.1).
server = "tcp://:6514"

## TLS Config
# tls_allowed_cacerts = ["/etc/telegraf/ca.pem"]
# tls_cert = "/etc/telegraf/cert.pem"
# tls_key = "/etc/telegraf/key.pem"

## Period between keep alive probes.
## 0 disables keep alive probes.
## Defaults to the OS configuration.
## Only applies to stream sockets (e.g. TCP).
# keep_alive_period = "5m"

## Maximum number of concurrent connections (default = 0).
## 0 means unlimited.
## Only applies to stream sockets (e.g. TCP).
# max_connections = 1024

## Read timeout (default = 500ms).
## 0 means unlimited.
# read_timeout = 500ms

## Whether to parse in best effort mode or not (default = false).
## By default best effort parsing is off.
# best_effort = false

## Character to prepend to SD-PARAMs (default = "_").
## A syslog message can contain multiple parameters and multiple identifiers within structured data section.
## Eg., [id1 name1="val1" name2="val2"][id2 name1="val1" nameA="valA"]
## For each combination a field is created.
## Its name is created concatenating identifier, sdparam_separator, and parameter name.
# sdparam_separator = "_"
```

#### Other configs

Other available configurations are:

- `keep_alive_period`, `max_connections` for stream sockets
- `read_timeout`
- `best_effort` to tell the parser to work until it is able to do and extract partial but valid info (more [here](https://github.com/influxdata/go-syslog#best-effort-mode))
- `sdparam_separator` to choose how to separate structured data param name from its structured data identifier

### Metrics

- syslog
- fields
- **version** (`uint16`)
- **severity_code** (`int`)
- **facility_code** (`int`)
- timestamp (`int`)
- procid (`string`)
- msgid (`string`)
- *sdid* (`bool`)
- *sdid . sdparam_separator . sdparam_name* (`string`)
- tags
- **severity** (`string`)
- **facility** (`string`)
- hostname (`string`)
- appname (`string`)

The name of fields in _italic_ corresponds to their runtime value.

The fields/tags which name is in **bold** will always be present when a valid Syslog message has been received.

### RSYSLOG integration

The following instructions illustrate how to configure a syslog transport sender as per RFC5425 - ie., using the octect framing technique - via RSYSLOG.

Install `rsyslog`.

Give it a configuration - ie., `/etc/rsyslog.conf`.

```
$ModLoad imuxsock # provides support for local system logging
$ModLoad imklog # provides kernel logging support
$ModLoad immark # provides heart-beat logs
$FileOwner root
$FileGroup root
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022
$WorkDirectory /var/spool/rsyslog # default location for work (spool) files
$ActionQueueType LinkedList # use asynchronous processing
$ActionQueueFileName srvrfwd # set file name, also enables disk mode
$ActionResumeRetryCount -1 # infinite retries on insert failure
$ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down
$IncludeConfig /etc/rsyslog.d/*.conf
```

Specify you want the octet framing technique enabled and the format of each syslog message to follow the RFC5424.

Create a file - eg., `/etc/rsyslog.d/50-default.conf` - containing:

```
*.* @@(o)127.0.0.1:6514;RSYSLOG_SyslogProtocol23Format
```

To complete the TLS setup please refer to [rsyslog docs](https://www.rsyslog.com/doc/v8-stable/tutorials/tls.html).

Notice that this configuration tells `rsyslog` to broadcast messages to `127.0.0.1>6514`.

So you have to configure this plugin accordingly.
Loading

0 comments on commit 32edb48

Please sign in to comment.