From 07d4f7a43039b29c544b5315ddfe5febdd768299 Mon Sep 17 00:00:00 2001 From: Jaime Soriano Pastor Date: Sun, 9 Dec 2018 20:16:22 +0100 Subject: [PATCH] Log by default to journald on systems with systemd (#8942) Add `-e` flag to command line when running beats from systemd so beats logs are collected by journald by default in systems with systemd. Add also some environment variables so command line flags can be more easily overriden with a drop-in unit or with `systemctl edit`. Add documentation about the use of Beats with systemd. --- CHANGELOG-developer.asciidoc | 1 + CHANGELOG.asciidoc | 1 + auditbeat/docs/setting-up-running.asciidoc | 4 + .../templates/linux/systemd.unit.tmpl | 5 +- filebeat/docs/setting-up-running.asciidoc | 4 + heartbeat/docs/setting-up-running.asciidoc | 4 + journalbeat/docs/setting-up-running.asciidoc | 3 + libbeat/docs/shared-systemd.asciidoc | 103 ++++++++++++++++++ metricbeat/docs/setting-up-running.asciidoc | 4 + packetbeat/docs/setting-up-running.asciidoc | 4 + 10 files changed, 132 insertions(+), 1 deletion(-) create mode 100644 libbeat/docs/shared-systemd.asciidoc diff --git a/CHANGELOG-developer.asciidoc b/CHANGELOG-developer.asciidoc index 7cb0cb7e3f5..fea932d0633 100644 --- a/CHANGELOG-developer.asciidoc +++ b/CHANGELOG-developer.asciidoc @@ -31,6 +31,7 @@ The list below covers the major changes between 6.3.0 and master only. used because the value is specified in magefile.go. {pull}7670[7670] - Outputs must implement String. {pull}6404[6404] - Renamed `-beat-name` CLI option used in `kibana_index_pattern.go` to `-beat` for consistency with other scripts in `dev-tools/cmd`. {pull}8615[8615] +- Systemd unit file template used on Linux packaging now includes environment variables to ease flag overriding. One of them includes the `-e` flag, making beats log to stderr by default on systemd uses. {pull}8942[8942] - Removed dashboards and index patterns generation for Kibana 5. {pull}8927[8927] - Move generator packages of Filebeat from `scripts/generator` to `generator`. {pull}9147[9147] diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc index 12959bb5147..f849cb68846 100644 --- a/CHANGELOG.asciidoc +++ b/CHANGELOG.asciidoc @@ -133,6 +133,7 @@ https://github.com/elastic/beats/compare/v6.5.0...v7.0.0-alpha1[View commits] *Affecting all Beats* - Dissect syntax change, use * instead of ? when working with field reference. {issue}8054[8054] +- On systems with systemd, the Beats log is now written to journald by default rather than file. To revert this behaviour override BEAT_LOG_OPTS with an empty value. {pull}8942[8942]. - Removed dashboards and index patterns generation for Kibana 5. {pull}8927[8927] *Auditbeat* diff --git a/auditbeat/docs/setting-up-running.asciidoc b/auditbeat/docs/setting-up-running.asciidoc index f76b2d13c31..eddc1c4e84c 100644 --- a/auditbeat/docs/setting-up-running.asciidoc +++ b/auditbeat/docs/setting-up-running.asciidoc @@ -22,6 +22,8 @@ This section includes additional information on how to set up and run * <> +* <> + //MAINTAINERS: If you add a new file to this section, make sure you update the bulleted list ^^ too. @@ -35,4 +37,6 @@ include::./running-on-docker.asciidoc[] include::./running-on-kubernetes.asciidoc[] +include::../../libbeat/docs/shared-systemd.asciidoc[] + include::../../libbeat/docs/shared-shutdown.asciidoc[] diff --git a/dev-tools/packaging/templates/linux/systemd.unit.tmpl b/dev-tools/packaging/templates/linux/systemd.unit.tmpl index 5725ba3e2a3..2b91a480fee 100644 --- a/dev-tools/packaging/templates/linux/systemd.unit.tmpl +++ b/dev-tools/packaging/templates/linux/systemd.unit.tmpl @@ -5,7 +5,10 @@ Wants=network-online.target After=network-online.target [Service] -ExecStart=/usr/share/{{.BeatName}}/bin/{{.BeatName}} -c /etc/{{.BeatName}}/{{.BeatName}}.yml -path.home /usr/share/{{.BeatName}} -path.config /etc/{{.BeatName}} -path.data /var/lib/{{.BeatName}} -path.logs /var/log/{{.BeatName}} +Environment=BEAT_LOG_OPTS="-e" +Environment=BEAT_CONFIG_OPTS="-c /etc/{{.BeatName}}/{{.BeatName}}.yml" +Environment=BEAT_PATH_OPTS="-path.home /usr/share/{{.BeatName}} -path.config /etc/{{.BeatName}} -path.data /var/lib/{{.BeatName}} -path.logs /var/log/{{.BeatName}}" +ExecStart=/usr/share/{{.BeatName}}/bin/{{.BeatName}} $BEAT_LOG_OPTS $BEAT_CONFIG_OPTS $BEAT_PATH_OPTS Restart=always [Install] diff --git a/filebeat/docs/setting-up-running.asciidoc b/filebeat/docs/setting-up-running.asciidoc index f76b2d13c31..eddc1c4e84c 100644 --- a/filebeat/docs/setting-up-running.asciidoc +++ b/filebeat/docs/setting-up-running.asciidoc @@ -22,6 +22,8 @@ This section includes additional information on how to set up and run * <> +* <> + //MAINTAINERS: If you add a new file to this section, make sure you update the bulleted list ^^ too. @@ -35,4 +37,6 @@ include::./running-on-docker.asciidoc[] include::./running-on-kubernetes.asciidoc[] +include::../../libbeat/docs/shared-systemd.asciidoc[] + include::../../libbeat/docs/shared-shutdown.asciidoc[] diff --git a/heartbeat/docs/setting-up-running.asciidoc b/heartbeat/docs/setting-up-running.asciidoc index a9991ceffde..bacd7e60342 100644 --- a/heartbeat/docs/setting-up-running.asciidoc +++ b/heartbeat/docs/setting-up-running.asciidoc @@ -20,6 +20,8 @@ This section includes additional information on how to set up and run * <> +* <> + //MAINTAINERS: If you add a new file to this section, make sure you update the bulleted list ^^ too. include::../../libbeat/docs/shared-directory-layout.asciidoc[] @@ -30,4 +32,6 @@ include::../../libbeat/docs/command-reference.asciidoc[] include::./running-on-docker.asciidoc[] +include::../../libbeat/docs/shared-systemd.asciidoc[] + include::../../libbeat/docs/shared-shutdown.asciidoc[] diff --git a/journalbeat/docs/setting-up-running.asciidoc b/journalbeat/docs/setting-up-running.asciidoc index aeed49f8051..b18bf67cf3c 100644 --- a/journalbeat/docs/setting-up-running.asciidoc +++ b/journalbeat/docs/setting-up-running.asciidoc @@ -17,6 +17,7 @@ This section includes additional information on how to set up and run * <> * <> * <> +* <> * <> @@ -28,4 +29,6 @@ include::../../libbeat/docs/keystore.asciidoc[] include::../../libbeat/docs/command-reference.asciidoc[] +include::../../libbeat/docs/shared-systemd.asciidoc[] + include::../../libbeat/docs/shared-shutdown.asciidoc[] diff --git a/libbeat/docs/shared-systemd.asciidoc b/libbeat/docs/shared-systemd.asciidoc new file mode 100644 index 00000000000..7192ad71504 --- /dev/null +++ b/libbeat/docs/shared-systemd.asciidoc @@ -0,0 +1,103 @@ +[[running-with-systemd]] +=== {beatname_uc} and systemd + +The DEB and RPM packages include a service unit for Linux systems with +systemd. On these systems, you can manage {beatname_uc} by using the usual +systemd commands. + +==== Start and stop {beatname_uc} + +Use `systemctl` to start or stop {beatname_uc}: + +["source", "sh", subs="attributes"] +------------------------------------------------ +systemctl start {beatname_lc} +------------------------------------------------ + +["source", "sh", subs="attributes"] +------------------------------------------------ +systemctl stop {beatname_lc} +------------------------------------------------ + +By default, the {beatname_uc} service starts automatically when the system +boots. To enable or disable auto start use: + +["source", "sh", subs="attributes"] +------------------------------------------------ +systemctl enable {beatname_lc} +------------------------------------------------ + +["source", "sh", subs="attributes"] +------------------------------------------------ +systemctl disable {beatname_lc} +------------------------------------------------ + + +==== {beatname_uc} status and logs + +To get the service status, use `systemctl`: + +["source", "sh", subs="attributes"] +------------------------------------------------ +systemctl status {beatname_lc} +------------------------------------------------ + +Logs are stored by default in journald. To view the Logs, use `journalctl`: + +["source", "sh", subs="attributes"] +------------------------------------------------ +journalctl -u {beatname_lc}.service +------------------------------------------------ + +NOTE: The unit file included in the packages sets the `-e` flag by default. +This flag makes {beatname_uc} log to stderr and disables other log outputs. +Systemd stores all output sent to stderr in journald. + +[float] +=== Customize systemd unit for {beatname_uc} + +The systemd service unit file includes environment variables that you can +override to change the default options. + +[cols="> +* <> + //MAINTAINERS: If you add a new file to this section, make sure you update the bulleted list ^^ too. include::../../libbeat/docs/shared-directory-layout.asciidoc[] @@ -34,4 +36,6 @@ include::./running-on-docker.asciidoc[] include::./running-on-kubernetes.asciidoc[] +include::../../libbeat/docs/shared-systemd.asciidoc[] + include::../../libbeat/docs/shared-shutdown.asciidoc[] diff --git a/packetbeat/docs/setting-up-running.asciidoc b/packetbeat/docs/setting-up-running.asciidoc index a9991ceffde..bacd7e60342 100644 --- a/packetbeat/docs/setting-up-running.asciidoc +++ b/packetbeat/docs/setting-up-running.asciidoc @@ -20,6 +20,8 @@ This section includes additional information on how to set up and run * <> +* <> + //MAINTAINERS: If you add a new file to this section, make sure you update the bulleted list ^^ too. include::../../libbeat/docs/shared-directory-layout.asciidoc[] @@ -30,4 +32,6 @@ include::../../libbeat/docs/command-reference.asciidoc[] include::./running-on-docker.asciidoc[] +include::../../libbeat/docs/shared-systemd.asciidoc[] + include::../../libbeat/docs/shared-shutdown.asciidoc[]