Update info about loading dashboards and index templates

auditbeat/docs/configuring-howto.asciidoc

@@ -23,9 +23,11 @@ The following topics describe how to configure {beatname_uc}:
 * <<configuration-general-options>>
 * <<{beatname_lc}-configuration-reloading>>
 * <<configuring-output>>
+* <<configuration-ssl>>
 * <<filtering-and-enhancing-data>>
 * <<configuring-ingest-node>>
 * <<configuration-path>>
+* <<setup-kibana-endpoint>>
 * <<configuration-dashboards>>
 * <<configuration-template>>
 * <<configuration-logging>>
@@ -46,12 +48,16 @@ include::./reload-configuration.asciidoc[]
 :allplatforms:
 include::../../libbeat/docs/outputconfig.asciidoc[]
 
+include::../../libbeat/docs/shared-ssl-config.asciidoc[]
+
 include::./auditbeat-filtering.asciidoc[]
 
 include::../../libbeat/docs/shared-config-ingest.asciidoc[]
 
 include::../../libbeat/docs/shared-path-config.asciidoc[]
 
+include::../../libbeat/docs/shared-kibana-config.asciidoc[]
+
 include::../../libbeat/docs/setup-config.asciidoc[]
 
 include::../../libbeat/docs/loggingconfig.asciidoc[]

auditbeat/docs/faq-ulimit.asciidoc

@@ -0,0 +1,29 @@
+[float]
+[[ulimit]]
+=== {beatname_uc} fails to watch folders because too many files are open?
+
+Because of the way file monitoring is implemented on macOS, you may see a
+warning similar to the following:
+
+[source,shell]
+----
+eventreader_fsnotify.go:42: WARN [audit.file] Failed to watch /usr/bin: too many
+open files (check the max number of open files allowed with 'ulimit -a')
+----
+
+To resolve this issue, run {beatname_uc} with the `ulimit` set to a larger
+value, for example:
+
+["source","sh",subs="attributes"]
+----
+sudo sh -c 'ulimit -n 8192 && ./{beatname_uc} -e
+----
+
+Or:
+
+["source","sh",subs="attributes"]
+----
+sudo su
+ulimit -n 8192
+./{beatname_lc} -e
+----

auditbeat/docs/faq.asciidoc

@@ -5,5 +5,8 @@ This section contains frequently asked questions about {beatname_uc}. Also check
 out the
 https://discuss.elastic.co/c/beats/{beatname_lc}[{beatname_uc} discussion forum].
 
+include::./faq-ulimit.asciidoc[]
+
 include::../../libbeat/docs/faq-limit-bandwidth.asciidoc[]
+
 include::../../libbeat/docs/shared-faq.asciidoc[]

auditbeat/docs/getting-started.asciidoc

@@ -1,8 +1,6 @@
 [id="{beatname_lc}-getting-started"]
 == Getting started with {beatname_uc}
 
-//TODO: The getting started content needs to be changed to reflect changes in command-line syntax, dashboard loading, etc.
-
 To get started with your own {beatname_uc} setup, install and configure these
 related products:
 
@@ -19,8 +17,9 @@ install, configure, and run {beatname_uc}:
 * <<{beatname_lc}-installation>>
 * <<{beatname_lc}-configuration>>
 * <<{beatname_lc}-template>>
+* <<load-kibana-dashboards>>
 * <<{beatname_lc}-starting>>
-* <<{beatname_lc}-sample-dashboards>>
+* <<view-kibana-dashboards>>
 * <<command-line-options>>
 * <<directory-layout>>
 
@@ -162,53 +161,43 @@ auditbeat.modules:
 -------------------------------------
 +
 If you accept the default configuration without specifying additional modules,
-{beatname_uc} uses a default configuration that's tailored to the operating system
-where {beatname_uc} is running.
+{beatname_uc} uses a configuration that's tailored to the operating system where
+{beatname_uc} is running.
 +
 See <<configuring-howto-{beatname_lc}>> for more details about configuring modules.
 
-. If you are sending output to Elasticsearch, set the IP address and port where
-{beatname_uc} can find the Elasticsearch installation:
+. If you are sending output to Elasticsearch (and not using Logstash), set the
+IP address and port where {beatname_uc} can find the Elasticsearch installation:
 +
 [source,yaml]
 ----------------------------------------------------------------------
 output.elasticsearch:
-  hosts: ["192.168.1.42:9200"]
+  hosts: ["127.0.0.1:9200"]
 ----------------------------------------------------------------------
 +
-If you are sending output to Logstash, see
+If you are sending output to Logstash, make sure you
 <<logstash-output,Configure the Logstash output>> instead.
 
-. If you plan to use the sample Kibana dashboards provided with {beatname_uc},
-configure the Kibana endpoint:
-+
-[source,yaml]
-----------------------------------------------------------------------
-setup.kibana:
-  host: "localhost:5601"
-----------------------------------------------------------------------
-+
---
-Where `host` is the hostname and port of the machine where Kibana is running,
-for example, `localhost:5601`.
-
-TIP: If you specify a path after the port number, you need to include
-the scheme and port: `http://localhost:5601/path`.
+include::../../libbeat/docs/step-configure-kibana-endpoint.asciidoc[]
 
---
+include::../../libbeat/docs/step-test-config.asciidoc[]
 
-//TODO: Add link that mentions security settings and points to docs about setup.kibana for more info.
-
-//REVIEWERS: Wondering if I should protocol, username, and password in the example. We tend to leave that out, but not sure if that's the best thing to do.
+include::../../libbeat/docs/step-look-at-config.asciidoc[]
 
 [id="{beatname_lc}-template"]
 === Step 3: Load the index template in Elasticsearch
 
 :allplatforms:
 include::../../libbeat/docs/shared-template-load.asciidoc[]
 
+[[load-kibana-dashboards]]
+=== Step 4: Set up the Kibana dashboards
+
+:allplatforms:
+include::../../libbeat/docs/dashboards.asciidoc[]
+
 [id="{beatname_lc}-starting"]
-=== Step 4: Start {beatname_uc}
+=== Step 5: Start {beatname_uc}
 
 Run {beatname_uc} by issuing the appropriate command for your platform.
 
@@ -242,6 +231,9 @@ need to change ownership of the configuration file (see
 {libbeat}/config-file-permissions.html[Config File Ownership and Permissions]
 in the _Beats Platform Reference_).
 
+If you see a warning about too many open files, you need to increase the
+`ulimit`. See the <<ulimit,FAQ>> for more details.
+
 *win:*
 
 ["source","sh",subs="attributes"]
@@ -267,15 +259,18 @@ Elasticsearch instance.
 On Windows, if you don't have cURL installed, simply point your browser to the
 URL.
 
-[id="{beatname_lc}-sample-dashboards"]
-=== Step 5: Load sample Kibana dashboards
+[[view-kibana-dashboards]]
+=== Step 6: View the sample Kibana dashboards
 
 To make it easier for you to start auditing the activities of users and
-processes on your system, we have created sample {beatname_uc} dashboards.
+processes on your system, we have created example {beatname_uc} dashboards.
+You loaded the dashboards earlier when you ran the `setup` command.
+
+include::../../libbeat/docs/opendashboards.asciidoc[]
+
 The dashboards are provided as examples. We recommend that you
 {kibana-ref}/dashboard.html[customize] them to meet your needs.
 
 image:./images/auditbeat-file-integrity-dashboard.png[Auditbeat File Integrity Dashboard]
 
-:allplatforms:
-include::../../libbeat/docs/dashboards.asciidoc[]
+

auditbeat/docs/index.asciidoc

@@ -2,8 +2,6 @@
 
 include::../../libbeat/docs/version.asciidoc[]
 
-:release-state: unreleased
-
 :libbeat: http://www.elastic.co/guide/en/beats/libbeat/{doc-branch}
 :kibana-ref: https://www.elastic.co/guide/en/kibana/{doc-branch}
 :beatsdevguide: http://www.elastic.co/guide/en/beats/devguide/{doc-branch}

auditbeat/docs/reload-configuration.asciidoc

@@ -45,3 +45,7 @@ auditbeat.modules:
     - /www/wordpress/wp-content
     - /www/wordpress/wp-includes
 ------------------------------------------------------------------------------
+
+NOTE: On systems with POSIX file permissions, all Beats configuration files are
+subject to ownership and file permission checks. If you encounter config loading
+errors related to file ownership, see {libbeat}/config-file-permissions.html.

filebeat/docs/configuring-howto.asciidoc

@@ -24,9 +24,11 @@ The following topics describe how to configure Filebeat:
 * <<configuration-general-options>>
 * <<filebeat-configuration-reloading>>
 * <<configuring-output>>
+* <<configuration-ssl>>
 * <<filtering-and-enhancing-data>>
 * <<configuring-ingest-node>>
 * <<configuration-path>>
+* <<setup-kibana-endpoint>>
 * <<configuration-dashboards>>
 * <<configuration-template>>
 * <<configuration-logging>>
@@ -50,6 +52,8 @@ include::./reload-configuration.asciidoc[]
 :allplatforms:
 include::../../libbeat/docs/outputconfig.asciidoc[]
 
+include::../../libbeat/docs/shared-ssl-config.asciidoc[]
+
 include::./load-balancing.asciidoc[]
 
 include::./filebeat-filtering.asciidoc[]
@@ -58,6 +62,8 @@ include::../../libbeat/docs/shared-config-ingest.asciidoc[]
 
 include::../../libbeat/docs/shared-path-config.asciidoc[]
 
+include::../../libbeat/docs/shared-kibana-config.asciidoc[]
+
 include::../../libbeat/docs/setup-config.asciidoc[]
 
 include::../../libbeat/docs/loggingconfig.asciidoc[]

filebeat/docs/getting-started.asciidoc

@@ -15,8 +15,9 @@ After installing the Elastic Stack, read the following topics to learn how to in
 * <<filebeat-configuration>>
 * <<config-filebeat-logstash>>
 * <<filebeat-template>>
+* <<load-kibana-dashboards>>
 * <<filebeat-starting>>
-* <<filebeat-index-pattern>>
+* <<view-kibana-dashboards>>
 * <<filebeat-modules-quickstart>>
 * <<command-line-options>>
 * <<directory-layout>>
@@ -157,6 +158,7 @@ default values for most configuration options.
 -------------------------------------------------------------------------------------
 filebeat.prospectors:
 - type: log
+  enabled: true
   paths:
     - /var/log/*.log
     #- c:\programdata\elasticsearch\logs\*
@@ -172,6 +174,7 @@ For the most basic Filebeat configuration, you can define a single prospector wi
 -------------------------------------------------------------------------------------
 filebeat.prospectors:
 - type: log
+  enabled: true
   paths:
     - /var/log/*.log
 -------------------------------------------------------------------------------------
@@ -185,26 +188,23 @@ To fetch all files from a predefined level of subdirectories, the following patt
 fetch log files from the `/var/log` folder itself. Currently it is not possible to recursively
 fetch all files in all subdirectories of a directory.
 
-. If you are sending output to Elasticsearch, set the IP address and port where Filebeat can find the Elasticsearch installation:
+. If you are sending output directly to Elasticsearch (and not using Logstash),
+set the IP address and port where Filebeat can find the Elasticsearch installation:
 +
 [source,yaml]
 ----------------------------------------------------------------------
 output.elasticsearch:
   hosts: ["192.168.1.42:9200"]
 ----------------------------------------------------------------------
 +
-If you are sending output to Logstash, see <<config-filebeat-logstash>> instead.
+If you are sending output to Logstash, make sure you
+configure the Logstash output in <<config-filebeat-logstash>>.
 
-TIP: To test your configuration file, change to the directory where the Filebeat
-binary is installed, and run Filebeat in the foreground with the following
-options specified: +./filebeat -configtest -e+. Make sure your config files are
-in the path expected by Filebeat (see <<directory-layout>>). If you
-installed from DEB or RPM packages, run +./filebeat.sh -configtest -e+.
+include::../../libbeat/docs/step-configure-kibana-endpoint.asciidoc[]
 
-Before starting Filebeat, you should look at the configuration options in the
-configuration file, for example `C:\Program Files\Filebeat\filebeat.yml` or
-`/etc/filebeat/filebeat.yml`. For more information about these options,
-see <<configuring-howto-filebeat>>.
+include::../../libbeat/docs/step-test-config.asciidoc[]
+
+include::../../libbeat/docs/step-look-at-config.asciidoc[]
 
 [[config-filebeat-logstash]]
 === Step 3: Configure Filebeat to use Logstash
@@ -218,15 +218,23 @@ include::../../libbeat/docs/shared-logstash-config.asciidoc[]
 :allplatforms:
 include::../../libbeat/docs/shared-template-load.asciidoc[]
 
+[[load-kibana-dashboards]]
+=== Step 5: Set up the Kibana dashboards
+
+:allplatforms:
+include::../../libbeat/docs/dashboards.asciidoc[]
+
 [[filebeat-starting]]
-=== Step 5: Start Filebeat
+=== Step 6: Start Filebeat
 
 Start Filebeat by issuing the appropriate command for your platform.
 
 NOTE: If you use an init.d script to start Filebeat on deb or rpm, you can't
 specify command line flags (see <<command-line-options>>). To specify flags,
 start Filebeat in the foreground.
 
+//REVIEWERS: With the new command syntax, should the deb and rpm instructions here say run instead of start?
+
 *deb:*
 
 [source,shell]
@@ -267,28 +275,33 @@ in the _Beats Platform Reference_).
 PS C:\Program Files\Filebeat> Start-Service filebeat
 ----------------------------------------------------------------------
 
+
 By default, Windows log files are stored in `C:\ProgramData\filebeat\Logs`.
 
 Filebeat is now ready to send log files to your defined output.
 
-[[filebeat-index-pattern]]
-=== Step 6: Load the Kibana index pattern
+[[view-kibana-dashboards]]
+=== Step 7: View the sample Kibana dashboards
 
-We don't offer prebuilt dashboards for visualizing Filebeat data. However, to make it easier for you to explore Filebeat
-data in Kibana, we've created a Filebeat index pattern: `filebeat-*`. To load this pattern, you can use the script
-that's provided for importing dashboards:
+To make it easier for you to explore Filebeat data in Kibana, we've created
+example {beatname_uc} dashboards. You loaded the dashboards earlier when you
+ran the `setup` command.
 
-[source,shell]
-----------------------------------------------------------------------
-./scripts/import_dashboards -only-index
-----------------------------------------------------------------------
+include::../../libbeat/docs/opendashboards.asciidoc[]
 
-For more information about running the script, see
-{beatsdevguide}/import-dashboards.html[Importing Existing Beat Dashboards] in the _Beats Platform Reference_.
+These dashboards are designed to work out-of-the box when you use
+<<filebeat-modules-overview,Filebeat modules>>. However, you can also use them
+as examples and {kibana-ref}/dashboard.html[customize] them to meet your needs
+even if you aren't using Filebeat modules.
 
-After you've created the index pattern, you can select the `filebeat-*` index pattern in Kibana to explore
-Filebeat data.
+To populate the example dashboards with data, you need to either
+<<configuring-ingest-node,define ingest node pipelines>> or use Logstash to
+parse the data into the fields expected by the dashboards. If you are using
+Logstash, see the
+{logstashdoc}/logstash-config-for-filebeat-modules.html[configuration examples]
+in the Logstash documentation for help parsing the log formats supported
+by the dashboards.
 
-image:./images/filebeat-discover-tab.png[]
+Here is an example of the Filebeat system dashboard:
 
-TIP: If you don't see `filebeat-*` in the list of available index patterns, try refreshing the page in your browser.
+image:./images/kibana-system.png[]

filebeat/docs/index.asciidoc

@@ -3,6 +3,7 @@
 include::../../libbeat/docs/version.asciidoc[]
 
 :libbeat: http://www.elastic.co/guide/en/beats/libbeat/{doc-branch}
+:kibana-ref: https://www.elastic.co/guide/en/kibana/{doc-branch}
 :beatsdevguide: http://www.elastic.co/guide/en/beats/devguide/{doc-branch}
 :packetbeat: http://www.elastic.co/guide/en/beats/packetbeat/{doc-branch}
 :metricbeat: http://www.elastic.co/guide/en/beats/metricbeat/{doc-branch}

filebeat/docs/migration.asciidoc

@@ -220,7 +220,7 @@ output.logstash:
     <<loadbalance>> configuration option.
 <2> Note that if the `ssl` settings are missing, then SSL is disabled. SSL is
     automatically enabled when you add any of the `ssl` options. For more information about
-    specific configuration options, see <<configuration-output-ssl>>.
+    specific configuration options, see <<configuration-ssl>>.
 
 
 [[changed-configuration-options]]