- Overview
- Module Description - What the module does and why it is useful
- Setup - The basics of getting started with graylog_collector
- Usage - Configuration options and additional functionality
- Reference
- Examples
- Limitations - OS compatibility, etc.
- Development - Guide for contributing to the module
- Authors
Puppet module to manage Graylog Collector
You should refer to the Graylog Collector Documentation at http://docs.graylog.org/en/latest/pages/collector.html to aide you in the use of this module. Most parameters map to the configuration options.
Manages the installation and configuration of Graylog Collector, including configuring inputs and outputs.
- Installs the Graylog Collector via archive or package
- Optionally manages package managment repos
- Optionally manages the user/group to run as
- Optionally provides an init script and optionally manages the service
- Manages the Graylog Collector configuration file
Requires an installation of Java, which is not managed by this module.
Try puppetlabs-java
If using Ubuntu or Debian and managing the graylog-collector repo with this module, you'll need puppetlabs-apt
- Declare the base class and provide a server url to connect to.
- Declare any inputs/outputs you'd like to manage.
The base class "entry point" for the module.
Manages input configurations
Manages output configurations
Manages installing Graylog Collector from archives
Manages the configuration for Graylog Collector
Manages the Graylog Collector service
Refer to http://docs.graylog.org/en/latest/pages/collector.html#global-settings
server_url
Required if enable_registration is true. Specifies the URL (including port) for the Graylog2 server.
enable_registration
Boolean. Default: true
collector_id
Default: 'file:/etc/graylog/collector/config/collector-id'
install_path
Default: '/usr/share'
Specifies the path to install Graylog Collector to.
Note: If installing via archive, this will append the version to the directory, as it's extracted from the tarball. A symlink will be created called "graylog-collector" that links to it.
config_dir
Default: '/etc/graylog/collector'
The directory for the Graylog Collector configuration file.
sysconfig_dir
Default: '/etc/default' or '/etc/sysconfig', depending on OS.
The sysconfig path. This is typically /etc/default or /etc/sysconfig
java_cmd
Default: '/usr/bin/java'
The absolute path to the java executable to use for running the Graylog
Collector.
java_opts
Default: undef
Any additional Java options to pass when running Graylog Collector.
user
Default: 'root'
The user to run Graylog Collector as. Keep in mind that the collector needs access to whatever files you feed to it.
group
Default: 'root'
The group for the collector. This is used for ownership of the config files.
manage_user
Default: false
Specifies whether this module should manage the user. If you're running as root, you probably shouldn't let this module manage the user.
manage_group
Default: false
Specifies whether this module should manage the group. If you're running as root, you probably shouldn't let this module manage the group.
manage_init
Default: undef
Specifies whether this module should manage the init script or init config file.
If install_from is set to package, this manage_init defaults to false
If installing from archive, it defaults to true
manage_service
Default: true
Specifies whether this module should manage the service.
service_ensure
Default: 'running'
State of the service.
service_enable
Default: true
Enable the service on boot or not.
service_name
Default: 'graylog-collector'
The name of the service to manage if manage_service is true.
install_from
Default: depends on platform
Possible values are package and archive
Defaults to package for:
- Ubuntu 12x
- Ubuntu 14x
- Debian 8
- EL 7
Defaults to archive for everything else.
Refer to the Graylog Collector documentation for information about package repositories.
version
Default: depends on installation method
For archive, it defaults to 0.4.1. For package, it defaults to installed
The version of Graylog Collector to install.
source_url
Default: 'https://packages.graylog2.org/releases/graylog-collector/graylog-collector-0.4.0.tgz'
This is not relevant if install_from is set to package
URL to download the tarball from for installation when install_from is
set to 'archive'
service_file
Absolute path to the location of the init script or service definition.
service_template
Optional custom template to use for the service file/init script.
manage_repo
Whether this module should manage the package repositories. This is only relevant if installing via package.
Refer to http://docs.graylog.org/en/latest/pages/collector.html#input-settings and http://docs.graylog.org/en/latest/pages/collector.html#input-output-routing
input_name
Default: $title
The name of the input to manage
path
Default: undef
If collecting an absolute path, this should refer to it.
type
Default: 'file'
path_glob_root
Default: undef
If you're collecting a file glob (e.g. *.log), this should refer to the base directory that contains that glob.
Example: '/usr/share/tomcat/logs'
path_glob_pattern
Default: undef
The glob pattern used for collection.
Example: '.log' or '.{err,log}'
content_splitter
Default: 'NEWLINE'
How to split the content. Basically - how to separate messages. Valid options are 'NEWLINE' or 'PATTERN'
content_splitter_pattern
Default: undef
If splitting on 'PATTERN', what regex should be used to separate log messages.
Note that regex escapes must be escaped. So a normal regex pattern of \d
should be \\d
charset
Default: 'UTF-8'
reader_interval
Default: '100ms'
global_message_fields
Hash.
Default: {}
Message fields allow for you to add your own fields to a log message. For example, appending the application name or some other metadata. The 'global' message fields parameter is intended for situations where you'd like to apply a global set of fields to all inputs, but allow for application-specific fields to be added. Since these are hashes, you would normally need a wrapper or some other trickery to combine a global set of fields with application-specific fields, or declare your global fields with every declaration of an input. This is useful, for example, when you'd like to set message fields with something like Hiera or in a base profile.
The global message fields will be merged with the 'message_fields' parameter.
Example:
global_message_fields {
'fqdn' => $::fqdn,
'org_environment' => $::org_environment,
}message_fields
Hash.
Default: {}
Refer to 'global_message_fields' above. These are fields in addition to the 'global_message_fields'. These take precedence, however.
global_message_fields {
'application_name' => 'tomcat',
'app' => 'pluto',
'logfile' => 'catalina.out',
}outputs
Default: undef
Array.
The outputs to send this input to. See graylog::collector::output You can
specify an output on the input or an input on the output (whoa).
Refer to http://docs.graylog.org/en/latest/pages/collector.html#output-settings and http://docs.graylog.org/en/latest/pages/collector.html#input-output-routing
output_name
Default: title
The name of the output to configure. Example: 'gelf'
type
Default: gelf
The type of output. Valid options are 'gelf' and 'stdout'
host
Default: undef
If type is 'gelf', this refers to the host to send logs to. The Graylog2 server should have a GELF TCP input configured and listening.
port
Default: undef
If type is 'gelf', this refers to the port to send logs to. The Graylog2 server should have a GELF TCP input configured and listening.
client_tls
Boolean.
Default: false
Specifies whether to use TLS to connect when the output is 'gelf'
client_tls_cert_chain_file
Default: undef
client_tls_verify_cert
Boolean.
Default: true
client_queue_size
Default: '512'
client_connect_timeout
Default: '5000'
client_reconnect_delay
Default: '1000'
client_tcp_no_delay
Boolean.
Default: true
client_send_buffer_size
Default: '-1'
inputs
Array.
Default: undef
Works like 'outputs' with graylog_collector::input. What inputs should
be sent to this output. You can leave this empty and specify them with the
input if you'd like.
Example of declaring the common configuration in a base profile:
class profile::base::graylog {
class { '::graylog_collector':
server_url => '8.8.8.8:12900',
}
graylog_collector::output { 'gelf':
type => 'gelf',
host => '8.8.8.8',
port => '12900',
}
}Example of managing an 'input' for a specific application:
class profile::apps::pluto {
graylog_collector::input { 'pluto_catalina_out':
path => "${catalina_base}/logs/catalina.out",
content_splitter => 'PATTERN',
content_splitter_pattern => '^(\\d{4}-\\d{2}-\\d{2}|\\d{2}-\\w{3}-\\d{4})\\s\\d{1,2}:\\d{1,2}:\\d{1,2}',
message_fields => {
'application_name' => 'tomcat',
'app' => 'pluto',
'logfile' => 'catalina.out',
}
}
}Example of a file glob:
class profile::apps::pluto {
graylog_collector::input { 'pluto_tomcat_logs':
path_glob_root => "${catalina_base}/logs",
path_glob_pattern => "*.{out,log,txt}",
content_splitter => 'PATTERN',
content_splitter_pattern => '^(\\d{4}-\\d{2}-\\d{2}|\\d{2}-\\w{3}-\\d{4})\\s\\d{1,2}:\\d{1,2}:\\d{1,2}',
message_fields => {
'application_name' => 'tomcat',
'app' => 'pluto',
}
}
}An example of using a collector to add 'global_message_fields' to all inputs:
Graylog_collector::Input <| |> {
outputs => [ 'gelf' ],
global_message_fields => {
'fqdn' => $::fqdn,
'org_environment' => $::org_environment,
}
}Tested on:
- CentOS 6
- CentOS 7
- Ubuntu 14.04
-
Extend package installation. Waiting on upstream for EL6. Graylog2/fpm-recipes#42 Also track https://github.com/Graylog2/fpm-recipes
-
Add additional operating system support
-
Add testing
-
Cleanups, especially around installation and service management
Contributions are more than welcome! Reporting issues or code contributions.
- Fork this repo
- Do your work
- Create a pull request
- Josh Beard ([email protected])
- @cinacio
- @leonkyneur
- @hggh