This Ansible role installs Consul, including establishing a filesystem structure and server or client agent configuration with support for some common operational features.
It can bootstrap a development or evaluation cluster of 3 server agents running in a Vagrant and VirtualBox based environment. See README_VAGRANT.md and the associated Vagrantfile for more details about the developer mode setup.
“Another flaw in the human character is that everybody wants to build and nobody wants to do maintenance.”
― Kurt Vonnegut, Hocus Pocus
Please note that this role is more concerned with the initial installation and bootstrapping of a running cluster environment and does not currently concern itself (all that much) with performing ongoing drif^H^H^H^H maintenance of an existing cluster.
Many users have expressed that the Vagrant based environment makes getting a working local Consul server cluster environment up and running an easy process — so this role will target that experience as a primary motivator for existing.
If you get some mileage from it in other ways, then all the better!
This role requires a FreeBSD, Debian, or RHEL based Linux distribution or Windows Server 2012 R2. It might work with other software versions, but is definitely known to work with the following specific software versions:
- Consul: 1.4.0
- Ansible: 2.6.4
- Alpine Linux: 3.8
- CentOS: 7
- Debian: 9
- FreeBSD: 11
- RHEL: 7
- OracleLinux: 7
- Ubuntu: 16.04
- Windows: Server 2012 R2
Note: Do not use the ansible option -l
to limit the hosts, as
this will break populating the variables which are required to be
populated for your play to work. If you do use `-l' you may encounter
'Undefined is not JSON serializable' errors in the template.
The role uses variables defined in these three sources:
defaults/main.yml
vars/*.yml
- Hosts inventory file (see
examples/vagrant_hosts
for an example)
NOTE: The label for servers in the hosts inventory file must be
[consul_instances]
as shown in the example. The role will not function
properly if the label name is anything else.
Many of these can also be further overridden by environment variables as well; the variables are named and described below:
- Version to install
- Default value: 1.4.0
- Dictionary for translating ansible_architecture to HashiCorp architecture naming convention
- Default value: dict
- System architecture as determined by
{{ consul_architecture_map[ansible_architecture] }}
- Default value: amd64, arm, or arm64 (determined at runtime)
- Node operating system name in lowercase representation
- Default value:
{{ ansible_os_family | lower }}
- Install python and package dependencies required for the role functions.
- Default value: yes
- Consul archive download URL
- Default value:
https://releases.hashicorp.com/consul/{{ consul_version }}/consul_{{ consul_version }}_{{ consul_os }}_{{ consul_architecture }}.zip
- Package SHA256 summaries URL
- Default value:
https://releases.hashicorp.com/consul/{{ consul_version }}/{{ consul_version }}_SHA256SUMS
- Binary installation path
- Default Linux value:
/usr/local/bin
- Default Windows value:
C:\ProgramData\consul\bin
- Base configuration file path
- Default Linux value:
/etc/consul
- Default Windows value:
C:\ProgramData\consul\config
- Additional configuration directory
- Default Linux value:
{{ consul_config_path }}/consul.d
- Default Windows value:
C:\ProgramData\consul\config.d
- Data path as defined in data_dir or -data-dir
- Default Linux value:
/var/consul
- Default Windows value:
C:\ProgramData\consul\data
- Log path for use in rsyslogd configuration on Linux.
- Default Linux value:
/var/log/consul
- Override with
CONSUL_LOG_PATH
environment variable
- Override with
- Default Windows value:
C:\ProgramData\consul\log
- Log file for use in rsyslogd configuration on Linux.
- Override with
CONSUL_LOG_FILE
environment variable
- Override with
- Default Linux value:
consul.log
- Syslog facility as defined in syslog_facility
- Override with
CONSUL_SYSLOG_FACILITY
environment variable
- Override with
- Default Linux value: local0
- Owner of
rsyslogd
process on Linux.consul_log_path
's ownership is set to this user on Linux.- Override with
SYSLOG_USER
environment variable
- Override with
- Default Linux value: syslog
- Group of user running
rsyslogd
process on Linux.consul_log_path
's group ownership is set to this group on Linux.- Override with
SYSLOG_GROUP
environment variable
- Override with
- Default value: adm
- Run path for PID file
- Default Linux value:
/var/run/consul
- Default Windows value:
C:\ProgramData\consul
- OS user
- Default Linux value: consul
- Default Windows value: LocalSystem
- OS group
- Default value: bin
- Inventory group name
- Override with
CONSUL_GROUP_NAME
environment variable
- Override with
- Default value: consul_instances
- Interval for reconnection attempts to LAN servers
- Default value: 30s
- Interval for reconnection attempts to WAN servers
- Default value: 30s
- Max reconnection attempts to LAN servers before failing, 0=infinit
- Default value: 0
- Max reconnection attempts to WAN servers before failing, 0=infinit
- Default value: 0
- List of LAN servers, not managed by this role, to join (ipv4 ipv6 or dns addresses)
- Default value: []
- List of WAN servers, not managed by this role, to join (ipv4 ipv6 or dns addresses)
- Default value: []
It's typically not necessary to manually alter this list.
- List of server nodes
- Default value: List of all nodes in
consul_group_name
withconsul_node_role
set to server or bootstrap
This feature makes it possible to gather the consul_advertise_address(_wan)
from
servers that are currently not targeted by the playbook.
To make this possible the delegate_facts
option is used; note that his
option has been problematic.
- Gather facts from servers that are not currently targeted
- Default value: 'no'
- Datacenter label
- Override with
CONSUL_DATACENTER
environment variable- Default value: dc1
- Override with
- Default value: dc1
- Consul domain name as defined in domain or -domain
- Override with
CONSUL_DOMAIN
environment variable
- Override with
- Default value: consul
- Consul node meta data (key-value)
- Supported in Consul version 0.7.3 or later
- Default value: {}
- Example:
consul_node_meta:
node_type: "my-custom-type"
node_meta1: "metadata1"
node_meta2: "metadata2"
- Log level as defined in log_level or -log-level
- Override with
CONSUL_LOG_LEVEL
environment variable
- Override with
- Default value: INFO
- Log to syslog as defined in enable_syslog or -syslog
- Override with
CONSUL_SYSLOG_ENABLE
environment variable
- Override with
- Default Linux value: true
- Default Windows value: false
- Consul network interface
- Override with
CONSUL_IFACE
environment variable
- Override with
- Default value:
{{ ansible_default_ipv4.interface }}
- Bind address
- Override with
CONSUL_BIND_ADDRESS
environment variable
- Override with
- Default value: default ipv4 address, or address of interface configured by
consul_iface
- Lan advertise address
- Default value:
consul_bind_address
- Wan advertise address
- Default value:
consul_bind_address
- Advanced advertise addresses settings
- Individual addresses kan be overwritten using the
consul_advertise_addresses_*
variables - Default value:
consul_advertise_addresses: serf_lan: "{{ consul_advertise_addresses_serf_lan | default(consul_advertise_address+':'+consul_ports.serf_lan) }}" serf_wan: "{{ consul_advertise_addresses_serf_wan | default(consul_advertise_address_wan+':'+consul_ports.serf_wan) }}" rpc: "{{ consul_advertise_addresses_rpc | default(consul_bind_address+':'+consul_ports.server) }}"
- Client address
- Default value: 127.0.0.1
- Advanced address settings
- Individual addresses kan be overwritten using the
consul_addresses_*
variables - Default value:
consul_addresses: dns: "{{ consul_addresses_dns | default(consul_client_address, true) }}" http: "{{ consul_addresses_http | default(consul_client_address, true) }}" https: "{{ consul_addresses_https | default(consul_client_address, true) }}" rpc: "{{ consul_addresses_rpc | default(consul_client_address, true) }}" grpc: "{{ consul_addresses_grpc | default(consul_client_address, true) }}"
- The official documentation on the Ports Used
- The ports mapping is a nested dict object that allows setting the bind ports for the following keys:
- dns - The DNS server, -1 to disable. Default 8600.
- http - The HTTP API, -1 to disable. Default 8500.
- https - The HTTPS API, -1 to disable. Default -1 (disabled).
- rpc - The CLI RPC endpoint. Default 8400. This is deprecated in Consul 0.8 and later.
- grpc - The gRPC endpoint, -1 to disable. Default -1 (disabled).
- serf_lan - The Serf LAN port. Default 8301.
- serf_wan - The Serf WAN port. Default 8302.
- server - Server RPC address. Default 8300.
For example, to enable the consul HTTPS API it is possible to set the variable as follows:
- Default values:
consul_ports:
dns: "{{ consul_ports_dns | default('8600', true) }}"
http: "{{ consul_ports_http | default('8500', true) }}"
https: "{{ consul_ports_https | default('-1', true) }}"
rpc: "{{ consul_ports_rpc | default('8400', true) }}"
serf_lan: "{{ consul_ports_serf_lan | default('8301', true) }}"
serf_wan: "{{ consul_ports_serf_wan | default('8302', true) }}"
server: "{{ consul_ports_server | default('8300', true) }}"
grpc: "{{ consul_ports_grpc | default('-1', true) }}"
Notice that the dict object has to use precisely the names stated in the
documentation! And all ports must be specified. Overwriting one or multiple
ports can be done using the consul_ports_*
variables.
- Node name (should not include dots)
- Default value:
{{ inventory_hostname_short }}
- List of upstream DNS servers
See recursors
- Override with
CONSUL_RECURSORS
environment variable
- Override with
- Default value: Empty list
- Whether to install and configure DNS API forwarding on port 53 using DNSMasq
- Override with
CONSUL_DNSMASQ_ENABLE
environment variable
- Override with
- Default value: false
- Whether to enable iptables rules for DNS forwarding to Consul
- Override with
CONSUL_IPTABLES_ENABLE
environment variable
- Override with
- Default value: false
- Add basic ACL config file
- Override with
CONSUL_ACL_POLICY
environment variable
- Override with
- Default value: false
- Enable ACLs
- Override with
CONSUL_ACL_ENABLE
environment variable
- Override with
- Default value: false
- TTL for ACL's
- Override with
CONSUL_ACL_TTL
environment variable
- Override with
- Default value: 30s
- ACL authoritative datacenter name
- Override with
CONSUL_ACL_DATACENTER
environment variable
- Override with
- Default value: dc1
- Default ACL down policy
- Override with
CONSUL_ACL_DOWN_POLICY
environment variable
- Override with
- Default value: allow
- Default ACL token, only set if provided
- Override with
CONSUL_ACL_TOKEN
environment variable
- Override with
- Default value: /
- Used for clients and servers to perform internal operations to the service catalog. See: acl_agent_token
- Override with
CONSUL_ACL_AGENT_TOKEN
environment variable
- Override with
- Default value: /
- A special access token that has agent ACL policy write privileges on each agent where it is configured
- Override with
CONSUL_ACL_AGENT_MASTER_TOKEN
environment variable
- Override with
- Default value: /
- Default ACL policy
- Override with
CONSUL_ACL_DEFAULT_POLICY
environment variable
- Override with
- Default value: allow
- ACL master token
- Override with
CONSUL_ACL_MASTER_TOKEN
environment variable
- Override with
- Default value: random uuid token
- Display generated ACL Master Token
- Override with
CONSUL_ACL_MASTER_TOKEN_DISPLAY
environment variable
- Override with
- Default value: false
- Enable ACL replication without token (makes it possible to set the token
trough the API)
- Override with
CONSUL_ACL_REPLICATION_TOKEN_ENABLE
environment variable
- Override with
- Default value: /
- ACL replication token
- Override with
CONSUL_ACL_REPLICATION_TOKEN_DISPLAY
environment variable
- Override with
- Default value: SN4K3OILSN4K3OILSN4K3OILSN4K3OIL
- Enable TLS
- Override with
CONSUL_ACL_TLS_ENABLE
environment variable
- Override with
- Default value: false
- Default source directory for TLS files
- Override with
CONSUL_ACL_TLS_ENABLE
environment variable
- Override with
- Default value:
{{ role_path }}/files
- User-specified source directory for TLS files
- Override with
CONSUL_TLS_SRC_FILES
environment variable
- Override with
- Default value:
{{ role_path }}/files
- Target directory for TLS files
- Override with
CONSUL_TLS_DIR
environment variable
- Override with
- Default value:
/etc/consul/ssl
- CA certificate filename
- Override with
CONSUL_TLS_CA_CRT
environment variable
- Override with
- Default value:
ca.crt
- Server certificate
- Override with
CONSUL_TLS_SERVER_CRT
environment variable
- Override with
- Default value:
server.crt
- Server key
- Override with
CONSUL_TLS_SERVER_KEY
environment variable
- Override with
- Default value:
server.key
- Copy from remote source if TLS files are already on host
- Default value: 'no'
- Enable Gossip Encryption
- Default value:
true
- If set, the keyring will not be persisted to a file. Any installed keys will be lost on shutdown, and only the given -encrypt key will be available on startup.
- Default value:
false
- Set the encryption key; should be the same across a cluster. If not present the key will be generated & retrieved from the bootstrapped server.
- Default value:
- Verify incoming connections
- Override with
CONSUL_TLS_VERIFY_INCOMING
environment variable
- Override with
- Default value: false
- Verify outgoing connections
- Override with
CONSUL_TLS_VERIFY_OUTGOING
environment variable
- Override with
- Default value: true
- Verify incoming connections on HTTPS endpoints (client certificates)
- Override with
CONSUL_TLS_VERIFY_INCOMING_HTTPS
environment variable
- Override with
- Default value: false
- Verify server hostname
- Override with
CONSUL_TLS_VERIFY_SERVER_HOSTNAME
environment variable
- Override with
- Default value: false
- Whether to download the files for installation directly on the remote hosts
- This is the only option on Windows as WinRM is somewhat limited in this scope
- Default value: false
- Whether to upgrade consul when a new version is specified
- The role does not handle the orchestration of a rolling update of servers followed by client nodes
- This option is not available for Windows, yet. (PR welcome)
- Default value: false
- Enable the consul ui?
- Default value: true
- Disable the consul update check?
- Default value: false
- Enable script based checks?
- Default value: false
- Raft protocol to use.
- Default value:
consul_version
<=0.7.0
: 1consul_version
>0.7.0
: 3
- The Consul role of the node, one of: bootstrap, server, or client
- Default value: client
One server should be designated as the bootstrap server, and the other servers will connect to this server. You can also specify client as the role, and Consul will be configured as a client agent instead of a server.
There are two methods to setup a cluster, the first one is to explicitly choose the bootstrap server, the other one is to let the servers elect a leader among themselves.
Here is an example of how the hosts inventory could be defined for a simple cluster of 3 servers, the first one being the designated bootstrap / leader:
[consul_instances]
consul1.consul consul_node_role=bootstrap
consul2.consul consul_node_role=server
consul3.consul consul_node_role=server
consul4.local consul_node_role=client
Or you can use the simpler method of letting them do their election process:
[consul_instances]
consul1.consul consul_node_role=server consul_bootstrap_expect=true
consul2.consul consul_node_role=server consul_bootstrap_expect=true
consul3.consul consul_node_role=server consul_bootstrap_expect=true
consul4.local consul_node_role=client
Note that this second form is the prefered one, because it is simpler.
Autopilot is a set of new features added in Consul 0.8 to allow for automatic operator-friendly management of Consul servers. It includes cleanup of dead servers, monitoring the state of the Raft cluster, and stable server introduction.
https://www.consul.io/docs/guides/autopilot.html
- Enable Autopilot config (will be written to bootsrapper node)
- Override with
CONSUL_AUTOPILOT_ENABLE
environment variable
- Override with
- Default value: false
Dead servers will periodically be cleaned up and removed from the Raft peer set, to prevent them from interfering with the quorum size and leader elections. This cleanup will also happen whenever a new server is successfully added to the cluster.
- Enable Autopilot config (will be written to bootsrapper node)
- Override with
CONSUL_AUTOPILOT_CLEANUP_DEAD_SERVERS
environment variable
- Override with
- Default value: false
Used in the serf health check to determine node health.
- Sets the threshold for time since last contact
- Override with
CONSUL_AUTOPILOT_LAST_CONTACT_THRESHOLD
environment variable
- Override with
- Default value: '200ms'
- Used in the serf health check to set a max-number of log entries nodes can trail the leader
- Override with
CONSUL_AUTOPILOT_MAX_TRAILING_LOGS
environment variable
- Override with
- Default value: 250
- Time to allow a new node to stabilize
- Override with
CONSUL_AUTOPILOT_SERVER_STABILIZATION_TIME
environment variable
- Override with
- Default value: '10s'
Consul Enterprise Only (requires that CONSUL_ENTERPRISE is set to true)
- Override with
CONSUL_AUTOPILOT_REDUNDANCY_ZONE_TAG
environment variable - Default value: 'az'
Consul Enterprise Only (requires that CONSUL_ENTERPRISE is set to true)
- Override with
CONSUL_AUTOPILOT_DISABLE_UPGRADE_MIGRATION
environment variable - Default value: false
Consul Enterprise Only (requires that CONSUL_ENTERPRISE is set to true)
- Override with
CONSUL_AUTOPILOT_UPGRADE_VERSION_TAG
environment variable - Default value: ''
As Consul loads the configuration from files and directories in lexical order,
typically merging on top of previously parsed configuration files, you may set
custom configurations via consul_config_custom
, which will be expanded into a file named config_z_custom.json
within your consul_config_path
which will
be loaded after all other configuration by default.
An example usage for enabling telemetry
:
vars:
consul_config_custom:
telemetry:
dogstatsd_addr: "localhost:8125"
dogstatsd_tags:
- "security"
- "compliance"
disable_hostname: true
The consul
binary works on most Linux platforms and is not distribution
specific. However, some distributions require installation of specific OS
packages with different package names.
- Consul package filename
- Default value:
{{ consul_version }}_linux_amd64.zip
- Consul package download URL
- Default value:
{{ consul_zip_url }}
- Consul download SHA256 summary
- Default value: SHA256 SUM
- List of OS packages to install
- Default value: list
- Consul package filename
- Default value:
{{ consul_version }}_linux_amd64.zip
- Consul package download URL
- Default value:
{{ consul_zip_url }}
- Consul download SHA256 summary
- Default value: SHA256 SUM
- List of OS packages to install
- Default value: list
- Consul package filename
- Default value:
{{ consul_version }}_linux_amd64.zip
- Consul package download URL
- Default value:
{{ consul_zip_url }}
- Consul download SHA256 summary
- Default value: SHA256 SUM
- List of OS packages to install
- Default value: list
- Integer value for systemd unit
RestartSec
option - Default value: 42
- Consul package filename
- Default value:
{{ consul_version }}_linux_amd64.zip
- Consul package download URL
- Default value:
{{ consul_zip_url }}
- Consul download SHA256 summary
- Default value: SHA256 SUM
- List of OS packages to install
- Default value: list
- Consul package filename
- Default value:
{{ consul_version }}_windows_amd64.zip
- Consul package download URL
- Default value:
{{ consul_zip_url }}
- Consul download SHA256 summary
- Default value: SHA256 SUM
- List of OS packages to install
- Default value: list
- List of Consul performance tuning items
- Default value: list
- Raft multiplier scales key Raft timing parameters
- Default value: 1
-
Node leave drain time is the dwell time for a server to honor requests while gracefully leaving
-
Default value: 5s
- RPC hold timeout is the duration that a client or server will retry internal RPC requests during leader elections
- Default value: 7s
Ansible requires GNU tar and this role performs some local use of the
unarchive module, so ensure that your system has gtar
installed and
in the PATH.
If you're on system with a different (i.e. BSD) tar
, like macOS and you
see odd errors during unarchive tasks, you could be missing gtar
.
Installing Ansible on Windows requires the PowerShell Community Extensions. These already installed on Windows Server 2012 R2 and onward. If you're attempting this role on Windows Server 2008 or earlier, you'll want to install the extensions here.
Basic installation is possible using the included site.yml
playbook:
ansible-playbook -i hosts site.yml
You can also pass variables in using the --extra-vars
option to the
ansible-playbook
command:
ansible-playbook -i hosts site.yml --extra-vars "consul_datacenter=maui"
Be aware that for clustering, the included site.yml
does the following:
- Executes consul role (installs Consul and bootstraps cluster)
- Reconfigures bootstrap node to run without bootstrap-expect setting
- Restarts bootstrap node
Basic support for ACLs is included in the role. You can set the environment
variables CONSUL_ACL_ENABLE
to true, and also set the
CONSUL_ACL_DATACENTER
environment variable to its correct value for your
environment prior to executing your playbook; for example:
CONSUL_ACL_ENABLE=true CONSUL_ACL_DATACENTER=maui \
CONSUL_ACL_MASTER_TOKEN_DISPLAY=true ansible-playbook -i uat_hosts aloha.yml
If you want the automatically generated ACL Master Token value emitted to
standard out during the play, set the environment variable
CONSUL_ACL_MASTER_TOKEN_DISPLAY
to true as in the above example.
If you want to use existing tokens, set the environment variables
CONSUL_ACL_MASTER_TOKEN
and CONSUL_ACL_REPLICATION_TOKEN
as well,
for example:
CONSUL_ACL_ENABLE=true CONSUL_ACL_DATACENTER=stjohn \
CONSUL_ACL_MASTER_TOKEN=0815C55B-3AD2-4C1B-BE9B-715CAAE3A4B2 \
CONSUL_ACL_REPLICATION_TOKEN=C609E56E-DD0B-4B99-A0AD-B079252354A0 \
CONSUL_ACL_MASTER_TOKEN_DISPLAY=true ansible-playbook -i uat_hosts sail.yml
There are a number of Ansible ACL variables you can override to further refine your initial ACL setup. They are not all currently picked up from environment variables, but do have some sensible defaults.
Check defaults/main.yml
to see how some of he defaults (i.e. tokens)
are automatically generated.
The role now includes support for DNS forwarding with Dnsmasq.
Enable like this:
ansible-playbook -i hosts site.yml --extra-vars "consul_dnsmasq_enable=true"
Then, you can query any of the agents via DNS directly via port 53, for example:
dig @consul1.consul consul3.node.consul
; <<>> DiG 9.8.3-P1 <<>> @consul1.consul consul3.node.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 29196
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;consul3.node.consul. IN A
;; ANSWER SECTION:
consul3.node.consul. 0 IN A 10.1.42.230
;; Query time: 42 msec
;; SERVER: 10.1.42.210#53(10.1.42.210)
;; WHEN: Sun Aug 7 18:06:32 2016
;;
- Address used by dnsmasq to query consul
- Default value:
consul_address.dns
- Defaults to 127.0.0.1 if consul's DNS is bound to all interfaces (eg
0.0.0.0
)
- dnsmasq cache-size
- If smaller then 0, the default dnsmasq setting will be used.
- Default value: -1
- Upstream DNS servers used by dnsmasq
- Default value: 8.8.8.8 and 8.8.4.4
- Reverse lookup subnets
- Default value: []
- Do not poll /etc/resolv.conf
- Default value: false
- Ignore /etc/resolv.conf file
- Default value: false
- Only allow requests from local subnets
- Default value: false
- Custom list of addresses to listen on.
- Default value: []
This role can also use iptables instead of Dnsmasq for forwarding DNS queries to Consul. You can enable it like this:
ansible-playbook -i hosts site.yml --extra-vars "consul_iptables_enable=true"
Note that iptables forwarding and Dnsmasq forwarding cannot be used simultaneously and the execution of the role will stop with error if such a configuration is specified.
You can enable TLS encryption by dropping a CA certificate, server
certificate, and server key into the role's files
directory.
By default these are named:
ca.crt
(can be overridden by {{ consul_tls_ca_crt }})server.crt
(can be overridden by {{ consul_tls_server_crt }})server.key
(can be overridden by {{ consul_server_key }})
Then either set the environment variable CONSUL_TLS_ENABLE=true
or use the
Ansible variable consul_tls_enable=true
at role runtime.
See examples/README_VAGRANT.md for details on quick Vagrant deployments under VirtualBox for development, evaluation, testing, etc.
BSD
Special thanks to the folks listed in CONTRIBUTORS.md for their contributions to this project.
Contributions are welcome, provided that you can agree to the terms outlined in CONTRIBUTING.md.