Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Service registration for IPv6 docker addresses (Fixes #3785) #3790

Merged
merged 7 commits into from
Feb 6, 2018
+245 −44
Merged

Service registration for IPv6 docker addresses (Fixes #3785) #3790

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
795d84e
Service registration for IPv6 docker addresses
42wim Jan 24, 2018
dea460b
* Change use_ipv6_address to advertise_ipv6_address.
42wim Jan 31, 2018
cdd55e4
Update documentation
42wim Feb 1, 2018
d0dd6ef
Add IPv6 support to travis docker
42wim Feb 1, 2018
51bf06f
Add AdvertiseIPv6Address test
42wim Feb 1, 2018
e486a27
docker: Skip IPv6 test if IPv6 disabled
schmichael Feb 6, 2018
5ff86f0
Add changelog entry for #3790
schmichael Feb 6, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 48 additions & 47 deletions client/driver/docker.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,51 +173,51 @@ type DockerVolumeDriverConfig struct {

// DockerDriverConfig defines the user specified config block in a jobspec
type DockerDriverConfig struct {
ImageName string `mapstructure:"image"` // Container's Image Name
LoadImage string `mapstructure:"load"` // LoadImage is a path to an image archive file
Command string `mapstructure:"command"` // The Command to run when the container starts up
Args []string `mapstructure:"args"` // The arguments to the Command
Entrypoint []string `mapstructure:"entrypoint"` // Override the containers entrypoint
IpcMode string `mapstructure:"ipc_mode"` // The IPC mode of the container - host and none
NetworkMode string `mapstructure:"network_mode"` // The network mode of the container - host, nat and none
NetworkAliases []string `mapstructure:"network_aliases"` // The network-scoped alias for the container
IPv4Address string `mapstructure:"ipv4_address"` // The container ipv4 address
IPv6Address string `mapstructure:"ipv6_address"` // the container ipv6 address
PidMode string `mapstructure:"pid_mode"` // The PID mode of the container - host and none
UTSMode string `mapstructure:"uts_mode"` // The UTS mode of the container - host and none
UsernsMode string `mapstructure:"userns_mode"` // The User namespace mode of the container - host and none
PortMapRaw []map[string]string `mapstructure:"port_map"` //
PortMap map[string]int `mapstructure:"-"` // A map of host port labels and the ports exposed on the container
Privileged bool `mapstructure:"privileged"` // Flag to run the container in privileged mode
SysctlRaw []map[string]string `mapstructure:"sysctl"` //
Sysctl map[string]string `mapstructure:"-"` // The sysctl custom configurations
UlimitRaw []map[string]string `mapstructure:"ulimit"` //
Ulimit []docker.ULimit `mapstructure:"-"` // The ulimit custom configurations
DNSServers []string `mapstructure:"dns_servers"` // DNS Server for containers
DNSSearchDomains []string `mapstructure:"dns_search_domains"` // DNS Search domains for containers
DNSOptions []string `mapstructure:"dns_options"` // DNS Options
ExtraHosts []string `mapstructure:"extra_hosts"` // Add host to /etc/hosts (host:IP)
Hostname string `mapstructure:"hostname"` // Hostname for containers
LabelsRaw []map[string]string `mapstructure:"labels"` //
Labels map[string]string `mapstructure:"-"` // Labels to set when the container starts up
Auth []DockerDriverAuth `mapstructure:"auth"` // Authentication credentials for a private Docker registry
AuthSoftFail bool `mapstructure:"auth_soft_fail"` // Soft-fail if auth creds are provided but fail
TTY bool `mapstructure:"tty"` // Allocate a Pseudo-TTY
Interactive bool `mapstructure:"interactive"` // Keep STDIN open even if not attached
ShmSize int64 `mapstructure:"shm_size"` // Size of /dev/shm of the container in bytes
WorkDir string `mapstructure:"work_dir"` // Working directory inside the container
Logging []DockerLoggingOpts `mapstructure:"logging"` // Logging options for syslog server
Volumes []string `mapstructure:"volumes"` // Host-Volumes to mount in, syntax: /path/to/host/directory:/destination/path/in/container
Mounts []DockerMount `mapstructure:"mounts"` // Docker volumes to mount
VolumeDriver string `mapstructure:"volume_driver"` // Docker volume driver used for the container's volumes
ForcePull bool `mapstructure:"force_pull"` // Always force pull before running image, useful if your tags are mutable
MacAddress string `mapstructure:"mac_address"` // Pin mac address to container
SecurityOpt []string `mapstructure:"security_opt"` // Flags to pass directly to security-opt
Devices []DockerDevice `mapstructure:"devices"` // To allow mounting USB or other serial control devices
CapAdd []string `mapstructure:"cap_add"` // Flags to pass directly to cap-add
CapDrop []string `mapstructure:"cap_drop"` // Flags to pass directly to cap-drop
ReadonlyRootfs bool `mapstructure:"readonly_rootfs"` // Mount the container’s root filesystem as read only
UseIPv6Address bool `mapstructure:"use_ipv6_address"` // Flag to use the GlobalIPv6Address from the container as the detected IP
ImageName string `mapstructure:"image"` // Container's Image Name
LoadImage string `mapstructure:"load"` // LoadImage is a path to an image archive file
Command string `mapstructure:"command"` // The Command to run when the container starts up
Args []string `mapstructure:"args"` // The arguments to the Command
Entrypoint []string `mapstructure:"entrypoint"` // Override the containers entrypoint
IpcMode string `mapstructure:"ipc_mode"` // The IPC mode of the container - host and none
NetworkMode string `mapstructure:"network_mode"` // The network mode of the container - host, nat and none
NetworkAliases []string `mapstructure:"network_aliases"` // The network-scoped alias for the container
IPv4Address string `mapstructure:"ipv4_address"` // The container ipv4 address
IPv6Address string `mapstructure:"ipv6_address"` // the container ipv6 address
PidMode string `mapstructure:"pid_mode"` // The PID mode of the container - host and none
UTSMode string `mapstructure:"uts_mode"` // The UTS mode of the container - host and none
UsernsMode string `mapstructure:"userns_mode"` // The User namespace mode of the container - host and none
PortMapRaw []map[string]string `mapstructure:"port_map"` //
PortMap map[string]int `mapstructure:"-"` // A map of host port labels and the ports exposed on the container
Privileged bool `mapstructure:"privileged"` // Flag to run the container in privileged mode
SysctlRaw []map[string]string `mapstructure:"sysctl"` //
Sysctl map[string]string `mapstructure:"-"` // The sysctl custom configurations
UlimitRaw []map[string]string `mapstructure:"ulimit"` //
Ulimit []docker.ULimit `mapstructure:"-"` // The ulimit custom configurations
DNSServers []string `mapstructure:"dns_servers"` // DNS Server for containers
DNSSearchDomains []string `mapstructure:"dns_search_domains"` // DNS Search domains for containers
DNSOptions []string `mapstructure:"dns_options"` // DNS Options
ExtraHosts []string `mapstructure:"extra_hosts"` // Add host to /etc/hosts (host:IP)
Hostname string `mapstructure:"hostname"` // Hostname for containers
LabelsRaw []map[string]string `mapstructure:"labels"` //
Labels map[string]string `mapstructure:"-"` // Labels to set when the container starts up
Auth []DockerDriverAuth `mapstructure:"auth"` // Authentication credentials for a private Docker registry
AuthSoftFail bool `mapstructure:"auth_soft_fail"` // Soft-fail if auth creds are provided but fail
TTY bool `mapstructure:"tty"` // Allocate a Pseudo-TTY
Interactive bool `mapstructure:"interactive"` // Keep STDIN open even if not attached
ShmSize int64 `mapstructure:"shm_size"` // Size of /dev/shm of the container in bytes
WorkDir string `mapstructure:"work_dir"` // Working directory inside the container
Logging []DockerLoggingOpts `mapstructure:"logging"` // Logging options for syslog server
Volumes []string `mapstructure:"volumes"` // Host-Volumes to mount in, syntax: /path/to/host/directory:/destination/path/in/container
Mounts []DockerMount `mapstructure:"mounts"` // Docker volumes to mount
VolumeDriver string `mapstructure:"volume_driver"` // Docker volume driver used for the container's volumes
ForcePull bool `mapstructure:"force_pull"` // Always force pull before running image, useful if your tags are mutable
MacAddress string `mapstructure:"mac_address"` // Pin mac address to container
SecurityOpt []string `mapstructure:"security_opt"` // Flags to pass directly to security-opt
Devices []DockerDevice `mapstructure:"devices"` // To allow mounting USB or other serial control devices
CapAdd []string `mapstructure:"cap_add"` // Flags to pass directly to cap-add
CapDrop []string `mapstructure:"cap_drop"` // Flags to pass directly to cap-drop
ReadonlyRootfs bool `mapstructure:"readonly_rootfs"` // Mount the container’s root filesystem as read only
AdvertiseIPv6Address bool `mapstructure:"advertise_ipv6_address"` // Flag to use the GlobalIPv6Address from the container as the detected IP
}

func sliceMergeUlimit(ulimitsRaw map[string]string) ([]docker.ULimit, error) {
Expand Down Expand Up @@ -675,7 +675,7 @@ func (d *DockerDriver) Validate(config map[string]interface{}) error {
"readonly_rootfs": {
Type: fields.TypeBool,
},
"use_ipv6_address": {
"advertise_ipv6_address": {
Type: fields.TypeBool,
},
},
Expand Down Expand Up @@ -888,8 +888,9 @@ func (d *DockerDriver) detectIP(c *docker.Container) (string, bool) {
}

ip = net.IPAddress
if d.driverConfig.UseIPv6Address {
if d.driverConfig.AdvertiseIPv6Address {
ip = net.GlobalIPv6Address
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, it seems like we probably want to set auto = true here as well. While this complicates the already unfortunately confusing advertise = "auto" logic, I think it will do what users expect by automatically advertising the routable address for the container when it's possible to discover it.

Perhaps a better question is: It should be exceedingly rare that someone wants use_ipv6_address = true and advertise = "host", correct?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(this has been addressed; leaving for historical purposes)

auto = true
}
ipName = name

Expand Down
8 changes: 3 additions & 5 deletions website/source/docs/drivers/docker.html.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -355,11 +355,9 @@ The `docker` driver supports the following configuration in the job spec. Only
]
}
```
* `use_ipv6_address` - (Optional) `true` or `false` (default). Use IPv6 Address
will use the containers IPv6 address (GlobalIPv6Address) when registering service checks and using
`address_mode = driver`.
See [service](/docs/job-specification/service.html) for details.

* `advertise_ipv6_address` - (Optional) `true` or `false` (default). Use the container's
IPv6 address (GlobalIPv6Address in Docker) when registering services and checks.
See [IPv6 Docker containers](/docs/job-specification/service.html#IPv6 Docker containers) for details.

* `readonly_rootfs` - (Optional) `true` or `false` (default). Mount
the container's filesystem as read only.
Expand Down
83 changes: 58 additions & 25 deletions website/source/docs/job-specification/service.html.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,9 +104,6 @@ does not automatically enable service discovery.
`address_mode="driver"`. Numeric ports may be used when in driver addressing
mode.

Docker and IPv6 containers: This setting is required if you want to register
the port of the (IPv6) service. See [below for examples.](#IPv6 docker containers)

- `tags` `(array<string>: [])` - Specifies the list of tags to associate with
this service. If this is not supplied, no tags will be assigned to the service
when it is registered.
Expand All @@ -127,10 +124,6 @@ does not automatically enable service discovery.
addresses. Task will fail if driver network cannot be determined. Only
implemented for Docker and rkt.

Docker and IPv6 containers: If you want to register the IPv6 address
of the container you'll have to enable this and specify `use_ipv6_address`
in the docker driver configuration. See [below for examples.](#IPv6 docker containers)

- `host` - Use the host IP and port.

### `check` Parameters
Expand All @@ -147,10 +140,6 @@ scripts.
[below for details.](#using-driver-address-mode) Unlike `port`, this setting
is *not* inherited from the `service`.

Docker and IPv6 containers: If you want to check the IPv6 address
of the container you'll have to enable this and specify `use_ipv6_address`
in the docker driver configuration. See [below for examples.](#IPv6 docker containers)

- `args` `(array<string>: [])` - Specifies additional arguments to the
`command`. This only applies to script-based health checks.

Expand Down Expand Up @@ -197,9 +186,6 @@ scripts.
default. In Nomad 0.7.1 or later numeric ports may be used if
`address_mode="driver"` is set on the check.

Docker and IPv6 containers: Using a numeric port is required if you want to
check the port of (IPv6) service. See [below for examples.](#IPv6 docker containers)

- `protocol` `(string: "http")` - Specifies the protocol for the http-based
health checks. Valid options are `http` and `https`.

Expand Down Expand Up @@ -477,17 +463,21 @@ In this case Nomad doesn't need to assign Redis any host ports. The `service`
and `check` stanzas can both specify the port number to advertise and check
directly since Nomad isn't managing any port assignments.

### IPv6 docker containers
### IPv6 Docker containers

The [Docker](/docs/drivers/docker.html#advertise_ipv6_address) driver supports the
`advertise_ipv6_address` parameter in it's configuration.

The [Docker](/docs/drivers/docker.html#use_ipv6_address) driver support the
`use_ipv6_address` parameter in it's configuration.
For the `service`stanza is no explicit `address_mode` required.
Services default to the `auto` address mode.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Services will automatically advertise the IPv6 address when advertise_ipv6_address is used.


Besides enabling this parameter you have to set `address_mode` parameter in
both `service` and `check` stanzas to `driver`.
Unlike services, checks do not have an `auto` address mode as there's no way
for Nomad to know which is the best address to use for checks. Consul needs
access to the address for any HTTP or TCP checks.

You also have explicily specify the `port` that will be registered and checked.
So you have to set `address_mode` parameter in the `check` stanza to `driver`.

For example
For example using `auto` address mode:

```hcl
job "example" {
Expand All @@ -499,7 +489,51 @@ job "example" {

config {
image = "redis:3.2"
use_ipv6_address = true
advertise_ipv6_address = true
port_map {
db = 6379
}
}

resources {
cpu = 500 # 500 MHz
memory = 256 # 256MB
network {
mbits = 10
port "db" {}
}
}

service {
name = "ipv6-redis"
port = db
check {
name = "ipv6-redis-check"
type = "tcp"
interval = "10s"
timeout = "2s"
port = db
address_mode = "driver"
}
}
}
}
}
```

Or using `address_mode=driver` for `service` and `check` with numeric ports:

```hcl
job "example" {
datacenters = ["dc1"]
group "cache" {

task "redis" {
driver = "docker"

config {
image = "redis:3.2"
advertise_ipv6_address = true
# No port map required!
}

Expand Down Expand Up @@ -529,9 +563,8 @@ job "example" {
}
```

With IPv6 Nomad doesn't need to assign Redis any host ports. The `service`
and `check` stanzas can both specify the port number to advertise and check
directly since Nomad isn't managing any port assignments.
The `service` and `check` stanzas can both specify the port number to
advertise and check directly since Nomad isn't managing any port assignments.


- - -
Expand Down