Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Service registration for IPv6 docker addresses (Fixes #3785) #3790

Merged
merged 7 commits into from
Feb 6, 2018
Merged

Service registration for IPv6 docker addresses (Fixes #3785) #3790

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
795d84e
Service registration for IPv6 docker addresses
42wim Jan 24, 2018
dea460b
* Change use_ipv6_address to advertise_ipv6_address.
42wim Jan 31, 2018
cdd55e4
Update documentation
42wim Feb 1, 2018
d0dd6ef
Add IPv6 support to travis docker
42wim Feb 1, 2018
51bf06f
Add AdvertiseIPv6Address test
42wim Feb 1, 2018
e486a27
docker: Skip IPv6 test if IPv6 disabled
schmichael Feb 6, 2018
5ff86f0
Add changelog entry for #3790
schmichael Feb 6, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 52 additions & 44 deletions client/driver/docker.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,50 +173,51 @@ type DockerVolumeDriverConfig struct {

// DockerDriverConfig defines the user specified config block in a jobspec
type DockerDriverConfig struct {
ImageName string `mapstructure:"image"` // Container's Image Name
LoadImage string `mapstructure:"load"` // LoadImage is a path to an image archive file
Command string `mapstructure:"command"` // The Command to run when the container starts up
Args []string `mapstructure:"args"` // The arguments to the Command
Entrypoint []string `mapstructure:"entrypoint"` // Override the containers entrypoint
IpcMode string `mapstructure:"ipc_mode"` // The IPC mode of the container - host and none
NetworkMode string `mapstructure:"network_mode"` // The network mode of the container - host, nat and none
NetworkAliases []string `mapstructure:"network_aliases"` // The network-scoped alias for the container
IPv4Address string `mapstructure:"ipv4_address"` // The container ipv4 address
IPv6Address string `mapstructure:"ipv6_address"` // the container ipv6 address
PidMode string `mapstructure:"pid_mode"` // The PID mode of the container - host and none
UTSMode string `mapstructure:"uts_mode"` // The UTS mode of the container - host and none
UsernsMode string `mapstructure:"userns_mode"` // The User namespace mode of the container - host and none
PortMapRaw []map[string]string `mapstructure:"port_map"` //
PortMap map[string]int `mapstructure:"-"` // A map of host port labels and the ports exposed on the container
Privileged bool `mapstructure:"privileged"` // Flag to run the container in privileged mode
SysctlRaw []map[string]string `mapstructure:"sysctl"` //
Sysctl map[string]string `mapstructure:"-"` // The sysctl custom configurations
UlimitRaw []map[string]string `mapstructure:"ulimit"` //
Ulimit []docker.ULimit `mapstructure:"-"` // The ulimit custom configurations
DNSServers []string `mapstructure:"dns_servers"` // DNS Server for containers
DNSSearchDomains []string `mapstructure:"dns_search_domains"` // DNS Search domains for containers
DNSOptions []string `mapstructure:"dns_options"` // DNS Options
ExtraHosts []string `mapstructure:"extra_hosts"` // Add host to /etc/hosts (host:IP)
Hostname string `mapstructure:"hostname"` // Hostname for containers
LabelsRaw []map[string]string `mapstructure:"labels"` //
Labels map[string]string `mapstructure:"-"` // Labels to set when the container starts up
Auth []DockerDriverAuth `mapstructure:"auth"` // Authentication credentials for a private Docker registry
AuthSoftFail bool `mapstructure:"auth_soft_fail"` // Soft-fail if auth creds are provided but fail
TTY bool `mapstructure:"tty"` // Allocate a Pseudo-TTY
Interactive bool `mapstructure:"interactive"` // Keep STDIN open even if not attached
ShmSize int64 `mapstructure:"shm_size"` // Size of /dev/shm of the container in bytes
WorkDir string `mapstructure:"work_dir"` // Working directory inside the container
Logging []DockerLoggingOpts `mapstructure:"logging"` // Logging options for syslog server
Volumes []string `mapstructure:"volumes"` // Host-Volumes to mount in, syntax: /path/to/host/directory:/destination/path/in/container
Mounts []DockerMount `mapstructure:"mounts"` // Docker volumes to mount
VolumeDriver string `mapstructure:"volume_driver"` // Docker volume driver used for the container's volumes
ForcePull bool `mapstructure:"force_pull"` // Always force pull before running image, useful if your tags are mutable
MacAddress string `mapstructure:"mac_address"` // Pin mac address to container
SecurityOpt []string `mapstructure:"security_opt"` // Flags to pass directly to security-opt
Devices []DockerDevice `mapstructure:"devices"` // To allow mounting USB or other serial control devices
CapAdd []string `mapstructure:"cap_add"` // Flags to pass directly to cap-add
CapDrop []string `mapstructure:"cap_drop"` // Flags to pass directly to cap-drop
ReadonlyRootfs bool `mapstructure:"readonly_rootfs"` // Mount the container’s root filesystem as read only
ImageName string `mapstructure:"image"` // Container's Image Name
LoadImage string `mapstructure:"load"` // LoadImage is a path to an image archive file
Command string `mapstructure:"command"` // The Command to run when the container starts up
Args []string `mapstructure:"args"` // The arguments to the Command
Entrypoint []string `mapstructure:"entrypoint"` // Override the containers entrypoint
IpcMode string `mapstructure:"ipc_mode"` // The IPC mode of the container - host and none
NetworkMode string `mapstructure:"network_mode"` // The network mode of the container - host, nat and none
NetworkAliases []string `mapstructure:"network_aliases"` // The network-scoped alias for the container
IPv4Address string `mapstructure:"ipv4_address"` // The container ipv4 address
IPv6Address string `mapstructure:"ipv6_address"` // the container ipv6 address
PidMode string `mapstructure:"pid_mode"` // The PID mode of the container - host and none
UTSMode string `mapstructure:"uts_mode"` // The UTS mode of the container - host and none
UsernsMode string `mapstructure:"userns_mode"` // The User namespace mode of the container - host and none
PortMapRaw []map[string]string `mapstructure:"port_map"` //
PortMap map[string]int `mapstructure:"-"` // A map of host port labels and the ports exposed on the container
Privileged bool `mapstructure:"privileged"` // Flag to run the container in privileged mode
SysctlRaw []map[string]string `mapstructure:"sysctl"` //
Sysctl map[string]string `mapstructure:"-"` // The sysctl custom configurations
UlimitRaw []map[string]string `mapstructure:"ulimit"` //
Ulimit []docker.ULimit `mapstructure:"-"` // The ulimit custom configurations
DNSServers []string `mapstructure:"dns_servers"` // DNS Server for containers
DNSSearchDomains []string `mapstructure:"dns_search_domains"` // DNS Search domains for containers
DNSOptions []string `mapstructure:"dns_options"` // DNS Options
ExtraHosts []string `mapstructure:"extra_hosts"` // Add host to /etc/hosts (host:IP)
Hostname string `mapstructure:"hostname"` // Hostname for containers
LabelsRaw []map[string]string `mapstructure:"labels"` //
Labels map[string]string `mapstructure:"-"` // Labels to set when the container starts up
Auth []DockerDriverAuth `mapstructure:"auth"` // Authentication credentials for a private Docker registry
AuthSoftFail bool `mapstructure:"auth_soft_fail"` // Soft-fail if auth creds are provided but fail
TTY bool `mapstructure:"tty"` // Allocate a Pseudo-TTY
Interactive bool `mapstructure:"interactive"` // Keep STDIN open even if not attached
ShmSize int64 `mapstructure:"shm_size"` // Size of /dev/shm of the container in bytes
WorkDir string `mapstructure:"work_dir"` // Working directory inside the container
Logging []DockerLoggingOpts `mapstructure:"logging"` // Logging options for syslog server
Volumes []string `mapstructure:"volumes"` // Host-Volumes to mount in, syntax: /path/to/host/directory:/destination/path/in/container
Mounts []DockerMount `mapstructure:"mounts"` // Docker volumes to mount
VolumeDriver string `mapstructure:"volume_driver"` // Docker volume driver used for the container's volumes
ForcePull bool `mapstructure:"force_pull"` // Always force pull before running image, useful if your tags are mutable
MacAddress string `mapstructure:"mac_address"` // Pin mac address to container
SecurityOpt []string `mapstructure:"security_opt"` // Flags to pass directly to security-opt
Devices []DockerDevice `mapstructure:"devices"` // To allow mounting USB or other serial control devices
CapAdd []string `mapstructure:"cap_add"` // Flags to pass directly to cap-add
CapDrop []string `mapstructure:"cap_drop"` // Flags to pass directly to cap-drop
ReadonlyRootfs bool `mapstructure:"readonly_rootfs"` // Mount the container’s root filesystem as read only
AdvertiseIPv6Address bool `mapstructure:"advertise_ipv6_address"` // Flag to use the GlobalIPv6Address from the container as the detected IP
}

func sliceMergeUlimit(ulimitsRaw map[string]string) ([]docker.ULimit, error) {
Expand Down Expand Up @@ -674,6 +675,9 @@ func (d *DockerDriver) Validate(config map[string]interface{}) error {
"readonly_rootfs": {
Type: fields.TypeBool,
},
"advertise_ipv6_address": {
Type: fields.TypeBool,
},
},
}

Expand Down Expand Up @@ -884,6 +888,10 @@ func (d *DockerDriver) detectIP(c *docker.Container) (string, bool) {
}

ip = net.IPAddress
if d.driverConfig.AdvertiseIPv6Address {
ip = net.GlobalIPv6Address
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, it seems like we probably want to set auto = true here as well. While this complicates the already unfortunately confusing advertise = "auto" logic, I think it will do what users expect by automatically advertising the routable address for the container when it's possible to discover it.

Perhaps a better question is: It should be exceedingly rare that someone wants use_ipv6_address = true and advertise = "host", correct?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(this has been addressed; leaving for historical purposes)

auto = true
}
ipName = name

// Don't auto-advertise IPs for default networks (bridge on
Expand Down
3 changes: 3 additions & 0 deletions website/source/docs/drivers/docker.html.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -355,6 +355,9 @@ The `docker` driver supports the following configuration in the job spec. Only
]
}
```
* `advertise_ipv6_address` - (Optional) `true` or `false` (default). Use the container's
IPv6 address (GlobalIPv6Address in Docker) when registering services and checks.
See [IPv6 Docker containers](/docs/job-specification/service.html#IPv6 Docker containers) for details.

* `readonly_rootfs` - (Optional) `true` or `false` (default). Mount
the container's filesystem as read only.
Expand Down
103 changes: 103 additions & 0 deletions website/source/docs/job-specification/service.html.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -463,6 +463,109 @@ In this case Nomad doesn't need to assign Redis any host ports. The `service`
and `check` stanzas can both specify the port number to advertise and check
directly since Nomad isn't managing any port assignments.

### IPv6 Docker containers

The [Docker](/docs/drivers/docker.html#advertise_ipv6_address) driver supports the
`advertise_ipv6_address` parameter in it's configuration.

For the `service`stanza is no explicit `address_mode` required.
Services default to the `auto` address mode.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Services will automatically advertise the IPv6 address when advertise_ipv6_address is used.


Unlike services, checks do not have an `auto` address mode as there's no way
for Nomad to know which is the best address to use for checks. Consul needs
access to the address for any HTTP or TCP checks.

So you have to set `address_mode` parameter in the `check` stanza to `driver`.

For example using `auto` address mode:

```hcl
job "example" {
datacenters = ["dc1"]
group "cache" {

task "redis" {
driver = "docker"

config {
image = "redis:3.2"
advertise_ipv6_address = true
port_map {
db = 6379
}
}

resources {
cpu = 500 # 500 MHz
memory = 256 # 256MB
network {
mbits = 10
port "db" {}
}
}

service {
name = "ipv6-redis"
port = db
check {
name = "ipv6-redis-check"
type = "tcp"
interval = "10s"
timeout = "2s"
port = db
address_mode = "driver"
}
}
}
}
}
```

Or using `address_mode=driver` for `service` and `check` with numeric ports:

```hcl
job "example" {
datacenters = ["dc1"]
group "cache" {

task "redis" {
driver = "docker"

config {
image = "redis:3.2"
advertise_ipv6_address = true
# No port map required!
}

resources {
cpu = 500 # 500 MHz
memory = 256 # 256MB
network {
mbits = 10
}
}

service {
name = "ipv6-redis"
port = 6379
address_mode = "driver"
check {
name = "ipv6-redis-check"
type = "tcp"
interval = "10s"
timeout = "2s"
port = 6379
address_mode = "driver"
}
}
}
}
}
```

The `service` and `check` stanzas can both specify the port number to
advertise and check directly since Nomad isn't managing any port assignments.


- - -

Expand Down