Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add documentation for proxy.expose configuration #7440

Merged
merged 2 commits into from
Apr 2, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions website/data/docs-navigation.js
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,7 @@ export default [
'dispatch_payload',
'env',
'ephemeral_disk',
'expose',
'group',
'job',
'lifecycle',
Expand Down
167 changes: 167 additions & 0 deletions website/pages/docs/job-specification/expose.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
---
layout: docs
page_title: expose Stanza - Job Specification
sidebar_title: expose
description: |-
The "expose" stanza allows specifying options for configuring Envoy expose
paths used in Consul Connect integration
---

# `expose` Stanza

<Placement
groups={['job', 'group', 'service', 'connect', 'sidecar_service', 'proxy', 'expose']}
/>

The `expose` stanza allows configuration of additional listeners for the default Envoy sidecar
proxy managed by Nomad for [Consul Connect](/guides/integrations/consul-connect). These
listeners create a bypass of the Connect TLS and network namespace isolation, enabling
non-Connect enabled services to make requests to specific paths through the sidecar proxy.

The `expose` configuration is valid within the context of a `proxy` stanza. Additional
information about Expose Path configurations for Envoy can be found in Consul's
[Expose Paths Configuration Reference](https://www.consul.io/docs/connect/registration/service-registration.html#expose-paths-configuration-reference).

Service [check](https://nomadproject.io/docs/job-specification/service/#check-parameters)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should add a check to the example below to help call attention to this.

configurations can use their [expose](/docs/job-specification/service#expose)
parameter to automatically generate expose path configurations for HTTP and gRPC checks.

```hcl
job "expose-example" {
datacenters = ["dc1"]
group "dashboard" {
network {
mode = "bridge"
port "expose" {
to = -1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ Should we just make it such that if mode != "host" then any not set to port will automatically be the port value? Maybe not as part of this PR but just a thought, I don't love exposing (sorry bad pun) users to the -1 special value.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ha, @schmichael and I chatted about this and figured maybe you knew the reason it was done this way =\

I'll add this to our fortnightly to reconsider

}
}

service {
name = "count-dashboard"
port = "9001"

connect {
sidecar_service {
proxy {
expose {
path {
path = "/metrics"
protocol = "http"
local_path_port = 9001
listener_port = "expose"
}
}
}
}
}

check {
name = "dashboard-health"
type = "http"
port = "expose"
path = "/health"
interval = "10s"
timeout = "3s"
expose = true
}
}
}
}
```

## `expose` Parameters

- `path` <code>([Path]: nil)</code> - A list of [Envoy Expose Path Configurations](/docs/job-specification/path)
to expose through Envoy.

### `path` Parameters

- `path` `(string: required)` - The HTTP or gRPC path to expose. The path must be prefixed
with a slash.
- `protocol` `(string: required)` - Sets the protocol of the listener. Must be
`http` or `http2`. For gRPC use `http2`.
- `local_path_port` `(int: required)` - The port the service is listening to for connections to
the configured `path`. Typically this will be the same as the `service.port` value, but
could be different if for example the exposed path is intended to resolve to another task
in the task group.
- `listener_port` <code>([Port]: required)</code> - The name of the port to use
for the exposed listener. The port should be configured to [map inside](/docs/job-specification/network#to)
the task's network namespace.


## `expose` Examples

The following example is configured to expose the `/metrics` endpoint of the Connect-enabled
`count-dashboard` service, using the `HTTP` protocol. `count-dashboard` is expected
to listen inside its namespace to port `9001`, and external services will be able to
reach its `/metrics` endpoint by connecting to the [network interface](https://nomadproject.io/docs/configuration/client/#network_interface)
of the node on the allocated `metrics` [Port](/docs/job-specification/network#port-parameters).

```hcl
service {
name = "count-dashboard"
port = "9001"

connect {
sidecar_service {
proxy {
expose {
path {
path = "/metrics"
protocol = "http"
local_path_port = 9001
listener_port = "metrics"
}
}
}
}
}
}
```

## `path` Examples

The following example is an expose configuration that exposes a `/metrics` endpoint
using the `http2` protocol (typical for gRPC), and an HTTP `/v2/health` endpoint.

```hcl
proxy {
expose {
path {
path = "/metrics"
protocol = "http2"
local_path_port = 9001
listener_port = "expose"
}
path {
path = "/v2/health"
protocol = "http"
local_path_port = 9001
listener_port = "expose"
}
}
}
```

### Exposing Service Checks

A common use case for `expose` is for exposing endpoints used in Consul service check
definitions. For these cases the [expose](/docs/job-specification/service#expose)
parameter in the service check stanza can be used to automatically generate the
expose path configuration.

```hcl
check {
type = "http"
name = "dashboard-health"
port = "expose"
path = "/health"
interval = "10s"
timeout = "3s"
expose = true
}
```

[path]: /docs/job-specification/expose#path-parameters 'Nomad Expose Path Parameters'
[port]: /docs/job-specification/network#port-parameters 'Nomad Port Parameters'
70 changes: 37 additions & 33 deletions website/pages/docs/job-specification/proxy.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,42 +19,46 @@ Connect](/docs/integrations/consul-connect). It is valid only
within the context of a `sidecar_service` stanza.

```hcl
job "countdash" {
datacenters = ["dc1"]
group "api" {
network {
mode = "bridge"
}
job "countdash" {
datacenters = ["dc1"]
group "api" {
network {
mode = "bridge"
}

service {
name = "count-api"
port = "9001"

connect {
sidecar_service {}
}
}
task "web" {
driver = "docker"
config {
image = "test/test:v1"
}
}
}
}
service {
name = "count-api"
port = "9001"

connect {
sidecar_service {
proxy {}
}
}
}
task "web" {
driver = "docker"
config {
image = "test/test:v1"
}
}
}
}
```

## `proxy` Parameters

- `local_service_address` `(string: "127.0.0.1")` - The address the local service binds to. Useful to
customize in clusters with mixed Connect and non-Connect services.
- `local_service_port` <code>(int:[port][])</code> - The port the local service binds to.
- `local_service_port` `(int:[port][])` - The port the local service binds to.
Usually the same as the parent service's port, it is useful to customize in clusters with mixed
Connect and non-Connect services
- `upstreams` <code>([upstreams][]: nil)</code> - Used to configure details of each upstream service that
this sidecar proxy communicates with.
- `config` <code>(map: nil)</code> - Proxy configuration that's opaque to Nomad and
- `expose` <code>([expose]: nil)</code> - Used to configure expose path configuration for Envoy.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- `expose` <code>([expose]: nil)</code> - Used to configure expose path configuration for Envoy.
- `expose` `([expose]: nil)` - Used to configure expose path configuration for Envoy.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without the <code> tags, the expose name isn't rendered as a link to the expose doc page like we want it to. Similar story with upstreams above. I think it has something to do with the brackets, but /shrug

See Consul's [Expose Paths Configuration Reference](https://www.consul.io/docs/connect/registration/service-registration.html#expose-paths-configuration-reference)
for more information.
- `config` `(map: nil)` - Proxy configuration that's opaque to Nomad and
passed directly to Consul. See [Consul Connect's
documentation](https://www.consul.io/docs/connect/proxies/envoy#dynamic-configuration)
for details.
Expand All @@ -64,15 +68,14 @@ within the context of a `sidecar_service` stanza.
The following example is a proxy specification that includes upstreams configuration.

```hcl
sidecar_service {
proxy {
upstreams {
destination_name = "count-api"
local_bind_port = 8080
}
}
}

sidecar_service {
proxy {
upstreams {
destination_name = "count-api"
local_bind_port = 8080
}
}
}
```

[job]: /docs/job-specification/job 'Nomad job Job Specification'
Expand All @@ -82,3 +85,4 @@ The following example is a proxy specification that includes upstreams configura
[sidecar_service]: /docs/job-specification/sidecar_service 'Nomad sidecar service Specification'
[upstreams]: /docs/job-specification/upstreams 'Nomad upstream config Specification'
[port]: /docs/job-specification/network#port-parameters 'Nomad network port configuration'
[expose]: /docs/job-specification/expose 'Nomad proxy expose configuration'
6 changes: 6 additions & 0 deletions website/pages/docs/job-specification/service.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,11 @@ scripts.
add the IP of the service and the port, so this is just the relative URL to
the health check endpoint. This is required for http-based health checks.

- `expose` `(bool: false)` - Specifies whether an [Expose Path](/docs/job-specification/expose#path-parameters)
should be automatically generated for this check. Only compatible with
Connect-enabled task-group services using the default Connect proxy. Check
must be of [`type`][type] `http` or `grpc`.

- `port` `(string: <varies>)` - Specifies the label of the port on which the
check will be performed. Note this is the _label_ of the port and not the port
number unless `address_mode = driver`. The port label must match one defined
Expand Down Expand Up @@ -684,6 +689,7 @@ advertise and check directly since Nomad isn't managing any port assignments.
[qemu]: /docs/drivers/qemu 'Nomad qemu Driver'
[restart_stanza]: /docs/job-specification/restart 'restart stanza'
[connect]: /docs/job-specification/connect 'Nomad Consul Connect Integration'
[type]: /docs/job-specification/service#type
[shutdowndelay]: /docs/job-specification/task#shutdown_delay
[killsignal]: /docs/job-specification/task#kill_signal
[killtimeout]: /docs/job-specification/task#kill_timeout