From 44f884eb5411647036ce0f330c147d413df06dec Mon Sep 17 00:00:00 2001 From: Seth Hoenig Date: Fri, 20 Mar 2020 12:22:14 -0600 Subject: [PATCH 1/2] docs: add documentation for proxy.expose configuration This changeset adds documentation changes for the new `proxy.expose` stanza as well as the `check.expose` parameter. The examples are centered around proposed changes for the "countdash" `dashboard-service` in [demo-consul-101](github.com/hashicorp/demo-consul-101/pull/6). The dashboard service will now serve two additonal endpoints - `/health/api` - `/metrics` which should serve nicely as expose-able paths. --- website/data/docs-navigation.js | 1 + .../pages/docs/job-specification/expose.mdx | 145 ++++++++++++++++++ .../pages/docs/job-specification/proxy.mdx | 70 +++++---- .../pages/docs/job-specification/service.mdx | 6 + 4 files changed, 189 insertions(+), 33 deletions(-) create mode 100644 website/pages/docs/job-specification/expose.mdx diff --git a/website/data/docs-navigation.js b/website/data/docs-navigation.js index efc3e5dfcb5..8173e8c5ae4 100644 --- a/website/data/docs-navigation.js +++ b/website/data/docs-navigation.js @@ -156,6 +156,7 @@ export default [ 'dispatch_payload', 'env', 'ephemeral_disk', + 'expose', 'group', 'job', 'lifecycle', diff --git a/website/pages/docs/job-specification/expose.mdx b/website/pages/docs/job-specification/expose.mdx new file mode 100644 index 00000000000..4212d8f931c --- /dev/null +++ b/website/pages/docs/job-specification/expose.mdx @@ -0,0 +1,145 @@ +--- +layout: docs +page_title: expose Stanza - Job Specification +sidebar_title: expose +description: |- + The "expose" stanza allows specifying options for configuring Envoy expose + paths used in Consul Connect integration +--- + +# `expose` Stanza + + + +The `expose` stanza allows configuration of additional listeners for the default Envoy sidecar +proxy managed by Nomad for [Consul Connect](/guides/integrations/consul-connect). These +listeners create a bypass of the Connect TLS and network namespace isolation, enabling +non-Connect enabled services to make requests to specific paths through the sidecar proxy. + +The `expose` configuration is valid within the context of a `proxy` stanza. Additional +information about Expose Path configurations for Envoy can be found in Consul's +[Expose Paths Configuration Reference](https://www.consul.io/docs/connect/registration/service-registration.html#expose-paths-configuration-reference). + +Service [check](https://nomadproject.io/docs/job-specification/service/#check-parameters) +configurations can use their [expose](/docs/job-specification/service#expose) +parameter to automatically generate expose path configurations for HTTP and gRPC checks. + +```hcl +job "expose-example" { + datacenters = ["dc1"] + group "dashboard" { + network { + mode = "bridge" + port "expose" { + to = -1 + } + } + + service { + name = "count-dashboard" + port = "9001" + + connect { + sidecar_service { + proxy { + expose { + path { + path = "/metrics" + protocol = "http" + local_path_port = 9001 + listener_port = "expose" + } + } + } + } + } + } + } +} +``` + +## `expose` Parameters + +- `path` ([Path]: nil) - A list of [Envoy Expose Path Configurations](/docs/job-specification/path) + to expose through Envoy. + +### `path` Parameters + +- `path` `(string: required)` - The HTTP or gRPC path to expose. The path must be prefixed + with a slash. +- `protocol` `(string: required)` - Sets the protocol of the listener. Must be + `http` or `http2`. For gRPC use `http2`. +- `local_path_port` `(int: required)` - The port the service is listening to for connections to + the configured `path`. Typically this will be the same as the `service.port` value, but + could be different if for example the exposed path is intended to resolve to another task + in the task group. +- `listener_port` ([Port]: required) - The name of the port to use + for the exposed listener. The port should be configured to [map inside](/docs/job-specification/network#to) + the task's network namespace. + + +## `expose` Examples + +The following example is configured to expose the `/metrics` endpoint of the Connect-enabled +`count-dashboard` service, using the `HTTP` protocol. `count-dashboard` is expected +to listen inside its namespace to port `9001`, and external services will be able to +reach its `/metrics` endpoint by connecting to the [network interface](https://nomadproject.io/docs/configuration/client/#network_interface) +of the node on the allocated `metrics` [Port](/docs/job-specification/network#port-parameters). + +```hcl +service { + name = "count-dashboard" + port = "9001" + + connect { + sidecar_service { + proxy { + expose { + path { + path = "/metrics" + protocol = "http" + local_path_port = 9001 + listener_port = "metrics" + } + } + } + } + } +} +``` + +## `path` Examples + +The following example is an expose configuration that exposes a `/metrics` endpoint +using the `http2` protocol (typical for gRPC), and an HTTP `/v2/health` endpoint. + +```hcl +proxy { + expose { + path { + path = "/metrics" + protocol = "http2" + local_path_port = 9001 + listener_port = "expose" + } + path { + path = "/v2/health" + protocol = "http" + local_path_port = 9001 + listener_port = "expose" + } + } +} +``` + +### Exposing Service Checks + +A common use case for `expose` is for exposing endpoints used in Consul service check +definitions. For these cases the [expose](/docs/job-specification/service#expose) +parameter in the service check stanza can be used to automatically generate the +expose path configuration. + +[path]: /docs/job-specification/expose#path-parameters 'Nomad Expose Path Parameters' +[port]: /docs/job-specification/network#port-parameters 'Nomad Port Parameters' diff --git a/website/pages/docs/job-specification/proxy.mdx b/website/pages/docs/job-specification/proxy.mdx index 73215802f35..2e7a946579f 100644 --- a/website/pages/docs/job-specification/proxy.mdx +++ b/website/pages/docs/job-specification/proxy.mdx @@ -19,42 +19,46 @@ Connect](/docs/integrations/consul-connect). It is valid only within the context of a `sidecar_service` stanza. ```hcl - job "countdash" { - datacenters = ["dc1"] - group "api" { - network { - mode = "bridge" - } +job "countdash" { + datacenters = ["dc1"] + group "api" { + network { + mode = "bridge" + } - service { - name = "count-api" - port = "9001" - - connect { - sidecar_service {} - } - } - task "web" { - driver = "docker" - config { - image = "test/test:v1" - } - } - } - } + service { + name = "count-api" + port = "9001" + connect { + sidecar_service { + proxy {} + } + } + } + task "web" { + driver = "docker" + config { + image = "test/test:v1" + } + } + } +} ``` ## `proxy` Parameters - `local_service_address` `(string: "127.0.0.1")` - The address the local service binds to. Useful to customize in clusters with mixed Connect and non-Connect services. -- `local_service_port` (int:[port][]) - The port the local service binds to. +- `local_service_port` `(int:[port][])` - The port the local service binds to. Usually the same as the parent service's port, it is useful to customize in clusters with mixed Connect and non-Connect services - `upstreams` ([upstreams][]: nil) - Used to configure details of each upstream service that this sidecar proxy communicates with. -- `config` (map: nil) - Proxy configuration that's opaque to Nomad and +- `expose` ([expose]: nil) - Used to configure expose path configuration for Envoy. + See Consul's [Expose Paths Configuration Reference](https://www.consul.io/docs/connect/registration/service-registration.html#expose-paths-configuration-reference) + for more information. +- `config` `(map: nil)` - Proxy configuration that's opaque to Nomad and passed directly to Consul. See [Consul Connect's documentation](https://www.consul.io/docs/connect/proxies/envoy#dynamic-configuration) for details. @@ -64,15 +68,14 @@ within the context of a `sidecar_service` stanza. The following example is a proxy specification that includes upstreams configuration. ```hcl - sidecar_service { - proxy { - upstreams { - destination_name = "count-api" - local_bind_port = 8080 - } - } - } - +sidecar_service { + proxy { + upstreams { + destination_name = "count-api" + local_bind_port = 8080 + } + } +} ``` [job]: /docs/job-specification/job 'Nomad job Job Specification' @@ -82,3 +85,4 @@ The following example is a proxy specification that includes upstreams configura [sidecar_service]: /docs/job-specification/sidecar_service 'Nomad sidecar service Specification' [upstreams]: /docs/job-specification/upstreams 'Nomad upstream config Specification' [port]: /docs/job-specification/network#port-parameters 'Nomad network port configuration' +[expose]: /docs/job-specification/expose 'Nomad proxy expose configuration' diff --git a/website/pages/docs/job-specification/service.mdx b/website/pages/docs/job-specification/service.mdx index 190e31c698a..3ecc143a52f 100644 --- a/website/pages/docs/job-specification/service.mdx +++ b/website/pages/docs/job-specification/service.mdx @@ -213,6 +213,11 @@ scripts. add the IP of the service and the port, so this is just the relative URL to the health check endpoint. This is required for http-based health checks. +- `expose` `(bool: false)` - Specifies whether an [Expose Path](/docs/job-specification/expose#path-parameters) + should be automatically generated for this check. Only compatible with + Connect-enabled task-group services using the default Connect proxy. Check + must be of [`type`][type] `http` or `grpc`. + - `port` `(string: )` - Specifies the label of the port on which the check will be performed. Note this is the _label_ of the port and not the port number unless `address_mode = driver`. The port label must match one defined @@ -684,6 +689,7 @@ advertise and check directly since Nomad isn't managing any port assignments. [qemu]: /docs/drivers/qemu 'Nomad qemu Driver' [restart_stanza]: /docs/job-specification/restart 'restart stanza' [connect]: /docs/job-specification/connect 'Nomad Consul Connect Integration' +[type]: /docs/job-specification/service#type [shutdowndelay]: /docs/job-specification/task#shutdown_delay [killsignal]: /docs/job-specification/task#kill_signal [killtimeout]: /docs/job-specification/task#kill_timeout From 5d82951cb64b5777cee7a686dd40bb6bde580444 Mon Sep 17 00:00:00 2001 From: Seth Hoenig Date: Thu, 2 Apr 2020 16:42:38 -0600 Subject: [PATCH 2/2] docs: add exposed TG service check examples to expose docs --- .../pages/docs/job-specification/expose.mdx | 22 +++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/website/pages/docs/job-specification/expose.mdx b/website/pages/docs/job-specification/expose.mdx index 4212d8f931c..5634aa999ed 100644 --- a/website/pages/docs/job-specification/expose.mdx +++ b/website/pages/docs/job-specification/expose.mdx @@ -55,6 +55,16 @@ job "expose-example" { } } } + + check { + name = "dashboard-health" + type = "http" + port = "expose" + path = "/health" + interval = "10s" + timeout = "3s" + expose = true + } } } } @@ -141,5 +151,17 @@ definitions. For these cases the [expose](/docs/job-specification/service#expose parameter in the service check stanza can be used to automatically generate the expose path configuration. +```hcl +check { + type = "http" + name = "dashboard-health" + port = "expose" + path = "/health" + interval = "10s" + timeout = "3s" + expose = true +} +``` + [path]: /docs/job-specification/expose#path-parameters 'Nomad Expose Path Parameters' [port]: /docs/job-specification/network#port-parameters 'Nomad Port Parameters'