Skip to content
This repository has been archived by the owner on Dec 4, 2024. It is now read-only.

Home

Jump to bottom
Justin Lee edited this page Mar 21, 2018 · 28 revisions

Welcome to the marathon-lb wiki!

Examples

Custom HTTP headers in health check

This example adds the Host header to the health check executed by HAProxy:

{
  "id":"app",
  "labels": {
    "HAPROXY_GROUP": "external",
    "HAPROXY_0_BACKEND_HTTP_HEALTHCHECK_OPTIONS": "  option  httpchk GET {healthCheckPath} HTTP/1.1\\r\\nHost:\\ www\n  timeout check {healthCheckTimeoutSeconds}s\n"
  }
}

Setting timeout for long-lived socket connections

If you're trying to run a TCP service which uses long-lived sockets through HAProxy, such as a MySQL instance, you'll need to set longer timeouts for the backend. Try the following:

{
  "id":"app",
  "labels":{
    "HAPROXY_GROUP":"external",
    "HAPROXY_0_BACKEND_HEAD":"backend {backend}\n  balance {balance}\n  mode {mode}\n  timeout server 30m\n  timeout client 30m\n"
  }
}

The example above will set the client and server timeout to 30 minutes for the specified backend.

SSL Termination at an Elastic Load Balancer

Sometimes you want to allow an ELB to terminate SSL for you, but you still want marathon-lb to redirect non-HTTPS requests. ELBs use HTTP headers to communicate that the request came in via a secure channel and has been decrypted. Specifically, if the X-Forwarded-Proto header is set to https, then the request was decrypted by the ELB.

Unless you tell HAProxy to look for the X-Forwarded-Proto header, the request will appear as if it's unencrypted and will get redirected using standard the rules.

"labels": {
  "HAPROXY_0_BACKEND_HTTP_OPTIONS": "  acl is_proxy_https hdr(X-Forwarded-Proto) https\n  redirect scheme https unless { ssl_fc } or is_proxy_https\n"
}

This configuration instructs marathon-lb to generate a backend rule that looks for the X-Forwarded-Proto header or a regular TLS connection and redirect if neither are specified.

Enabling TLS v1.0

TLS v1.0 is deprecated, and no longer supported by the default MLB config. If you require TLS v1.0 support, you must supply a custom template for HAPROXY_HEAD. To do this, add a template URI to your MLB app definition like this:

  {
    "id":"/marathon-lb",
    "uris":["https://downloads.mesosphere.com/marathon/marathon-lb/templates-with-tls-10.tgz"]
  }

In order for this to work, you must be using a version of Marathon-LB based on debian:stretch or below. Marathon-LB based on debian:buster does not OS-level support for TLSv1.0 or TLSv1.1.

Specifically, these versions of Marathon-LB do NOT support TLSv1.0/TLSv1.1:

  • 1.8.1
  • 1.11.1

Disable Binding to Service Ports

If you do not want MLB to listen on service ports, you may disable the frontend definitions:

  {
    "labels": {
      "HAPROXY_GROUP": "external",
      "HAPROXY_0_FRONTEND_HEAD": "",
      "HAPROXY_0_FRONTEND_BACKEND_GLUE": ""
    }
  }

Wildcard vhosts resolution

If you want all subdomains for a given domain to resolve to a particular backend (for HTTP and HTTPS), use the following labels. Note that there is a period . required before the {hostname} in the HAPROXY_0_HTTPS_FRONTEND_ACL label. Note: be sure to disable the vhost map option by removing the --haproxy-map argument, if present.

{
  "labels": {
    "HAPROXY_0_BACKEND_WEIGHT": "-1",
    "HAPROXY_GROUP": "external",
    "HAPROXY_0_HTTP_FRONTEND_ACL": "  acl host_{cleanedUpHostname} hdr_end(host) -i {hostname}\n  use_backend {backend} if host_{cleanedUpHostname}\n",
    "HAPROXY_0_HTTPS_FRONTEND_ACL": "  use_backend {backend} if {{ ssl_fc_sni -m end .{hostname} }}\n",
    "HAPROXY_0_VHOST": "example.com"
  }
}

Enabling HAProxy logging

HAProxy uses socket based logging, and it's configured by default to log to /dev/log. To use HAProxy's logging, you must mount /dev/log into the container and enable logging for any backends or frontends you want to log. Afterward, you can examine the logs with journalctl. First, mount the volume into your /marathon-lb app:

{
  "id": "/marathon-lb",
  "container": {
    "type": "DOCKER",
    "volumes": [
      {
        "containerPath": "/dev/log",
        "hostPath": "/dev/log",
        "mode": "RW"
      }
    ],
    "docker": {
      "image": "mesosphere/marathon-lb:latest",
      "network": "HOST",
      "privileged": true,
      "parameters": [],
      "forcePullImage": true
    }
  }
}

Now, we'll set option httplog on one backend to enable logging. In this example, I'm using my personal website:

{
  "id": "/my-crappy-website",
  "cmd": null,
  "cpus": 0.5,
  "mem": 64,
  "disk": 0,
  "instances": 2,
  "container": {
    "type": "DOCKER",
    "volumes": [],
    "docker": {
      "image": "brndnmtthws/my-crappy-website",
      "network": "BRIDGE",
      "portMappings": [
        {
          "containerPort": 80,
          "hostPort": 0,
          "servicePort": 10012,
          "protocol": "tcp",
          "labels": {}
        }
      ],
      "privileged": false,
      "parameters": [],
      "forcePullImage": true
    }
  },
  "healthChecks": [
    {
      "path": "/",
      "protocol": "HTTP",
      "portIndex": 0,
      "gracePeriodSeconds": 10,
      "intervalSeconds": 15,
      "timeoutSeconds": 2,
      "maxConsecutiveFailures": 3,
      "ignoreHttp1xx": false
    }
  ],
  "labels": {
    "HAPROXY_0_USE_HSTS": "true",
    "HAPROXY_0_REDIRECT_TO_HTTPS": "true",
    "HAPROXY_GROUP": "external",
    "HAPROXY_0_BACKEND_HTTP_OPTIONS": "  option httplog\n  option forwardfor\n  http-request set-header X-Forwarded-Port %[dst_port]\n  http-request add-header X-Forwarded-Proto https if { ssl_fc }\n",
    "HAPROXY_0_VHOST": "diddyinc.com,www.diddyinc.com"
  },
  "portDefinitions": [
    {
      "port": 10012,
      "protocol": "tcp",
      "labels": {}
    }
  ]
}

Note: Enabling the httplog option will only affect the backend for the service port. To enable it for ports 80 and 443, you must modify the global HAProxy template.

Now, if you SSH into any public slave, you can view the logs using journalctl:

# journalctl -f -l SYSLOG_IDENTIFIER=haproxy

Custom HAProxy error response

You may specify a custom HAProxy error response by overriding the default errorfile directive. For example, you could return a redirect to a different backend in the event that no backends are available. To do this, add a template URI to your MLB app definition like this:

  {
    "id":"/marathon-lb",
    "uris":["https://downloads.mesosphere.com/marathon/marathon-lb/templates-custom-500-response.tar.gz"]
  }

The example above simply returns a custom 503 page, but you could instead return a redirect (by updating the templates/500.http file within the archive), like this:

HTTP/1.1 302 Found
Location: http://my-redirect-handler.computers.com/

Templates

(Pulled from https://docs.mesosphere.com/1.8/usage/service-discovery/marathon-lb/advanced/)

Marathon-LB has a templating feature for specifying custom HAProxy configuration parameters. Templates can be set either globally (for all apps), or on a per-app basis using labels. Let’s demonstrate an example of how to specify our own global template. Here’s the template we’ll use:

Global Template

To specify a global template:

  1. On your local machine, create a file called HAPROXY_HEAD in a directory called templates with the contents below:

    global
  daemon
  log /dev/log local0
  log /dev/log local1 notice
  maxconn 4096
  tune.ssl.default-dh-param 2048
defaults
  log global
  retries 3
  maxconn 3000
  timeout connect 5s
  timeout client 30s
  timeout server 30s
  option redispatch
listen stats
  bind 0.0.0.0:9090
  balance mode http
  stats enable monitor-uri /_haproxy_health_check

    In the code above, the following items have changed from the default: maxconn, timeout client, and timeout server.

  2. Tar or zip the file. Here's an example of how to do this:

#!/bin/bash

mkdir -p templates
cat >templates/HAPROXY_HEAD <<EOL
global
  daemon
  log /dev/log local0
  log /dev/log local1 notice
  maxconn 4096
  tune.ssl.default-dh-param 2048
defaults
  log               global
  retries           3
  maxconn           3000
  timeout connect   5s
  timeout client    30s
  timeout server    30s
  option            redispatch
listen stats
  bind 0.0.0.0:9090
  balance
  mode http
  stats enable
  monitor-uri /_haproxy_health_check
EOL

tar czf templates.tgz templates/

Take the file you created (templates.tgz if you use the script), and make it available from an HTTP server. If you’d like to use the sample one, use this URI: https://downloads.mesosphere.com/marathon/marathon-lb/templates.tgz

  1. Augment the Marathon-LB config by saving the following JSON in a file called options.json:

    {
  "marathon-lb": {
    "template-url":"https://downloads.mesosphere.com/marathon/marathon-lb/templates.tgz"
  }
}

  2. Launch the new Marathon-LB:

    dcos package install --options=options.json marathon-lb

    Your customized Marathon-LB HAProxy instance will now be running with the new template. [A full list of the templates available can be found here][2].

Clone this wiki locally