feat: multiport support

[buchdag]

This PR adds the ability to proxy to multiple ports on a single container using a new VIRTUAL_HOST_MULTIPORTS variable that take a YAML (or JSON) representation of the desired hostname / path / port / dest setup and completely override the VIRTUAL_HOST, VIRTUAL_PORT, VIRTUAL_PATH and VIRTUAL_DEST variables.

See #1504 for the previous discussion on this feature.

Thanks to @pini-gh for the actual feature's code.

Close #560
Close #911
Close #939
Close #1042
Close #1066
Close #1112
Close #1382
Close #1463
Close #1504
Close #2050

Supersede #259, #1405 and #2130

[buchdag]

Merging VIRTUAL_HOST and VIRTUAL_HOST_MULTIPORT seems to stop working when we start mixing VIRTUAL_HOST configs with and without VIRTUAL_PORT.

This work (no virtual path):

services:
  merged-singleport-1: # 172.18.0.2
    image: nginx:alpine
    environment:
      VIRTUAL_HOST: merged.nginx-proxy.tld

  merged-singleport-2: # 172.18.0.3
    image: nginx:alpine
    environment:
      VIRTUAL_HOST: merged.nginx-proxy.tld
      VIRTUAL_PORT: "81"

  merged-multiports: # 172.18.0.4
    image: nginx:alpine
    environment:
      VIRTUAL_HOST_MULTIPORTS: |-
        merged.nginx-proxy.tld:
          "/":
            port: 82

# merged.nginx-proxy.tld/
upstream merged.nginx-proxy.tld {
    # Container: nginx-proxy-merged-multiports-1
    # ...
    # Container: nginx-proxy-merged-singleport-1-1
    # ...
    # Container: nginx-proxy-merged-singleport-2-1
    # ...
    server 172.18.0.4:82;
    server 172.18.0.2:80;
    server 172.18.0.3:81;
}
server {
    server_name merged.nginx-proxy.tld;
    # ...
    location / {
        proxy_pass http://merged.nginx-proxy.tld;
        set $upstream_keepalive false;
    }
}

This work too (only virtual path):

services:
  merged-singleport-1: # 172.18.0.2
    image: nginx:alpine
    environment:
      VIRTUAL_HOST: merged.nginx-proxy.tld
      VIRTUAL_PATH: "/foo"

  merged-singleport-2: # 172.18.0.3
    image: nginx:alpine
    environment:
      VIRTUAL_HOST: merged.nginx-proxy.tld
      VIRTUAL_PATH: "/foo"
      VIRTUAL_PORT: "81"

  merged-multiports: # 172.18.0.4
    image: nginx:alpine
    environment:
      VIRTUAL_HOST_MULTIPORTS: |-
        merged.nginx-proxy.tld:
          "/foo":
            port: 82

# merged.nginx-proxy.tld/foo
upstream merged.nginx-proxy.tld-6dbd548cc03e44b8b44b6e68e56255ce4273ae49 {
    # Container: nginx-proxy-merged-multiports-1
    # ...
    # Container: nginx-proxy-merged-singleport-1-1
    # ...
    # Container: nginx-proxy-merged-singleport-2-1
    # ...
    server 172.18.0.4:82;
    server 172.18.0.2:80;
    server 172.18.0.3:81;
}
server {
    server_name merged.nginx-proxy.tld;
    # ...
    location /foo {
        proxy_pass http://merged.nginx-proxy.tld-6dbd548cc03e44b8b44b6e68e56255ce4273ae49;
        set $upstream_keepalive false;
    }
    location / {
        return 404;
    }
}

This work too (root path on multiport container only):

services:
  merged-singleport-2: # 172.18.0.3
    image: nginx:alpine
    environment:
      VIRTUAL_HOST: merged.nginx-proxy.tld
      VIRTUAL_PATH: "/foo"
      VIRTUAL_PORT: "81"

  merged-multiports: # 172.18.0.4
    image: nginx:alpine
    environment:
      VIRTUAL_HOST_MULTIPORTS: |-
        merged.nginx-proxy.tld:
          "/":
            port: 82
          "/foo":
            port: 83

# merged.nginx-proxy.tld/
upstream merged.nginx-proxy.tld {
    # Container: nginx-proxy-merged-multiports-1
    # ...
    server 172.18.0.4:82;
}
# merged.nginx-proxy.tld/foo
upstream merged.nginx-proxy.tld-6dbd548cc03e44b8b44b6e68e56255ce4273ae49 {
    # Container: nginx-proxy-merged-multiports-1
    # ...
    # Container: nginx-proxy-merged-singleport-2-1
    # ...
    server 172.18.0.4:83;
    server 172.18.0.3:81;
}
server {
    server_name merged.nginx-proxy.tld;
    # ...
    location / {
        proxy_pass http://merged.nginx-proxy.tld;
        set $upstream_keepalive false;
    }
    location /foo {
        proxy_pass http://merged.nginx-proxy.tld-6dbd548cc03e44b8b44b6e68e56255ce4273ae49;
        set $upstream_keepalive false;
    }
}

This doesn't (the first container's config is seemingly discarded):

services:
  merged-singleport-1: # 172.18.0.2
    image: nginx:alpine
    environment:
      VIRTUAL_HOST: merged.nginx-proxy.tld

  merged-singleport-2: # 172.18.0.3
    image: nginx:alpine
    environment:
      VIRTUAL_HOST: merged.nginx-proxy.tld
      VIRTUAL_PATH: "/foo"
      VIRTUAL_PORT: "81"

  merged-multiports: # 172.18.0.4
    image: nginx:alpine
    environment:
      VIRTUAL_HOST_MULTIPORTS: |-
        merged.nginx-proxy.tld:
          "/":
            port: 82
          "/foo":
            port: 84

# merged.nginx-proxy.tld/
upstream merged.nginx-proxy.tld {
    # Container: nginx-proxy-merged-multiports-1
    # ...
    # /!\ nginx-proxy-singleport-1-1 should be here /!\
    server 172.18.0.4:82;
}
# merged.nginx-proxy.tld/foo
upstream merged.nginx-proxy.tld-6dbd548cc03e44b8b44b6e68e56255ce4273ae49 {
    # Container: nginx-proxy-merged-multiports-1
    # ...
    # Container: nginx-proxy-merged-singleport-2-1
    # ...
    server 172.18.0.4:83;
    server 172.18.0.3:81;
}
server {
    server_name merged.nginx-proxy.tld;
    # ...
    location / {
        proxy_pass http://merged.nginx-proxy.tld;
        set $upstream_keepalive false;
    }
    location /foo {
        proxy_pass http://merged.nginx-proxy.tld-6dbd548cc03e44b8b44b6e68e56255ce4273ae49;
        set $upstream_keepalive false;
    }
}

[pini-gh]

This doesn't (the first container's config is seemingly discarded):

It is the case on the branch main as well.

Because this block:

    {{- $tmp_paths := groupBy $containers "Env.VIRTUAL_PATH" }}
    {{- $has_virtual_paths := gt (len $tmp_paths) 0}}
    {{- if not $has_virtual_paths }}
        {{- $tmp_paths = dict "/" $containers }}
    {{- end }}

silently discards containers with no VIRTUAL_PATH variable when at least one exists with.

[pini-gh]

this block:

    {{- $tmp_paths := groupBy $containers "Env.VIRTUAL_PATH" }}
    {{- $has_virtual_paths := gt (len $tmp_paths) 0}}
    {{- if not $has_virtual_paths }}
        {{- $tmp_paths = dict "/" $containers }}
    {{- end }}

silently discards containers with no VIRTUAL_PATH variable when at least one exists with.

Could be replaced with:

    {{- $tmp_paths := dict }}
    {{- range $path, $path_containers := groupBy $containers "Env.VIRTUAL_PATH" }}
        {{ $_ := set $tmp_paths $path $path_containers }}
    {{- end }}
    {{- $has_virtual_paths := gt (len $tmp_paths) 0}}
    {{- $_ := unset $tmp_paths "/" }}
    {{- $slash_containers := $containers }}
    {{- range $path, $path_containers := $tmp_paths }}
        {{ range $container := $path_containers }}
            {{ $slash_containers = without $slash_containers $container }}
        {{- end }}
    {{- end }}
    {{- if (gt (len $slash_containers) 0) }}
        {{- $_ := set $tmp_paths "/" $slash_containers }}
    {{- end }}

I couldn't find a simpler way because docker-gen map type is not compatible with sprig dict.

[buchdag]

I've tried that earlier an ran into the same dockergen map / sprig dict incompatibility.

I think a cleaner approach might be to add a groupByWithDefault function to docker-gen that would work like this :

{{- $tmp_paths := groupByWithDefault $containers "Env.VIRTUAL_PATH" "/" }}

I have it ready on a local branch and tested it with a modified template, seems to work like a charm.

While I'm at it, maybe groupByMultiWithDefault / groupByLabelWithDefault would be useful to clean the template too ?

[pini-gh]

While I'm at it, maybe groupByMultiWithDefault / groupByLabelWithDefault would be useful to clean the template too ?

Sure.

[buchdag]

I think we're pretty much done in term of documentation and tests here, marking the PR as ready for review.

[pini-gh]

LGTM. Please go ahead 👍

[SchoNie]

Very nice work @pini-gh & @buchdag 👍