Consul Connect over IPv6 (except tproxy)

[gulducat]

Mostly resolves #7905 -- #23882 introduced IPv6 support to Nomad's "bridge" network mode, and this extends that to Consul Connect (which also requires "bridge" mode). I say "mostly" because Transparent Proxy still does not work (the Consul CNI plugin does not do any ip6tables at the moment for its extra functionality).

Along the way, I found that since we were always setting Connect/Envoy's bind_address to "0.0.0.0", the user couldn't pick anything else (like I had tried "::"). In particular, even with this PR auto-detecting IPv6, I imagine a user might like to set it to "" (empty), so that Consul proxy-defaults config can come into play. I did not add a config option for the client along these lines, but an individual job could set it like so:

connect {
  sidecar_service {
    proxy {
      config {
        bind_address = "" # let consul proxy-defaults handle it
      }

Or set it to whatever they may like.

---

My preferred way to replicate the behavior is (on a host/network with ipv6 support), enable ipv6 on the Nomad bridge and prefer ipv6 for services on a client (per #23388):

client {
  enabled = true

  bridge_network_subnet_ipv6 = "fd00:a110:c8::/120"
  preferred_address_family   = "ipv6"
}

Consul can run in dev mode. consul agent -dev

Then use the basic countdash example:

$ nomad job init -connect -short
Example job file written to example.nomad.hcl

example.nomad.hcl

job "countdash" {

  group "api" {
    network {
      mode = "bridge"
    }

    service {
      name = "count-api"
      port = "9001"

      connect {
        sidecar_service {}
      }
    }

    task "web" {
      driver = "docker"

      config {
        image          = "hashicorpdev/counter-api:v3"
        auth_soft_fail = true
      }
    }
  }

  group "dashboard" {
    network {
      mode = "bridge"

      port "http" {
        static = 9002
        to     = 9002
      }
    }

    service {
      name = "count-dashboard"
      port = "9002"

      connect {
        sidecar_service {
          proxy {
            upstreams {
              destination_name = "count-api"
              local_bind_port  = 8080
            }
          }
        }
      }
    }

    task "dashboard" {
      driver = "docker"

      env {
        COUNTING_SERVICE_URL = "http://${NOMAD_UPSTREAM_ADDR_count_api}"
      }

      config {
        image          = "hashicorpdev/counter-dashboard:v3"
        auth_soft_fail = true
      }
    }
  }
}

The alloc and service addresses will be ipv6, health checks pass, and the counter counts.

[tgross]

This kind of invalid input just blows up downstream, right?

[gulducat]

Yes, with this "anything" for example, the sidecar logs will show envoy squawking like

[2024-10-14 20:36:19.901][1][warning][config] [source/extensions/config_subscription/grpc/delta_subscription_state.cc:269] delta config for type.googleapis.com/envoy.config.listener.v3.Listener rejected: Error adding/updating listener(s) public_listener:anything:28417: malformed IP address: anything
[2024-10-14 20:36:19.901][1][warning][config] [source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:138] gRPC config for type.googleapis.com/envoy.config.listener.v3.Listener rejected: Error adding/updating listener(s) public_listener:anything:28417: malformed IP address: anything

It will continue running, but the "Connect Sidecar Listening" health check fails.

So maybe it doesn't quite "blow up" but it's not too hard to track down.