Intra cluster DNS resolution for nginx ingress hosts

[sp3nx0r]

Ran into this looking at examples for service setup like MinIO / Prometheus that reference the external DNS names (example.com) for intra cluster resolution. Example for MinIO. CoreDNS doesn't know anything about example.com which is handled over in the separate k8s_gateway CoreDNS deployment. So my question is what's the preferred way to reference intra cluster resources (like Thanos calling out to MinIO, what do you put as the address, s3.example.com or s3.default.svc.cluster.local). Use the <service>.<namespace>.svc.cluster.local name? Looking at @onedr0p 's config, it is split between his two clusters (so maybe that's the magic, it always ends up forward out to the router DNS). But OOTB, the existing config doesn't allow for this resolution. One could use rewrite directive in the cluster's coredns to resolve that, but if you get fancy with namespaces that makes it harder (or you're going to have several rewrites). Or I suppose add it to the node's /etc/resolv.conf. Curious what the "best" route would be (and totally volunteering to write that up in the README)

[onedr0p]

If it's in the same cluster I would use k8s service addr (s3.default.svc.cluster.local).

[sp3nx0r]

That would work. Only downside is you wouldn't get the nginx HTTPS fronting for that service.

[onedr0p]

If you can directly access the service inside the cluster, you don't need TLS for that. I do this all the time for workloads in my cluster that need to communicate.

https://github.com/search?q=repo%3Aonedr0p%2Fhome-ops%20cluster.local&type=code