SPIFFE and Envoy to achieve cross-cluster mTLS where every cluster utilizes an independent SPIFFE trust domain

[anvega]

Title: SPIFFE and Envoy to achieve cross-cluster mTLS where every cluster utilizes an independent SPIFFE trust domain.

Description:
There is a strong desire from the community for the ability to use SPIFFE and Envoy together to achieve a “Federated Identity and Authentication scheme” in which each of multiple cluster manages its own SPIFFE trust domain. This is of particular importance for cross-cluster communication in environments which view clusters as different security and/or administrative domains.

The way SPIFFE validation works involves inspecting the trust domain of a presented SVID, and selecting the correct bundle based on the trust domain name. This allows trust domains to remain fully isolated and prevents one trust domain from minting certificates with identities belonging to another.

Envoy does not currently support this kind of validation today, either via CA pinning, native SPIFFE mTLS support or otherwise. A workaround to this deficiency, is to use SNI as a routing key for incoming TLS sessions. Each cluster uses a dedicated URL for contacting a foreign cluster service, and the foreing cluster's Envoy uses this URL (from the SNI) to determine the appropriate validation context. Taking this approach, it is not possible for a compromised cluster to impersonate workloads in other clusters.

From discussion with @lizan there might be alternate approaches to consider such as implementing validation through an extension.

Relevant Links:
#9284 "Enhance envoy listener TLS to support multiple trust domain"
SPIFFE Trust Domain and Bundle Spec

[htuch]

Assigned to @lizan for triage.

[lizan]

This makes sense. My idea during my discussion with @anvega is that we:

• Making an extension point into certificate validation
• Add an extension to support this custom validation for SPIFFE trust bundle, to select correct trusted CA from URI.

WDYT? @ggreenway @PiotrSikora @asraa

[ggreenway]

An extension point for cert validation makes a lot of sense to me. +1