Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support cross-namespace BackendRefs in HTTPRoutes #806

Merged
merged 9 commits into from
Jul 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 5 additions & 6 deletions docs/gateway-api-compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ This document describes which Gateway API resources NGINX Kubernetes Gateway sup
| [TLSRoute](#tlsroute) | Not supported |
| [TCPRoute](#tcproute) | Not supported |
| [UDPRoute](#udproute) | Not supported |
| [ReferenceGrant](#referencegrant) | Partially supported |
| [ReferenceGrant](#referencegrant) | Supported |
| [Custom policies](#custom-policies) | Not supported |

## Terminology
Expand Down Expand Up @@ -152,19 +152,18 @@ Fields:

### ReferenceGrant

> Status: Partially supported.

NKG only supports ReferenceGrants that permit Gateways to reference Secrets.
> Status: Supported.
> Support Level: Core

Fields:
* `spec`
* `to`
* `group` - supported.
* `kind` - partially supported. Only `Secret`.
* `kind` - supports `Secret` and `Service`.
* `name`- supported.
* `from`
* `group` - supported.
* `kind` - partially supported. Only `Gateway`.
* `kind` - supports `Gateway` and `HTTPRoute`.
* `namespace`- supported.

### Custom Policies
Expand Down
128 changes: 128 additions & 0 deletions examples/cross-namespace-routing/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
# Example

In this example, we expand on the simple [cafe-example](../cafe-example) by using a ReferenceGrant to route to backends
in a different namespace from our HTTPRoutes.

## Running the Example

## 1. Deploy NGINX Kubernetes Gateway

1. Follow the [installation instructions](/docs/installation.md) to deploy NGINX Gateway.

1. Save the public IP address of NGINX Kubernetes Gateway into a shell variable:

```
GW_IP=XXX.YYY.ZZZ.III
```

1. Save the port of NGINX Kubernetes Gateway:

```
GW_PORT=<port number>
```

## 2. Deploy the Cafe Application

1. Create the cafe namespace and cafe application:

```
kubectl apply -f cafe-ns-and-app.yaml
```

1. Check that the Pods are running in the `cafe` namespace:

```
kubectl -n cafe get pods
NAME READY STATUS RESTARTS AGE
coffee-6f4b79b975-2sb28 1/1 Running 0 12s
tea-6fb46d899f-fm7zr 1/1 Running 0 12s
```

## 3. Configure Routing

1. Create the `Gateway`:

```
kubectl apply -f gateway.yaml
```

1. Create the `HTTPRoute` resources:

```
kubectl apply -f cafe-routes.yaml
```
1. Create the `ReferenceGrant`:

```
kubectl apply -f reference-grant.yaml
```
This ReferenceGrant allows all HTTPRoutes in the `default` namespace to reference all Services in the `cafe`
namespace.

## 4. Test the Application

To access the application, we will use `curl` to send requests to the `coffee` and `tea` Services.

To get coffee:

```
curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee
Server address: 10.12.0.18:80
Server name: coffee-7586895968-r26zn
```

To get tea:

```
curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea
Server address: 10.12.0.19:80
Server name: tea-7cd44fcb4d-xfw2x
```

## 5. Remove the ReferenceGrant

To restrict access to Services in the `cafe` Namespace, we can delete the ReferenceGrant we created in
Step 3:

```
kubectl delete -f reference-grant.yaml
```

Now, if we try to access the application over HTTP, we will get an internal server error:
```
curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea

<html>
<head><title>500 Internal Server Error</title></head>
<body>
<center><h1>500 Internal Server Error</h1></center>
<hr><center>nginx/1.25.1</center>
</body>
</html>
```

kate-osborn marked this conversation as resolved.
Show resolved Hide resolved
You can also check the conditions of the HTTPRoutes `coffee` and `tea` to verify that the reference is not permitted:

```
kubectl describe httproute coffee

Condtions:
Message: Backend ref to Service cafe/coffee not permitted by any ReferenceGrant
Observed Generation: 1
Reason: RefNotPermitted
Status: False
Type: ResolvedRefs
Controller Name: k8s-gateway.nginx.org/nginx-gateway-controller
```

```
kubectl describe httproute tea

Condtions:
Message: Backend ref to Service cafe/tea not permitted by any ReferenceGrant
Observed Generation: 1
Reason: RefNotPermitted
Status: False
Type: ResolvedRefs
Controller Name: k8s-gateway.nginx.org/nginx-gateway-controller
```
74 changes: 74 additions & 0 deletions examples/cross-namespace-routing/cafe-ns-and-app.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
apiVersion: v1
kind: Namespace
metadata:
name: cafe
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: coffee
namespace: cafe
spec:
replicas: 1
selector:
matchLabels:
app: coffee
template:
metadata:
labels:
app: coffee
spec:
containers:
- name: coffee
image: nginxdemos/nginx-hello:plain-text
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: coffee
namespace: cafe
spec:
ports:
- port: 80
targetPort: 8080
protocol: TCP
name: http
selector:
app: coffee
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: tea
namespace: cafe
spec:
replicas: 1
selector:
matchLabels:
app: tea
template:
metadata:
labels:
app: tea
spec:
containers:
- name: tea
image: nginxdemos/nginx-hello:plain-text
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: tea
namespace: cafe
spec:
ports:
- port: 80
targetPort: 8080
protocol: TCP
name: http
selector:
app: tea
39 changes: 39 additions & 0 deletions examples/cross-namespace-routing/cafe-routes.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
name: coffee
spec:
parentRefs:
- name: gateway
sectionName: http
hostnames:
- "cafe.example.com"
rules:
- matches:
- path:
type: PathPrefix
value: /coffee
backendRefs:
- name: coffee
namespace: cafe
port: 80
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
name: tea
spec:
parentRefs:
- name: gateway
sectionName: http
hostnames:
- "cafe.example.com"
rules:
- matches:
- path:
type: Exact
value: /tea
backendRefs:
- name: tea
namespace: cafe
port: 80
13 changes: 13 additions & 0 deletions examples/cross-namespace-routing/gateway.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
name: gateway
labels:
domain: k8s-gateway.nginx.org
spec:
gatewayClassName: nginx
listeners:
- name: http
port: 80
protocol: HTTP
hostname: "*.example.com"
13 changes: 13 additions & 0 deletions examples/cross-namespace-routing/reference-grant.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
name: access-to-cafe-services
namespace: cafe
spec:
to:
- group: ""
kind: Service
from:
- group: gateway.networking.k8s.io
kind: HTTPRoute
namespace: default
2 changes: 2 additions & 0 deletions examples/https-termination/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,8 @@ curl: (7) Failed to connect to cafe.example.com port 443 after 0 ms: Connection
You can also check the conditions of the Gateway `https` Listener to verify the that the reference is not permitted:

```
kubectl describe gateway gateway

Name: https
Conditions:
Last Transition Time: 2023-06-26T20:23:56Z
Expand Down
2 changes: 1 addition & 1 deletion examples/https-termination/reference-grant.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
name: allow-default-to-cafe-secret
name: access-to-cafe-secret
namespace: certificate
spec:
to:
Expand Down
Loading