Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Don't merge, PR to check diff] Tls route support mar20 #3

Open
wants to merge 47 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
5e72748
Add hacky tls-passthrough config
liwenwu-amazon Mar 1, 2024
543dd3b
Add back logic to overwrite aws sdk
liwenwu-amazon Mar 1, 2024
eb78a56
Add back the capability of using different SDK
liwenwu-amazon Mar 1, 2024
6741333
Add code to extract default target group for TLS route
liwenwu-amazon Mar 2, 2024
714a514
Add default TG for TLS-passthrough route
liwenwu-amazon Mar 2, 2024
75ad82c
skip configure rule for TLSroute
liwenwu-amazon Mar 4, 2024
d666779
Add more TLS-PASSTHROUGH configs
liwenwu-amazon Mar 4, 2024
f5f2355
Add TLSRoute CRD and an example on TLSRoute
liwenwu-amazon Mar 7, 2024
bd77253
Add a nginix tls server service yaml
liwenwu-amazon Mar 8, 2024
9bc05d4
Clone GRPCRoute handling code to TLSRoute handling code
liwenwu-amazon Mar 9, 2024
9245653
Fix typo and use TLSRoute instead of TCPRoute
liwenwu-amazon Mar 11, 2024
8927704
Add yaml for nginx test server which can be used for TLS traffic
liwenwu-amazon Mar 11, 2024
177f655
Add TLS route example
liwenwu-amazon Mar 11, 2024
46083ea
Update tlsroute
liwenwu-amazon Mar 11, 2024
16758bc
Add more tls route type
liwenwu-amazon Mar 11, 2024
ab585ed
Add more debug info
liwenwu-amazon Mar 11, 2024
dc01191
Add TLSRoute type for router type
liwenwu-amazon Mar 11, 2024
7502039
Allow targetgroup policy attached to serviceExport
liwenwu-amazon Mar 13, 2024
74aee06
Add example on blue/green TLSRoute example
liwenwu-amazon Mar 13, 2024
4091d6f
Add support for weight target groups for default rule used by TLS
liwenwu-amazon Mar 14, 2024
02cd1f8
Cleanup the debugging msg
liwenwu-amazon Mar 18, 2024
370bc05
Cleanup debug msg
liwenwu-amazon Mar 18, 2024
b915533
Cleanup the debugging msg
liwenwu-amazon Mar 18, 2024
74504e9
Cleanup the debug msgs
liwenwu-amazon Mar 18, 2024
6e953f2
Cleanup debug msgs
liwenwu-amazon Mar 18, 2024
7b69e35
Cleanup debug msgs
liwenwu-amazon Mar 18, 2024
4fde6ec
Fixed unit test failure
liwenwu-amazon Mar 19, 2024
8676226
Add rbac permission for tlsroute
liwenwu-amazon Mar 19, 2024
aa15649
Temporary backout the change for amd platform
liwenwu-amazon Mar 19, 2024
57d55f9
Update deploy-v1.0.3-tls.yaml to point to alpha image
liwenwu-amazon Mar 19, 2024
6a4721b
Add readme on how to upgrade to tlsroute alpha support
liwenwu-amazon Mar 19, 2024
08c4a9b
Update tlsroute passthrough doc
liwenwu-amazon Mar 20, 2024
f4161e7
clarify the alpha branch
liwenwu-amazon Mar 20, 2024
60d9d86
Remove env in the alpha deployment yaml
liwenwu-amazon Mar 20, 2024
7b38476
Update readme and use public ecr image
liwenwu-amazon Mar 20, 2024
6577feb
Add TLS_PASSTHROUGH listener unit test
liwenwu-amazon Mar 22, 2024
f06386e
Add unit tests to TLSRoute
liwenwu-amazon Mar 25, 2024
002b672
Add more unit tests to tlsroute
liwenwu-amazon Mar 25, 2024
f91ba15
Add tls section into gateway object
liwenwu-amazon Apr 2, 2024
d94d471
Add initial tlsroute test
liwenwu-amazon Apr 3, 2024
7ea38c1
Add tlsroute test which has the configuration portion
liwenwu-amazon Apr 3, 2024
46b63ff
Add traffic test
liwenwu-amazon Apr 3, 2024
f207674
Cleanup the test code
liwenwu-amazon Apr 3, 2024
27e4012
Add tlsroute2serviceimport e2e test
liwenwu-amazon Apr 4, 2024
9fa94fc
Add logic to handle delete target group which is referenced by TLSRoute
liwenwu-amazon Apr 5, 2024
3a73a3f
Add logic to handle target-group tagging used by TLSroute
liwenwu-amazon Apr 5, 2024
eb518a4
Update FOCUS string for tlsroute2serviceimport test
liwenwu-amazon Apr 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
# Build the manager binary
FROM --platform=$BUILDPLATFORM golang:1.20.5 as builder
FROM docker.io/library/golang:1.20.5 as builder

WORKDIR /workspace
# Copy the Go Modules manifests
COPY go.mod go.mod
COPY go.sum go.sum
COPY scripts/aws_sdk_model_override/aws-sdk-go/go.mod scripts/aws_sdk_model_override/aws-sdk-go/go.mod
COPY scripts/aws_sdk_model_override/aws-sdk-go/go.sum scripts/aws_sdk_model_override/aws-sdk-go/go.sum

# cache deps before building and copying source so that we don't need to re-download as much
# and so that source changes don't invalidate our downloaded layer
Expand All @@ -20,11 +22,11 @@ ARG TARGETOS
ARG TARGETARCH

# Build
RUN CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -a -o manager main.go
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -o manager main.go

# Use distroless as minimal base image to package the manager binary
# Refer to https://github.com/GoogleContainerTools/distroless for more details
FROM --platform=$TARGETPLATFORM gcr.io/distroless/static:nonroot
FROM gcr.io/distroless/static:nonroot
WORKDIR /
COPY --from=builder /workspace/manager .
USER 65532:65532
Expand Down
3 changes: 2 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ test: ## Run tests.
.PHONY: toolchain
toolchain: ## Install developer toolchain
./hack/toolchain.sh
./setup.sh

##@ Deployment

Expand Down Expand Up @@ -158,4 +159,4 @@ webhook-e2e-test:
./suites/webhook/... \
--ginkgo.focus="${FOCUS}" \
--ginkgo.skip="${SKIP}" \
--ginkgo.v
--ginkgo.v
902 changes: 902 additions & 0 deletions config/crds/bases/gateway.networking.k8s.io_tlsroutes.yaml

Large diffs are not rendered by default.

26 changes: 26 additions & 0 deletions config/rbac/cluster-role-controller.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,32 @@ rules:
- get
- patch
- update
- apiGroups:
- gateway.networking.k8s.io
resources:
- tlsroutes
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- gateway.networking.k8s.io
resources:
- tlsroutes/finalizers
verbs:
- update
- apiGroups:
- gateway.networking.k8s.io
resources:
- tlsroutes/status
verbs:
- get
- patch
- update
- apiGroups:
- application-networking.k8s.aws
resources:
Expand Down
Binary file added docs/images/tlsroute-multi-cluster.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
245 changes: 245 additions & 0 deletions docs/tls-passthrough.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,245 @@
# TLS Passthrough Support (alpha)

[Kubernetes Gateway API](https://gateway-api.sigs.k8s.io/guides/tls/) lays out the general guidelines on how to configure TLS passthrough. Here are examples on how to use them against AWS VPC Lattice.

## Upgrade Controller

### Install gateway API TLSRoute CRD

Here is [TLSRoute CRD](https://github.com/liwenwu-amazon/aws-application-networking-k8-publics/blob/tls-route-support/config/crds/bases/gateway.networking.k8s.io_tlsroutes.yaml).


Here is alpha branch repo: https://github.com/liwenwu-amazon/aws-application-networking-k8-publics/tree/tls-route-support

```
# clone the TLS support alpha repo
git clone [email protected]:liwenwu-amazon/aws-application-networking-k8-publics.git

# install CRD
kubectl apply -f config/crds/bases/gateway.networking.k8s.io_tlsroutes.yaml

# Verfiy TLSRoute CRD
kubectl get crd tlsroutes.gateway.networking.k8s.io
NAME CREATED AT
tlsroutes.gateway.networking.k8s.io 2024-03-07T23:16:22Z

```

### Upgrade Controller Image

```
kubectl apply -f example/deploy-v1.0.3-tls.yaml
```

## Setup

### Configure TLS Passthrough Listener on Gateway

```
kubectl apply -f examples/gateway-tls-passthrough.yaml
```

```
# tls listener config snips ...
apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
name: my-hotel-tls
spec:
gatewayClassName: amazon-vpc-lattice
listeners:
...
- name: tls
protocol: TLS
port: 443
tls:
mode: Passthrough
...
```

### Configure TLSRoute

```
# configure nginx backend
kubectl apply -f example/nginx-tls.yaml

# configure nginx tls route
kubectl apply -f examples/nginx-route-tls.yaml


```

```
# Verfiy controller has reconciled nginx-tls route
kubectl get tlsroute nginx-tls -o yaml
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: TLSRoute
metadata:
annotations:
application-networking.k8s.aws/lattice-assigned-domain-name: nginx-tls-default-0ddade421111c485e.7d67968.lattice.us-west-2.on.aws
...
name: nginx-tls
namespace: default
...


```

```
# Verify TLS Passthrough Traffic
kubectl get pod
NAME READY STATUS RESTARTS AGE
app-694bb8f486-49rv4 1/1 Running 3 (5d9h ago) 26d
app-694bb8f486-jsqjk 1/1 Running 3 (5d9h ago) 26d

kubectl exec -ti app-694bb8f486-49rv4 sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.

# using lattice created domain name
sh-4.2# curl -k -v https://nginx-tls-default-0ddade421111c485e.7d67968.mercury-svcs-gamma.us-west-2.on.aws
* Trying 169.254.171.32:443...
* Connected to nginx-tls-default-0ddade421111c485e.7d67968.mercury-svcs-gamma.us-west-2.on.aws (169.254.171.32) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
* CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: C=US; ST=wa; L=seattle; O=aws; OU=lattice; CN=liwen.ssl-test.com; [email protected]
* start date: Mar 5 21:26:24 2024 GMT


# use customer defined name
curl -k -v https://nginx-test.my-test.com --resolve nginx-test.my-test.com:443:169.254.171.32
* Added nginx-test.my-test.com:443:169.254.171.32 to DNS cache
* Hostname nginx-test.my-test.com was found in DNS cache
* Trying 169.254.171.32:443...
* Connected to nginx-test.my-test.com (169.254.171.32) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
* CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: C=US; ST=wa; L=seattle; O=aws; OU=lattice; CN=liwen.ssl-test.com; [email protected]


```

### Configure TLSRoute span multiple Kubernetes Clusters

![tlsoute multi cluster](./images/tlsroute-multi-cluster.png)

```
# cluster-1
# create tls-rate1 Kubernetes Service
kubectl apply -f examples/tls-rate1.yaml

```

#### Configure ServieExport using TCP protocol in cluster2

```
# cluster-2
# create tls-rate2 Kubernetes Service
kubectl apply -f examples/tls-rate2.yaml

# create serviceexport
kubectl apply -f examples/tls-rate2-export.yaml

# create targetgroup policy to configure TCP protocol for tls-rate2
kubectl apply -f examples/tls-rate-policy.yaml

```

```
# snips of serviceexport config
apiVersion: application-networking.k8s.aws/v1alpha1
kind: ServiceExport
metadata:
name: tls-rate-2
annotations:
application-networking.k8s.aws/federation: "amazon-vpc-lattice"

# snips of targetgroup policy config
apiVersion: application-networking.k8s.aws/v1alpha1
kind: TargetGroupPolicy
metadata:
name: tls-rate2
spec:
targetRef:
group: "application-networking.k8s.aws"
kind: ServiceExport
name: tls-rate2
protocol: TCP
protocolVersion: HTTP1

```

#### Configure ServiceImport in cluster1

```
kubectl apply -f examples/tls-rate2-import.yaml
```

#### Configure TLSRoute for bluegreen deployment

```
kubectl apply -f examples/tls-rate-bluegreen.yaml
```

```
# snip of TLSRoute span multiple Kubernetes Clusters
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: TLSRoute
metadata:
name: tls-rate
spec:
hostnames:
- tls-rate.my-test.com
parentRefs:
- name: my-hotel-tls
sectionName: tls
rules:
- backendRefs:
- name: tls-rate1
kind: Service
port: 443
weight: 10
- name: tls-rate2 <---------- to 2nd Kubernetes Cluster
kind: ServiceImport
port: 443
weight: 90




```
Loading
Loading