Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi cluster doc improve #533

Merged
merged 14 commits into from
Nov 23, 2023
51 changes: 51 additions & 0 deletions docs/guides/multi-cluster.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Recommended Multi-Cluster Architecture

Here is a recommended multi-cluster architecture if you'd like to setup kubernetes service-to-service communications across multiple clusters.

Suppose your organization would like to have one large infrastructure that have many clusters in one AWS account. It's recommended your infrastructure includes the following components:
- One manually created VPC Lattice service network, (it could be created by either AWS Console, CLI, CloudFormation, Terraform or any other tools)
- Create vpc service network associations between VPC Lattice service network and each config cluster's VPC and workload clusters' VPCs
- Multiple workload cluster(s), that are used to run application workload(s). Workload cluster(s) should only have following workloads related kubernetes objects:
- Application workload(s) (`Pod(s)`, `Deployment(s)` etc.)
- `Service(s)` for application workload(s)
- `ServiceExport(s)`, that export kubernetes application Service(s) to the "config cluster"
- One extra dedicated "config cluster", which acts as a "service network control plane" and it should include following kubernetes objects:
- One `Gateway` that has __same name__ as the manually created VPC Lattice service network name
- `ServiceImport(s)`, that reference to kubernetes application services that are exported from workload cluster(s)
- `HTTPRoute(s)`,`GRPCRoute(s)`, that have rules backendRefs to `ServiceImport(s)` that referring kubernetes application service(s) in workload cluster(s)

![config cluster and multiple workload clusters](../images/multi-cluster.png)


Above architecture is recommended for following reasons:
- Putting all Route(s) and only one Gateway in the config cluster is easier to manage and have less chance to have conflict.
- Reduce blast radius. If there is any issue in the config cluster, it won't affect the applications in the workload cluster(s).
- It is easier to separate permissions and duties by different roles in your organization. For example:
- The infra operator role only has permissions and manages the resource in the config cluster and VPC Lattice resource,
- The application developer role only has permissions and manages the resource in the workload cluster(s).



Following steps show you how to set up this recommended multi-cluster architecture with 1 config cluster and 2 workload clusters.
1. Create 3 k8s clusters: `config cluster`, `workload cluster-1`, `workload cluster-2`. Install AWS Gateway API Controller in each cluster, you could follow this instruction [deploy.md](deploy.md)
1. Create a VPC Lattice `ServiceNetwork` with name `my-gateway`
1. Create `VPCServiceNetworkAssociation(s)` between previous step created service network and each config cluster's VPC and workload clusters' VPCs
1. Setup following resource in the workload cluster1:
```
kubectl apply -f examples/service-1.yaml
kubectl apply -f examples/service-1-export.yaml
```
1. Setup following resource in the workload cluster2:
```
kubectl apply -f examples/service-2.yaml
kubectl apply -f examples/service-2-export.yaml
```
1. Setup following resource in the config cluster:
```
kubectl apply -f examples/my-gateway.yaml
kubectl apply -f examples/my-httproute.yaml
kubectl apply -f examples/service-1-import.yaml
kubectl apply -f examples/service-2-import.yaml
```
1. At this point, the connectivity setup finished, pods in workload cluster1 are able to communicate with `service-2` in workload cluster2 (and vice versa) via the `my-httproute` DNS name.
1. Furthermore, you could have more workload clusters to join the `my-gateway` service network by creating the `ServiceNewtorkAssociation(s)`, workloads there all be able to communicate with `service-1` and `service-2` via the `my-httproute` DNS name and path matching.
180 changes: 0 additions & 180 deletions docs/guides/multi-sn.md

This file was deleted.

Binary file modified docs/images/GatewayUserGuideFigures.pptx
Binary file not shown.
Binary file added docs/images/multi-cluster.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed docs/images/multi-sn.png
Binary file not shown.
10 changes: 10 additions & 0 deletions examples/my-gateway.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
name: my-gateway
spec:
gatewayClassName: amazon-vpc-lattice
listeners:
- name: http
protocol: HTTP
port: 80
23 changes: 23 additions & 0 deletions examples/my-httproute.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
name: my-httproute
spec:
parentRefs:
- name: my-hotel
sectionName: http
rules:
- backendRefs:
- name: service-1
kind: ServiceImport
matches:
- path:
type: PathPrefix
value: /service-1
- backendRefs:
- name: service-2
kind: ServiceImport
matches:
- path:
type: PathPrefix
value: /service-2
6 changes: 6 additions & 0 deletions examples/service-1-export.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
apiVersion: application-networking.k8s.aws/v1alpha1
kind: ServiceExport
metadata:
name: service-1
annotations:
application-networking.k8s.aws/federation: "amazon-vpc-lattice"
9 changes: 9 additions & 0 deletions examples/service-1-import.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
apiVersion: application-networking.k8s.aws/v1alpha1
kind: ServiceImport
metadata:
name: service-1
spec:
type: ClusterSetIP
ports:
- port: 80
protocol: TCP
36 changes: 36 additions & 0 deletions examples/service-1.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: service-1
labels:
app: service-1
spec:
replicas: 2
selector:
matchLabels:
app: service-1
template:
metadata:
labels:
app: service-1
spec:
containers:
- name: service-1
image: public.ecr.aws/x2j8p8w7/http-server:latest
env:
- name: PodName
value: "service-1 handler pod"


---
apiVersion: v1
kind: Service
metadata:
name: service-1
spec:
selector:
app: service-1
ports:
- protocol: TCP
port: 80
targetPort: 8090
6 changes: 6 additions & 0 deletions examples/service-2-export.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
apiVersion: application-networking.k8s.aws/v1alpha1
kind: ServiceExport
metadata:
name: service-2
annotations:
application-networking.k8s.aws/federation: "amazon-vpc-lattice"
9 changes: 9 additions & 0 deletions examples/service-2-import.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
apiVersion: application-networking.k8s.aws/v1alpha1
kind: ServiceImport
metadata:
name: service-2
spec:
type: ClusterSetIP
ports:
- port: 80
protocol: TCP
36 changes: 36 additions & 0 deletions examples/service-2.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: service-2
labels:
app: service-2
spec:
replicas: 2
selector:
matchLabels:
app: service-2
template:
metadata:
labels:
app: service-2
spec:
containers:
- name: service-2
image: public.ecr.aws/x2j8p8w7/http-server:latest
env:
- name: PodName
value: "service-2 handler pod"


---
apiVersion: v1
kind: Service
metadata:
name: service-2
spec:
selector:
app: service-2
ports:
- protocol: TCP
port: 80
targetPort: 8090