Skip to content

Commit

Permalink
feat: Update Wireguard example to new layout
Browse files Browse the repository at this point in the history
  • Loading branch information
@bryantbiggs
bryantbiggs committed Sep 12, 2023
1 parent 7bb4e70 commit edb8492
Show file tree
Hide file tree
Showing 8 changed files with 352 additions and 286 deletions.
22 changes: 11 additions & 11 deletions .github/scripts/plan-examples.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,28 +3,28 @@
import re


def get_examples():
def get_patterns():
"""
Get all Terraform example root directories using their respective `versions.tf`;
Get all pattern root directories using their respective `main.tf`;
returning a string formatted json array of the example directories minus those that are excluded
"""
exclude = {
'examples/appmesh-mtls', # excluded until Rout53 is setup
'examples/blue-green-upgrade/environment',
'examples/blue-green-upgrade/modules/eks_cluster',
'examples/istio-multi-cluster/1.cluster1', # relies on remote state
'examples/istio-multi-cluster/2.cluster2', # relies on remote state
'examples/privatelink-access',
'patterns/appmesh-mtls', # excluded until Rout53 is setup
'patterns/blue-green-upgrade/environment',
'patterns/blue-green-upgrade/modules/eks_cluster',
'patterns/istio-multi-cluster/1.cluster1', # relies on remote state
'patterns/istio-multi-cluster/2.cluster2', # relies on remote state
'patterns/privatelink-access',
}

projects = {
x.replace('/versions.tf', '')
for x in glob.glob('patterns/**/versions.tf', recursive=True)
x.replace('/main.tf', '')
for x in glob.glob('patterns/**/main.tf', recursive=True)
if not re.match(r'^.+/_', x)
}

print(json.dumps(list(projects.difference(exclude))))


if __name__ == '__main__':
get_examples()
get_patterns()
150 changes: 127 additions & 23 deletions patterns/wireguard-with-cilium/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,27 +1,56 @@
# Transparent Encryption with Cilium and Wireguard

This pattern demonstrates Cilium configured in CNI chaining mode with VPC CNI and with Wireguard transparent encryption enabled on an Amazon EKS cluster.
This pattern demonstrates Cilium configured in CNI chaining mode with the VPC CNI and with Wireguard transparent encryption enabled on an Amazon EKS cluster.

- [Cilium CNI Chaining Documentation](https://docs.cilium.io/en/v1.12/gettingstarted/cni-chaining-aws-cni/)
- [Cilium Wireguard Encryption Documentation](https://docs.cilium.io/en/v1.12/gettingstarted/encryption-wireguard/)
- [Cilium CNI Chaining Documentation](https://docs.cilium.io/en/stable/installation/cni-chaining-aws-cni/)
- [Cilium Wireguard Encryption Documentation](https://docs.cilium.io/en/stable/security/network/encryption-wireguard/)

## Areas of Interest

- `eks.tf` contains the cluster configuration and the deployment of Cilium.
- There are no specific requirements from an EKS perspective, other than the Linux Kernel version used by the OS must be 5.10+.
On Amazon EKS, this is available starting with EKS 1.24, or users can utilize the Bottlerocket OS for EKS < 1.23
- `sample.tf` provides a sample application used to demonstrate the encrypted connectivity. This is optional and not required for the pattern.

## Deploy

See [here](https://aws-ia.github.io/terraform-aws-eks-blueprints/getting-started/#prerequisites) for the prerequisites and steps to deploy this pattern.

## Validate

1. List the daemonsets
1. Get the Cilium status from one of the Cilium pods.

Under the `Encryption` field, it should state `Wireguard` with a PubKey.
`NodeEncryption: Disabled` is expected since `NodeEncryption` was not enabled
via the Helm values provided.

```sh
kubectl get ds -n kube-system
kubectl -n kube-system exec -ti ds/cilium -- cilium status
```

```text
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
aws-node 2 2 2 2 2 <none> 156m
cilium 2 2 2 2 2 kubernetes.io/os=linux 152m
kube-proxy 2 2 2 2 2 <none> 156m
Defaulted container "cilium-agent" out of: cilium-agent, config (init), mount-cgroup (init), apply-sysctl-overwrites (init), mount-bpf-fs (init), clean-cilium-state (init), install-cni-binaries (init)
KVStore: Ok Disabled
Kubernetes: Ok 1.27+ (v1.27.4-eks-2d98532) [linux/amd64]
Kubernetes APIs: ["EndpointSliceOrEndpoint", "cilium/v2::CiliumClusterwideNetworkPolicy", "cilium/v2::CiliumEndpoint", "cilium/v2::CiliumNetworkPolicy", "cilium/v2::CiliumNode", "cilium/v2alpha1::CiliumCIDRGroup", "core/v1::Namespace", "core/v1::Pods", "core/v1::Service", "networking.k8s.io/v1::NetworkPolicy"]
KubeProxyReplacement: False [eth0 10.0.45.128 (Direct Routing), eth1 10.0.40.206]
Host firewall: Disabled
CNI Chaining: aws-cni
Cilium: Ok 1.14.1 (v1.14.1-c191ef6f)
NodeMonitor: Listening for events on 2 CPUs with 64x4096 of shared memory
Cilium health daemon: Ok
IPAM: IPv4: 1/254 allocated from 10.0.1.0/24,
IPv4 BIG TCP: Disabled
IPv6 BIG TCP: Disabled
BandwidthManager: Disabled
Host Routing: Legacy
Masquerading: Disabled
Controller Status: 20/20 healthy
Proxy Status: No managed proxy redirect
Global Identity Range: min 256, max 65535
Hubble: Ok Current/Max Flows: 4095/4095 (100.00%), Flows/s: 1.58 Metrics: Disabled
Encryption: Wireguard [NodeEncryption: Disabled, cilium_wg0 (Pubkey: Es25c2idJtRzE0/FKAOvKPJ7ybRmZ23KrufK3HOuZTY=, Port: 51871, Peers: 1)]
Cluster health: Probe disabled
```

2. Open a shell inside the cilium container
Expand All @@ -30,39 +59,114 @@ See [here](https://aws-ia.github.io/terraform-aws-eks-blueprints/getting-started
kubectl -n kube-system exec -ti ds/cilium -- bash
```

3. Verify Encryption is enabled

```sh
cilium status | grep Encryption
```

```text
Encryption: Wireguard [cilium_wg0 (Pubkey: b2krgbHgaCsVWALMnFLiS/RekhhcE36PXEjQ7T8+mW0=, Port: 51871, Peers: 1)]
```

4. Install [`tcpdump`](https://www.tcpdump.org/)
3. Install [`tcpdump`](https://www.tcpdump.org/)

```sh
apt-get update
apt-get install -y tcpdump
```

5. Start a packet capture on `cilium_wg0` and verify you see payload in clear text, it means the traffic is encrypted with wireguard
4. Start a packet capture on `cilium_wg0` and verify you see payload in clear text, it means the traffic is encrypted with wireguard

```sh
tcpdump -A -c 40 -i cilium_wg0 | grep "Welcome to nginx!"
```

```text
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on cilium_wg0, link-type RAW (Raw IP), snapshot length 262144 bytes
<title>Welcome to nginx!</title>
<h1>Welcome to nginx!</h1>
...
40 packets captured
40 packets received by filter
0 packets dropped by kernel
```

5. Deploy the Cilium connectivity resources to check and evaluate connectivity:

```sh
kubectl create ns cilium-test
kubectl apply -n cilium-test -f https://raw.githubusercontent.com/cilium/cilium/v1.14.1/examples/kubernetes/connectivity-check/connectivity-check.yaml
```

```text
deployment.apps/echo-a created
deployment.apps/echo-b created
deployment.apps/echo-b-host created
deployment.apps/pod-to-a created
deployment.apps/pod-to-external-1111 created
deployment.apps/pod-to-a-denied-cnp created
deployment.apps/pod-to-a-allowed-cnp created
deployment.apps/pod-to-external-fqdn-allow-google-cnp created
deployment.apps/pod-to-b-multi-node-clusterip created
deployment.apps/pod-to-b-multi-node-headless created
deployment.apps/host-to-b-multi-node-clusterip created
deployment.apps/host-to-b-multi-node-headless created
deployment.apps/pod-to-b-multi-node-nodeport created
deployment.apps/pod-to-b-intra-node-nodeport created
service/echo-a created
service/echo-b created
service/echo-b-headless created
service/echo-b-host-headless created
ciliumnetworkpolicy.cilium.io/pod-to-a-denied-cnp created
ciliumnetworkpolicy.cilium.io/pod-to-a-allowed-cnp created
ciliumnetworkpolicy.cilium.io/pod-to-external-fqdn-allow-google-cnp created
```

6. View the logs of any of the connectivity tests to view the results:

```sh
kubectl logs echo-a-6575c98b7d-xknsv -n cilium-test
```

```text
\{^_^}/ hi!
Loading /default.json
Done
Resources
http://:8080/private
http://:8080/public
Home
http://:8080
Type s + enter at any time to create a snapshot of the database
Watching...
GET /public 200 7.063 ms - 57
GET /public 200 3.126 ms - 57
GET /public 200 3.039 ms - 57
GET /public 200 2.776 ms - 57
GET /public 200 3.087 ms - 57
GET /public 200 2.781 ms - 57
GET /public 200 2.977 ms - 57
GET /public 200 2.596 ms - 57
GET /public 200 2.991 ms - 57
GET /public 200 2.708 ms - 57
GET /public 200 3.066 ms - 57
GET /public 200 2.616 ms - 57
GET /public 200 2.875 ms - 57
GET /public 200 2.689 ms - 57
GET /public 200 2.800 ms - 57
GET /public 200 2.556 ms - 57
GET /public 200 3.238 ms - 57
GET /public 200 2.538 ms - 57
GET /public 200 2.890 ms - 57
GET /public 200 2.666 ms - 57
GET /public 200 2.729 ms - 57
GET /public 200 2.580 ms - 57
GET /public 200 2.919 ms - 57
GET /public 200 2.630 ms - 57
GET /public 200 2.857 ms - 57
GET /public 200 2.716 ms - 57
GET /public 200 1.693 ms - 57
GET /public 200 2.715 ms - 57
GET /public 200 2.729 ms - 57
GET /public 200 2.655 ms - 57
```

## Destroy

{%
Expand Down
102 changes: 102 additions & 0 deletions patterns/wireguard-with-cilium/eks.tf
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
################################################################################
# Cluster
################################################################################

module "eks" {
source = "terraform-aws-modules/eks/aws"
version = "~> 19.16"

cluster_name = local.name
cluster_version = "1.27"
cluster_endpoint_public_access = true

# EKS Addons
cluster_addons = {
coredns = {}
kube-proxy = {}
vpc-cni = {}
}

vpc_id = module.vpc.vpc_id
subnet_ids = module.vpc.private_subnets

eks_managed_node_groups = {
initial = {
instance_types = ["m5.large"]
# Cilium Wireguard requires Linux Kernel 5.10 or above.
# For EKS 1.24 and above, the AMI the Kernel version is 5.10
# For EKS 1.23 and below, you need to use Bottlerocket OS. For example:
# ami_type = "BOTTLEROCKET_x86_64"
# platform = "bottlerocket"
min_size = 1
max_size = 3
desired_size = 2
}
}
# Extend node-to-node security group rules
node_security_group_additional_rules = {
# Cilium Wireguard Port https://github.com/cilium/cilium/blob/main/Documentation/security/network/encryption-wireguard.rst
ingress_cilium_wireguard = {
description = "Allow Cilium Wireguard node to node"
protocol = "udp"
from_port = 51871
to_port = 51871
type = "ingress"
self = true
}
}

tags = local.tags
}

################################################################################
# Kubectl Output
################################################################################

output "configure_kubectl" {
description = "Configure kubectl: make sure you're logged in with the correct AWS profile and run the following command to update your kubeconfig"
value = "aws eks --region ${local.region} update-kubeconfig --name ${module.eks.cluster_name}"
}

################################################################################
# EKS Blueprints Addons
################################################################################

module "eks_blueprints_addons" {
source = "aws-ia/eks-blueprints-addons/aws"
version = "~> 1.7"

cluster_name = module.eks.cluster_name
cluster_endpoint = module.eks.cluster_endpoint
cluster_version = module.eks.cluster_version
oidc_provider_arn = module.eks.oidc_provider_arn

helm_releases = {
cilium = {
name = "cilium"
chart = "cilium"
version = "1.14.1"
repository = "https://helm.cilium.io/"
description = "Cilium Add-on"
namespace = "kube-system"
create_namespace = false

values = [
<<-EOT
cni:
chainingMode: aws-cni
enableIPv4Masquerade: false
tunnel: disabled
endpointRoutes:
enabled: true
l7Proxy: false
encryption:
enabled: true
type: wireguard
EOT
]
}
}

tags = local.tags
}
Loading

0 comments on commit edb8492

Please sign in to comment.