This repository contains a quick demo on secrets management with Bitnami Sealed Secrets in a Flux workflow.
To follow this tutorial, you need:
- FluxCLI
>= 0.14
(installation guide) - Minikube
>= v1.20
(installation guide) - Kubeseal
=v0.16.0
(installation guide) - GitHub account
Create a GitHub personal access token following this guide. Remember to grant all repo
permissions.
Once created, export the following environment variables:
export GITHUB_TOKEN=<TOKEN_ID>
export GITHUB_USER=<MY_GITHUB_USER> (e.g. nikever)
At the root level of this repository, execute:
export REPO_DIR=$(PWD)
Start a minikube cluster:
cd $REPO_DIR/minikube
make setup
By default, the command spins up a one node Kubernetes cluster of version v1.19.4
in a VirtualBox VM (CPUs=2, Memory=4096MB, Disk=20000MB).
You can pass additional parameters to change the default values. Please referer to this Makefile for further details on the cluster creation.
- Open a terminal and bootstrap Flux with
fluxcli
:
flux bootstrap github \
--owner $GITHUB_USER \
--repository demo-flux-secrets-fleet \
--branch main \
--path ./minikube-cluster \
--personal
The bootstrap
command:
- creates a
demo-flux-secrets-fleet
repository - generates Flux components manifests
- commits Flux components manifests to the
main
branch - installs the Flux components in the
flux-system
namespace - configures the target cluster to synchronize with the
./minikube-cluster
path inside the repository
- While you wait for the commands to complete, you can verify that the pods in the
flux-system
are becoming running:
kubectl get pods -n flux-system -w
- Clone the
demo-flux-secrets-fleet
repository
git clone https://github.com/$GITHUB_USER/demo-flux-secrets-fleet
cd demo-flux-secrets-fleet/
- Create a
hello-secret
folder inside theminikube-cluster
folder:
mkdir hello-secret
- Create a Flux
GitRepository
manifest pointing to the Hello Secret repository's master branch:
flux create source git hello-secret \
--url https://github.com/nikever/kubernetes-hello-secret \
--branch main \
--interval 1m \
--export \
> ./hello-secret/hello-secret-source.yaml
- Create a Flux
Kustomization
manifest to build and apply the kustomize directory located in the Hello Secret repository under themanifests
folder.
flux create kustomization hello-secret \
--source=hello-secret \
--path="./manifests" \
--prune=true \
--interval=1m \
--export \
> ./hello-secret/hello-secret-kustomization.yaml
- Deploy via GitOps
git add -A && git commit -m "Deploy Hello Secret App"
git push
- Wait for Flux to reconcile everything:
watch flux get sources git
watch flux get kustomizations
You can force the reconciliation with
flux reconcile source git flux-system
- Check the resources deployed:
kubectl get all -n hello
Output:
NAME READY STATUS RESTARTS AGE
pod/hello-app-6cb6ddd95f-2chmq 0/1 CreateContainerConfigError 0 54s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/hello-svc NodePort 10.103.216.81 <none> 8080:30964/TCP 54s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/hello-app 0/1 1 0 54s
NAME DESIRED CURRENT READY AGE
replicaset.apps/hello-app-6cb6ddd95f 1 1 0 54s
As you can see, the pod pod/hello-app-6cb6ddd95f-2chmq
is in the CreateContainerConfigError
status. To become running, the pod needs a hello-secret
secret, which is currently not deployed:
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Failed 43s (x8 over 2m6s) kubelet Error: secret "hello-secret" not found
- Create a
sealed-secrets
folder inside theminikube-cluster
folder:
mkdir sealed-secrets
- Create a
HelmRepository
manifest
flux create source helm sealed-secrets \
--url https://bitnami-labs.github.io/sealed-secrets \
--interval 1h \
--export \
> ./sealed-secrets/sealed-secrets-source.yaml
- Create Helm release manifest:
flux create helmrelease sealed-secrets \
--interval=1h \
--release-name=sealed-secrets \
--target-namespace=flux-system \
--source=HelmRepository/sealed-secrets \
--chart=sealed-secrets \
--chart-version=">=1.15.0-0" \
--crds=CreateReplace \
--export \
> ./sealed-secrets/sealed-secrets-release.yaml
- Deploy via GitOps:
git add -A && git commit -m "Deploy Bitnami sealed secrets"
git push
- Wait for Flux to reconcile everything:
watch flux get sources git
You can force the reconciliation with
flux reconcile source git flux-system
- Check the resources deployed:
kubectl get pods -n flux-system | grep sealed-secrets
At startup, the sealed-secrets controller generates a 4096-bit RSA key pair and persists the private and public keys as Kubernetes secrets in the flux-system
namespace.
- Retrieve the public key with:
kubeseal --fetch-cert \
--controller-name=sealed-secrets \
--controller-namespace=flux-system \
> pub-sealed-secrets.pem
The public key can be safely stored in Git and can be used to encrypt secrets without direct access to the Kubernetes cluster.
- Create the secret manifest:
kubectl create secret generic -n hello hello-secret \
--from-literal=secret=1234 \
--dry-run=client \
-o yaml > hello-secret.yaml
- Seal the secret using
kubeseal
:
kubeseal --format=yaml --cert=pub-sealed-secrets.pem \
< hello-secret.yaml \
> hello-secret-sealed.yaml
- Remove the plain secret and commit the sealed one to deploy it:
rm hello-secret.yaml
git add hello-secret-sealed.yaml && git commit -m "Add sealed hello-secret"
git push
- Wait for the
SealedSecret
to be deployed:
kubectl get SealedSecret -n hello -w
You can force the reconciliation with
flux reconcile source git flux-system
The hello-secret
app contains a containerized Go web server application that responds to all HTTP requests with the value of the SECRET
environment variable.
- Restart the
hello-app
pod:
kubectl delete pod -n hello $(kubectl get pods -n hello -o=jsonpath='{.items..metadata.name}')
- Check that now is running:
kubectl get pods -n hello
Output:
NAME READY STATUS RESTARTS AGE
hello-app-6cb6ddd95f-558ts 1/1 Running 0 36s
- Check that the
hello-secret
app is finally working:
curl -H "Host: hello-secret.info" $(minikube ip)
Output:
Hello, secret!
Secret: 1234
- Delete the
minikube
cluster:
cd $REPO_DIR/minikube
make delete
- Delete the
demo-flux-secrets-fleet
repository (optional).