Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the Pulumi Kubernetes Operator #141

Merged
merged 8 commits into from
Sep 21, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions eks-anywhere-common/Addons/Partner/Pulumi/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Pulumi

This folder contains Kubernetes manifest that install the [Pulumi Kubernetes Operator](https://www.pulumi.com/docs/using-pulumi/continuous-delivery/pulumi-kubernetes-operator/) along with resources necessary to run the automated tests under `../Testers/Pulumi` in this repository.
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
apiVersion: v1
kind: Namespace
metadata:
name: pulumi
labels:
aws.conformance.vendor: pulumi
aws.conformance.vendor-solution: pulumi-kubernetes-operator
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# TODO
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
apiVersion: v1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this configmap is for tester, this should be moved to Testers/Pulumi folder.

kind: ConfigMap
metadata:
name: pulumi-tester
namespace: pulumi
data:
pulumi-org: aws-partnership
10 changes: 10 additions & 0 deletions eks-anywhere-common/Addons/Partner/Pulumi/pulumi-tester-role.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
apiVersion: rbac.authorization.k8s.io/v1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this configmap is for tester, this should be moved to Testers/Pulumi folder.

kind: Role
metadata:
name: pulumi-tester-role
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it atypical for an object name to have its type as a suffix. In other words, I would name this pulumi-tester. Likewise for the binding.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. I do not like smurf naming.

namespace: pulumi
rules:
- apiGroups: ["pulumi.com"]
resources: ["stacks", "programs"]
verbs: ["*"]
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
apiVersion: rbac.authorization.k8s.io/v1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this configmap is for tester, this should be moved to Testers/Pulumi folder.

kind: RoleBinding
metadata:
name: pulumi-tester-rolebinding
namespace: pulumi
subjects:
- kind: ServiceAccount
name: pulumi-tester
roleRef:
kind: Role
name: pulumi-tester-role
apiGroup: rbac.authorization.k8s.io
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this configmap is for tester, this should be moved to Testers/Pulumi folder.

apiVersion: v1
kind: ServiceAccount
metadata:
name: pulumi-tester
namespace: pulumi
169 changes: 169 additions & 0 deletions eks-anywhere-common/Testers/Pulumi/pulumi-tester-job.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: pulumi-tester-configmap

This comment was marked as resolved.

namespace: pulumi
data:
pulumi-k8s-operator-test.sh: |-
#!/bin/bash
set -e

if ! which curl ; then
echo "curl not found. Installing curl."
install_packages curl
fi

if ! which kubectl ; then
echo "kubectl not found. Installing kubectl"
apt-get update && \
apt-get install -y --no-install-recommends ca-certificates && \
update-ca-certificates
apt-get update && \
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" && \
chmod +x kubectl && \
mv kubectl /usr/local/bin/
fi

RANDOM_SUFFIX=$(date +%s)-$RANDOM
MANIFEST_FILENAME=/tmp/pulumi-test-stack-${RANDOM_SUFFIX}.yaml

# Create a new, unique stack name. The name of the stack must be unique for each
# run for 2 reasons:
# 1. To ensure that each test run on the current platform starts from a known
# state, free from any previously test runs.
# 2. To ensure that we don't get colliding updates as AWS runs the test
# simultaneously on different EKS-A platforms.
TEST_PULUMI_STACK_NAME=test-${RANDOM_SUFFIX}

STACKPATH=${PULUMI_ORG}/eks-pulumi-operator-test/${TEST_PULUMI_STACK_NAME}

echo ""
echo "Writing out test manifest file '${MANIFEST_FILENAME}'"
# Note that while we use a random Pulumi stack name within the Stack resource,
# we keep the name of the Kubernetes Stack and Program resources static. This is
# intentional. If a test fails, we don't want it to leave superfluous Stack and
# Program K8s resources behind because the operator will keep trying to
# reconcile them. (The test failed - they should never be re-run again.)
# Instead, we want to reuse the same Kubernetes resource over and over but have
# it generate a new, uniquely-named Pulumi stack.
cat << EOF > $MANIFEST_FILENAME
apiVersion: pulumi.com/v1
kind: Program
metadata:
name: eks-pulumi-operator-test
namespace: pulumi
program:
resources:
myRandomPet:
type: random:RandomPet
outputs:
petName: \${myRandomPet.id}
---
apiVersion: pulumi.com/v1
kind: Stack
metadata:
name: eks-pulumi-operator-test
namespace: pulumi
spec:
stack: ${STACKPATH}
programRef:
name: eks-pulumi-operator-test
destroyOnFinalize: true
envRefs:
PULUMI_ACCESS_TOKEN:
type: Secret
secret:
name: pulumi-access-token
key: value
EOF


echo ""
echo "Deploying sample stack and program."
kubectl apply -f $MANIFEST_FILENAME

echo ""
echo "Waiting for the operator to deploy the stack."
sleep 10

echo ""
echo "Verifying that the stack exists."
curl \
--fail \
-H "Accept: application/vnd.pulumi+8" \
-H "Content-Type: application/json" \
-H "Authorization: token $PULUMI_ACCESS_TOKEN" \
https://api.pulumi.com/api/stacks/${STACKPATH}

echo ""
echo "Destroying K8s Stack resource"
kubectl delete -n pulumi stacks/eks-pulumi-operator-test

echo ""
echo "Waiting for the operator to remove the stack"
sleep 10

echo ""
echo "Verifying the stack no longer exists"
STATUSCODE=$(curl \
-s \
-o /dev/null \
--w "%{http_code}" \
-H "Accept: application/vnd.pulumi+8" \
-H "Content-Type: application/json" \
-H "Authorization: token $PULUMI_ACCESS_TOKEN" \
https://api.pulumi.com/api/stacks/${STACKPATH}
)

if test $STATUSCODE -ne 404; then
echo "ERROR: Expected HTTP status code 404 from the Pulumi Cloud API when querying the stack. Got HTTP status code $STATUSCODE instead."
false
fi

# This is for purely for running the script locally. Since the K8s tester Job is
# run in an ephemeral container, deleting the file is unnecessary in that
# context:
echo ""
echo "Deleting test manifest file '${MANIFEST_FILENAME}'"
rm ${MANIFEST_FILENAME}
---
apiVersion: batch/v1
kind: Job
jkodroff marked this conversation as resolved.
Show resolved Hide resolved
metadata:
# labels:
# app: script-job
name: pulumi-k8s-operator-test
namespace: pulumi
spec:
backoffLimit: 1
template:
spec:
serviceAccountName: pulumi-tester
containers:
- command:
- bash
- /scripts/pulumi-k8s-operator-test.sh
image: "bitnami/minideb:bookworm"
name: script
env:
- name: PULUMI_ACCESS_TOKEN
valueFrom:
secretKeyRef:
name: pulumi-access-token
key: value
- name: PULUMI_ORG
valueFrom:
configMapKeyRef:
name: pulumi-tester
key: pulumi-org
volumeMounts:
- name: pulumi-tester-configmap
mountPath: /scripts/pulumi-k8s-operator-test.sh
subPath: pulumi-k8s-operator-test.sh
readOnly: false
restartPolicy: Never
volumes:
- name: pulumi-tester-configmap
configMap:
name: pulumi-tester-configmap
defaultMode: 0777