Add the Pulumi Kubernetes Operator #141

jkodroff · 2023-09-19T18:51:11Z

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

elamaran11

@jkodroff Thankyou for a very quick work on the PR. Couple of important things :

You are missing an External Secret resource to reference your secret. here is an example.
We need a CronJob which runs in schedule vs a Job. Check this for an example.

eks-anywhere-common/Testers/Pulumi/pulumi-tester-job.yaml

EronWright · 2023-09-19T20:40:55Z

eks-anywhere-common/Addons/Partner/Pulumi/pulumi-operator-helmrepository.yaml

+apiVersion: source.toolkit.fluxcd.io/v1beta2
+kind: HelmRepository
+metadata:
+  name: pulumi-kubernetes-operator


Consider naming this pulumi rather than pulumi-kubernetes-operator since, in concept, it could serve other charts too.

I made this change.

EronWright · 2023-09-19T20:41:53Z

eks-anywhere-common/Addons/Partner/Pulumi/pulumi-tester-role.yaml

+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name: pulumi-tester-role


I think it atypical for an object name to have its type as a suffix. In other words, I would name this pulumi-tester. Likewise for the binding.

Good call. I do not like smurf naming.

eks-anywhere-common/Testers/Pulumi/pulumi-tester-job.yaml

+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: pulumi-tester-configmap


jkodroff · 2023-09-19T20:58:33Z

@elamaran11 Did my best to get the ExternalSecret right, but won't have time to test. It should should require a minor tweak at worst.

elamaran11 · 2023-09-19T21:25:40Z

@jkodroff Who will fix the above changes pointed by @EronWright. let me know when done i can test it.

elamaran11

@jkodroff One comment on the token reference.

elamaran11 · 2023-09-19T21:27:00Z

eks-anywhere-common/Testers/Pulumi/pulumi-tester-cronjob.yaml

+                  valueFrom:
+                    secretKeyRef:
+                      name: pulumi-access-token
+                      key: value


@jkodroff This doesnt seem to match with your ExternalSecret k8s rsource. If the key is pulumi-access-token, it should match.

I believe this is now fixed.

I would recommend giving a meaningful name like accessToken for key.

elamaran11

Minor feedback but i can work on the testing tomorrow.

elamaran11 · 2023-09-19T22:25:57Z

eks-anywhere-common/Testers/Pulumi/.gitignore

@@ -0,0 +1 @@
+kubeconfig.yml


Please remove this file.

Do you mean the .gitignore file? What is the rationale?

Why do we need this .gitignore file. We have one at root.

EronWright · 2023-09-20T18:02:35Z

eks-anywhere-common/Addons/Partner/Pulumi/pulumi-operator-helmrelease.yaml

+  targetNamespace: pulumi
+  chart:
+    spec:
+      chart: pulumi-kubernetes-operator


Should we specify a chart version, e.g. 0.3.0?

yes thats mandatory, nice catch

Did you test the helm release did it work?

I fixed this to use 0.3.*, does that seem right @elamaran11 ?

Where is the change? I dont see the version.

elamaran11

@jkodroff @EronWright Have some feedback on the PR which needs to be fixed.

ExternalSecret KeyName and SecretRef is not correct so the installation fails. Also having meaningful name for secretKey
Secret Reference in cronjob should have a meaningful name as aligned to comment 1.

elamaran11 · 2023-09-20T20:00:42Z

eks-anywhere-common/Addons/Partner/Pulumi/pulumi-access-token-externalsecret.yaml

+  data:
+    - secretKey: pulumi-access-token # which key it's going to be stored
+      remoteRef:
+        key: value # Our secret-name goes here


I would rather have a specific name for the KVP than value

secretKey should be a something reasonable like accessToken and key should the keyname in secrets manager which is pulumi-access-token

elamaran11 · 2023-09-20T20:00:55Z

eks-anywhere-common/Addons/Partner/Pulumi/pulumi-operator-helmrelease.yaml

+  targetNamespace: pulumi
+  chart:
+    spec:
+      chart: pulumi-kubernetes-operator


Did you test the helm release did it work?

elamaran11 · 2023-09-20T20:01:39Z

eks-anywhere-common/Addons/Partner/Pulumi/pulumi-tester-configmap.yaml

@@ -0,0 +1,7 @@
+apiVersion: v1


If this configmap is for tester, this should be moved to Testers/Pulumi folder.

elamaran11 · 2023-09-20T20:01:46Z

eks-anywhere-common/Addons/Partner/Pulumi/pulumi-tester-role.yaml

@@ -0,0 +1,10 @@
+---
+apiVersion: rbac.authorization.k8s.io/v1


If this configmap is for tester, this should be moved to Testers/Pulumi folder.

elamaran11 · 2023-09-20T20:01:53Z

eks-anywhere-common/Addons/Partner/Pulumi/pulumi-tester-rolebinding.yaml

@@ -0,0 +1,13 @@
+---
+apiVersion: rbac.authorization.k8s.io/v1


If this configmap is for tester, this should be moved to Testers/Pulumi folder.

elamaran11 · 2023-09-20T20:01:58Z

eks-anywhere-common/Addons/Partner/Pulumi/pulumi-tester-serviceaaccount.yaml

@@ -0,0 +1,6 @@
+---


If this configmap is for tester, this should be moved to Testers/Pulumi folder.

elamaran11 · 2023-09-20T20:02:07Z

eks-anywhere-common/Addons/Partner/Pulumi/.gitignore

@@ -0,0 +1 @@
+kubeconfig.yml


remove this file

Use alpine:k8s:1.26.2 image. Use kubectl wait rather than sleep.

elamaran11

@jkodroff @EronWright I would recommend making all changes listed in the PR and also test it locally once along with sharing a screen shot on the test job working locally. Once you have it, i can proceed with testing. At this point the testing is blocked due to errors with test job secrets.

elamaran11 · 2023-09-21T00:45:18Z

eks-anywhere-common/Addons/Partner/Pulumi/pulumi-operator-helmrelease.yaml

+  targetNamespace: pulumi
+  chart:
+    spec:
+      chart: pulumi-kubernetes-operator


Where is the change? I dont see the version.

EronWright · 2023-09-21T00:57:39Z

@elamaran11 Thanks for your patience, I encountered a small delay in pushing the changes, due to a permissions issue on the fork. I believe I've addressed the feedback; of course we'll discuss further once you can see them.

Meanwhile, here's the local test output (tested using EKS):

pulumi-k8s-operator-test-28254296-wd2c4 script 
pulumi-k8s-operator-test-28254296-wd2c4 script Writing out test manifest file '/tmp/pulumi-test-stack-1695257760-31679.yaml'
pulumi-k8s-operator-test-28254296-wd2c4 script 
pulumi-k8s-operator-test-28254296-wd2c4 script Deploying sample stack and program.
pulumi-k8s-operator-test-28254296-wd2c4 script program.pulumi.com/eks-pulumi-operator-test unchanged
pulumi-k8s-operator-test-28254296-wd2c4 script stack.pulumi.com/eks-pulumi-operator-test created
pulumi-k8s-operator-test-28254296-wd2c4 script 
pulumi-k8s-operator-test-28254296-wd2c4 script Waiting for the operator to deploy the stack.
pulumi-k8s-operator-test-28254296-wd2c4 script stack.pulumi.com/eks-pulumi-operator-test condition met
pulumi-k8s-operator-test-28254296-wd2c4 script 
pulumi-k8s-operator-test-28254296-wd2c4 script Verifying that the stack exists.
pulumi-k8s-operator-test-28254296-wd2c4 script {"orgName":"eron-pulumi-corp","projectName":"eks-pulumi-operator-test","stackName":"test-1695257760-31679","activeUpdate":"687b45a4-2d70-47b6-8031-3073fb35590e","tags":{"gitHub:owner":"eron-pulumi-corp","pulumi:project":"eks-pulumi-operator-test","pulumi:runtime":"yaml","pulumi:secrets_provider":"service"},"version":1}
pulumi-k8s-operator-test-28254296-wd2c4 script 
pulumi-k8s-operator-test-28254296-wd2c4 script Destroying K8s Stack resource
pulumi-k8s-operator-test-28254296-wd2c4 script stack.pulumi.com "eks-pulumi-operator-test" deleted
pulumi-k8s-operator-test-28254296-wd2c4 script 
pulumi-k8s-operator-test-28254296-wd2c4 script Waiting for the operator to remove the stack
pulumi-k8s-operator-test-28254296-wd2c4 script 
pulumi-k8s-operator-test-28254296-wd2c4 script Verifying the stack no longer exists
pulumi-k8s-operator-test-28254296-wd2c4 script 
pulumi-k8s-operator-test-28254296-wd2c4 script Deleting test manifest file '/tmp/pulumi-test-stack-1695257760-31679.yaml'

EronWright · 2023-09-21T01:50:12Z

@elamaran11 the changes have been pushed, please take another look.

elamaran11

@EronWright Code changes look good for both the helm deployment and the test job. I will start to test this in my lab today. Thankyou

elamaran11 · 2023-09-21T13:39:29Z

Hi @EronWright I see the stack is created and destroyed and test job concluded successfully. This is good news. I can say my test is concluded in EKS-A Baremetal environment. I will continue my tests in other environment. jc though, i dont see the stack in the Pulumi console is it because of instant create and delete?

❯ k logs pulumi-k8s-operator-test-002-df9kq -n pulumi                                                             ─╯

Writing out test manifest file '/tmp/pulumi-test-stack-1695303142-10787.yaml'

Deploying sample stack and program.
program.pulumi.com/eks-pulumi-operator-test unchanged
stack.pulumi.com/eks-pulumi-operator-test created

Waiting for the operator to deploy the stack.
stack.pulumi.com/eks-pulumi-operator-test condition met

Verifying that the stack exists.
{"orgName":"aws-partnership","projectName":"eks-pulumi-operator-test","stackName":"test-1695303142-10787","activeUpdate":"b9968d89-db2f-4634-b3c2-68a3da3f55a9","tags":{"gitHub:owner":"aws-partnership","pulumi:project":"eks-pulumi-operator-test","pulumi:runtime":"yaml","pulumi:secrets_provider":"service"},"version":1}

Destroying K8s Stack resource
stack.pulumi.com "eks-pulumi-operator-test" deleted

Waiting for the operator to remove the stack

Verifying the stack no longer exists

Deleting test manifest file '/tmp/pulumi-test-stack-1695303142-10787.yaml'

elamaran11 · 2023-09-21T14:21:26Z

@EronWright I got to understand its instant and also i saw the same in the console for quick few seconds. The product is validated across EKS-A BareMetal, vSphere and EKS Local cluster. EKS-A Snowball is still pending, i will complete this testing today or tomorrow when i get to the lab.

elamaran11

Everything works fine. Congratulations and Thankyou @EronWright @jkodroff . Please continue to maintain your software for version upgrades via PRs.

Initial pass - still need to add Helm resources.

3e42ad2

jkodroff marked this pull request as draft September 19, 2023 18:51

elamaran11 requested changes Sep 19, 2023

View reviewed changes

eks-anywhere-common/Testers/Pulumi/pulumi-tester-job.yaml Outdated Show resolved Hide resolved

jkodroff added 3 commits September 19, 2023 16:36

Fix up Flux/Helm resources.

7c56b82

Remove erroneous tgz file.

dd7c44c

Turn tester Job to CronJob.

0890ec6

EronWright reviewed Sep 19, 2023

View reviewed changes

Add external secret.

5399b8c

jkodroff marked this pull request as ready for review September 19, 2023 20:57

elamaran11 requested changes Sep 19, 2023

View reviewed changes

Correct external secret.

46f58f3

elamaran11 reviewed Sep 19, 2023

View reviewed changes

EronWright reviewed Sep 20, 2023

View reviewed changes

elamaran11 requested changes Sep 20, 2023

View reviewed changes

EronWright added 2 commits September 20, 2023 17:35

Rename HelmRepository

bfe22e2

Encapsulate the tester resources.

24f124b

Use alpine:k8s:1.26.2 image. Use kubectl wait rather than sleep.

elamaran11 requested changes Sep 21, 2023

View reviewed changes

elamaran11 reviewed Sep 21, 2023

View reviewed changes

elamaran11 approved these changes Sep 21, 2023

View reviewed changes

elamaran11 merged commit b6d522c into aws-samples:main Sep 21, 2023

EronWright deleted the pulumi-k8s-operator branch September 22, 2023 17:04

		@@ -0,0 +1 @@
		kubeconfig.yml

		@@ -0,0 +1 @@
		kubeconfig.yml

Add the Pulumi Kubernetes Operator #141

Add the Pulumi Kubernetes Operator #141

Conversation

jkodroff commented Sep 19, 2023

elamaran11 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as resolved.

jkodroff commented Sep 19, 2023

elamaran11 commented Sep 19, 2023

elamaran11 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elamaran11 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

EronWright Sep 20, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elamaran11 left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elamaran11 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

EronWright commented Sep 21, 2023

EronWright commented Sep 21, 2023

elamaran11 left a comment

Choose a reason for hiding this comment

elamaran11 commented Sep 21, 2023

elamaran11 commented Sep 21, 2023

elamaran11 left a comment

Choose a reason for hiding this comment

EronWright Sep 20, 2023 •

edited

Loading

elamaran11 left a comment •

edited

Loading