Helm chart redesign for Quarkus-based runtimes #626

adutra · 2025-01-08T13:58:54Z

This is a major redesign of the Helm chart to make it compliant with Quarkus, but also improving a lot of aspects of the existing chart, such as the persistence secret management and the bootstrap jobs, to name a few.

Summary of changes

Persistence

Thanks to the improvement brought by #613, it's not necessary anymore to have an init container create a conf.jar. From now on, the user provides a secret with their persistence.xml, and that file is mounted on each Polaris pod directly.

Enhanced services

The services for Polaris now are separated in 3 categories:

Main service: the service that serves API requests; it is usually behind an Ingress or LoadBalancer
Management service: the service for metrics and health; this is usually ClusterIP and headless
Extra services: if any of the services needs to be exposed with different settings to different consumers, this section can be used.

Each service section has the same structure and allows to configure one or more ports.

Observability

New sections were added for logging, tracing, metrics, with also the ability to create a ServiceMonitor.

Bootstrap Job

The bootstrap section now configures the bootstrap job. Thanks to the Polaris Admin Tool introduced in #605, the jobs now use the tool to bootstrap realms.

I am not convinced personally that this bootstrap job has a huge value, compared to just running the admin tool directly. But I didn't want to remove it.

Advanced Config

A new advancedConfig section can be used to customize Polaris deployments in any possible way, even when the Helm chart does not expose the desired setting.

adutra · 2025-01-24T16:36:49Z

helm/polaris/README.md

 ```

 ### Uninstalling the chart

 ```bash
-$ helm uninstall --namespace polaris polaris
+helm uninstall --namespace polaris polaris
 ```

 ## Values


Note: this is automatically generated by helm-docs.

adutra · 2025-01-24T16:52:33Z

This is now ready for review.

@MonkeyCanCode I would love to get your review on this please :-)

I know we could add more unit tests, but I wanted to get your general feeling first. We can complete test coverage little by little imho.

MonkeyCanCode · 2025-01-24T16:57:37Z

This is now ready for review.

@MonkeyCanCode I would love to get your review on this please :-)

I know we could add more unit tests, but I wanted to get your general feeling first. We can complete test coverage little by little imho.

Great. Nice work. I will review those later this weekend

adutra · 2025-01-24T20:24:03Z

helm/polaris/values.yaml

+  # -- The root credentials to create during the bootstrap. If you don't provide credentials for the
+  # root principal of each realm to bootstrap, random credentials will be generated.
+  # Each entry in the array must be of the form: realm,clientId,clientSecret
+  credentials: []


This is admittedly not great. I opened #878 to improve this and use secrets instead. But for now we need to stick with credentials in clear text.

MonkeyCanCode · 2025-01-26T05:11:43Z

helm/polaris/templates/_helpers.tpl

+{{- if has $portNumber (values $ports) -}}
+{{- fail (printf "service.ports[%d]: port number already taken: %v" $i $portNumber) -}}
+{{- end -}}
+{{- $_ := set $ports $port.name $portNumber }}


nits: lines 261-264 and 276-278, may want to add 2 more space for each of them to fix indentation (still valid yaml files)

MonkeyCanCode · 2025-01-26T05:33:53Z

helm/polaris/templates/tests/test-connection.yaml

-      args: ['{{ include "polaris.fullname" . }}:{{ index .Values.service.ports "polaris-metrics" }}/q/health']
+      args:
+        - --spider
+        - '{{ include "polaris.fullnameWithSuffix" (list . "mgmt") }}:{{ get (first .Values.managementService.ports) "port" }}/q/health/ready'
  restartPolicy: Never


Seem likes now the pod will take some time to start up. From my local testing, this will always fail. Should we consider add an init container to sleep for X seconds before run the connection test? Here is a sample change:

initContainers: - name: sleep image: busybox command: ['sleep', '30']

Hmm it wasn't failing for me, the test pod always comes up after the Polaris pod reaches ready state.

I don't mind changing but waiting 30 seconds seems a lot, how about we poll the health endpoint X times until it succeeds?

Yes, that will be good. Yeah, on my setup it was taking 6-7 seconds somehow. But yeah, good to add some retry etc.

MonkeyCanCode · 2025-01-26T06:00:50Z

helm/polaris/templates/_helpers.tpl

+          name: {{ tpl .Values.authentication.tokenBroker.secret.name . }}
+          items:
+          {{- if eq .Values.authentication.tokenBroker.type "rsa-key-pair" }}
+          - key: {{ tpl .Values.authentication.tokenBroker.secret.publicKey . }}


nits: lines 173-175 and 179-180, may want to add 2 more space for each of them to fix indentation (still valid yaml files)

MonkeyCanCode · 2025-01-26T06:07:03Z

.github/workflows/helm.yml

@@ -99,7 +99,7 @@ jobs:
        if: steps.list-changed.outputs.changed == 'true'
        run: |
          eval $(minikube -p minikube docker-env)
-          ./gradlew :polaris-quarkus-server:assemble \
+          ./gradlew :polaris-quarkus-server:assemble :polaris-quarkus-admin:assemble \
            -Dquarkus.container-image.build=true \
            -PeclipseLinkDeps=com.h2database:h2:2.3.232


While testing the helm, I noticed the bootstrap pod will shows the bootstrap is completed. However, when trying to use the polaris CLI against the server, it will say the instance is not yet bootstrapped. Not sure if this is known issue?

Are you using in-memory persistence? Because with in-memory it won't work indeed. The bootstrap job can only be used with EclipseLink.

Also, H2 will not work either because it's in-memory.

@MonkeyCanCode I think we should switch our tests to use Postgres, wdyt? H2 won't give us a "real" experience and bootstrapping will be broken.

I just tested with postgres and the bootstrap job worked and I was able to connect to Polaris after.

The only annoying thing is that we will need support for purge as well. I will try to add that too.

Yes, I was using this branch for testing and bootstrap is probably hitting in-memory persistency for h2database i think? I also agreed we should use somewhat real db people may be using (e.g. Postgres) instead of using h2database (kinds left people to think this project is not close to ready to be use).

The latest main branch seems to be pretty stable (I only did some basic testing...planning to do more later this weekend). So maybe good to introduce those changes now?

is probably hitting in-memory persistency for h2database i think?

Yup it was using H2 in-memory.

So maybe good to introduce those changes now?

I just pushed a commit that switches our tests to postgres. It's working well in CI.

When doing local testing though, if you install the chart more than once, you need to either kill the postgres pod or --set bootstrap.enabled=false to avoid an error when bootstrapping the realm, since it was already bootstrapped previously.

MonkeyCanCode · 2025-01-27T02:12:45Z

helm/polaris/templates/deployment.yaml

          env:
+            {{- if .Values.storage.secret.name }}


Maybe use a helper function for secret to env?

MonkeyCanCode · 2025-01-27T02:49:06Z

helm/polaris/README.md

+If necessary, load the Docker images into Minikube:
+
+```bash
+eval $(minikube -p minikube docker-env)


nits: I used kind a lot more as oppose to minikube (both are great...kind is very lightwight and super fast). I am okay with switching to minikube, we may want to update https://github.com/apache/polaris/blob/main/run.sh as well to match the same.

Ah that's true. Well, let's mention both! I will add a note about Kind.

adutra force-pushed the quarkus-helm branch 4 times, most recently from 09657d1 to f66d623 Compare January 8, 2025 17:21

adutra mentioned this pull request Jan 8, 2025

Helm: add securityContext to initContainers #602

Merged

adutra force-pushed the quarkus-helm branch 13 times, most recently from 0ac047b to 434a847 Compare January 13, 2025 21:36

This was referenced Jan 14, 2025

Refactor RealmContext to RealmId #741

Merged

Build viable Docker images for Polaris using Quarkus #610

Merged

adutra force-pushed the quarkus-helm branch 3 times, most recently from 68cbdfc to acee0c9 Compare January 15, 2025 09:50

adutra force-pushed the quarkus-helm branch from acee0c9 to 74f5669 Compare January 24, 2025 16:33

adutra changed the title ~~[WIP] Helm chart redesign for Quarkus-based runtimes~~ Helm chart redesign for Quarkus-based runtimes Jan 24, 2025

adutra commented Jan 24, 2025

View reviewed changes

adutra force-pushed the quarkus-helm branch from 74f5669 to d5014bb Compare January 24, 2025 16:43

adutra marked this pull request as ready for review January 24, 2025 16:51

adutra requested review from ashvina and dennishuo as code owners January 24, 2025 16:51

adutra requested review from snazy, RussellSpitzer, takidau, flyrain and ebyhr as code owners January 24, 2025 16:51

adutra force-pushed the quarkus-helm branch 4 times, most recently from 67bf183 to 69c1265 Compare January 24, 2025 18:38

adutra commented Jan 24, 2025

View reviewed changes

MonkeyCanCode reviewed Jan 27, 2025

View reviewed changes

adutra force-pushed the quarkus-helm branch from 69c1265 to dff46d1 Compare January 27, 2025 10:51

adutra added 10 commits January 28, 2025 14:41

Helm chart redesign for Quarkus-based runtimes

9b92ce1

review

d5bfffc

secretToEnv

8240926

mention Kind in README.md

2d21f4b

suppress annoying output

03aaf2d

switch tests to postgres

010adca

nit

3bd7f86

add license header

7b787c2

purge false by default

35ec47d

tpl realm

36faea7

adutra force-pushed the quarkus-helm branch from d8cfe58 to 36faea7 Compare January 28, 2025 13:41

fix extra services template

75d5c0c

adutra force-pushed the quarkus-helm branch from 1d50986 to 75d5c0c Compare January 29, 2025 14:39

adutra added 3 commits January 29, 2025 15:53

fix polaris.containerPorts

93c74d7

fix polaris.secretToEnv

5c6231d

fix ServiceMonitor

6e88142

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Helm chart redesign for Quarkus-based runtimes #626

Helm chart redesign for Quarkus-based runtimes #626

adutra commented Jan 8, 2025 •

edited

Loading

adutra Jan 24, 2025

adutra commented Jan 24, 2025

MonkeyCanCode commented Jan 24, 2025

adutra Jan 24, 2025

MonkeyCanCode Jan 26, 2025

MonkeyCanCode Jan 26, 2025

adutra Jan 27, 2025

MonkeyCanCode Jan 27, 2025

MonkeyCanCode Jan 26, 2025

MonkeyCanCode Jan 26, 2025

adutra Jan 27, 2025

adutra Jan 27, 2025

adutra Jan 27, 2025

MonkeyCanCode Jan 27, 2025

adutra Jan 27, 2025

MonkeyCanCode Jan 27, 2025

MonkeyCanCode Jan 27, 2025

adutra Jan 27, 2025

Helm chart redesign for Quarkus-based runtimes #626

Are you sure you want to change the base?

Helm chart redesign for Quarkus-based runtimes #626

Conversation

adutra commented Jan 8, 2025 • edited Loading

Summary of changes

Persistence

Enhanced services

Observability

Bootstrap Job

Advanced Config

Choose a reason for hiding this comment

adutra commented Jan 24, 2025

MonkeyCanCode commented Jan 24, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adutra commented Jan 8, 2025 •

edited

Loading