charts,salt,tests: Add podAntiAffinity support for Dex

This commit add ability to configure `podAntiAffinity` for Dex from CSC.

Patche the Dex helm chart to add support for `strategy` on Dex
deployment, as the default one does not make sense for our Dex
deployment
See dexidp/helm-charts#66

Render chart to salt state using
```
./charts/render.py dex charts/dex.yaml charts/dex \
  --namespace metalk8s-auth \
  --service-config dex metalk8s-dex-config \
  metalk8s/addons/dex/config/dex.yaml.j2 metalk8s-auth \
  > salt/metalk8s/addons/dex/deployed/chart.sls
```

See: #3574

CHANGELOG.md

@@ -12,6 +12,11 @@
   hostname, so that if it's possible each `CoreDNS` pods will sit on different infra node
   (PR[#3579](https://github.com/scality/metalk8s/pull/3579))
 
+- Allow to manage soft and hard `podAntiAffinity` for `Dex` from Cluster
+  and Services Configurations, with a default soft anti-affinity on hostname,
+  so that if it's possible each `Dex` pods will sit on a different infra node
+  (PR[#3614](https://github.com/scality/metalk8s/pull/3614))
+
 ### Removals
 
 - Removed the PDF support for documentation, replaced it with the HTML output

charts/dex.yaml

@@ -14,6 +14,16 @@ tolerations:
 
 replicaCount: '__var__(dex.spec.deployment.replicas)'
 
+strategy:
+  rollingUpdate:
+    maxUnavailable: 1
+  type: RollingUpdate
+
+
+# NOTE: We use `__var_quoted__` as `affinity` is rendered as YAML
+# so result will be quoted
+affinity: '__var_quoted__(salt.metalk8s_service_configuration.get_pod_affinity(dex.spec.deployment.affinity, {"app.kubernetes.io/name": "dex"}, "metalk8s-auth"))'
+
 # grpc support
 grpc:
   enabled: false

charts/dex/templates/deployment.yaml

@@ -11,6 +11,10 @@ spec:
   selector:
     matchLabels:
       {{- include "dex.selectorLabels" . | nindent 6 }}
+  {{- with .Values.strategy }}
+  strategy:
+    {{- toYaml . | nindent 4 }}
+  {{- end }}
   template:
     metadata:
       annotations:

salt/metalk8s/addons/dex/config/dex.yaml.j2

@@ -20,6 +20,14 @@ spec:
   # Deployment configuration
   deployment:
     replicas: 2
+    affinity:
+      podAntiAffinity:
+        soft:
+          - topologyKey: kubernetes.io/hostname
+        #   - topologyKey: my.second.important/topologyKey
+        #     weight: 42
+        # hard:
+        #   - topologyKey: kubernetes.io/hostname
 
   # Dex server configuration
   config:

salt/metalk8s/addons/dex/deployed/chart.sls

@@ -124,6 +124,10 @@ spec:
     matchLabels:
       app.kubernetes.io/instance: dex
       app.kubernetes.io/name: dex
+  strategy:
+    rollingUpdate:
+      maxUnavailable: 1
+    type: RollingUpdate
   template:
     metadata:
       annotations:
@@ -133,6 +137,8 @@ spec:
         app.kubernetes.io/instance: dex
         app.kubernetes.io/name: dex
     spec:
+      affinity: {% endraw -%}{{ salt.metalk8s_service_configuration.get_pod_affinity(dex.spec.deployment.affinity,
+        {"app.kubernetes.io/name": "dex"}, "metalk8s-auth") }}{%- raw %}
       containers:
       - args:
         - dex

tests/post/features/service_configuration.feature

@@ -29,3 +29,13 @@ Feature: Cluster and Services Configurations
         When we update the CSC 'spec.rules.node_exporter.node_filesystem_space_filling_up.warning.hours' to '48'
         And we apply the 'metalk8s.addons.prometheus-operator.deployed' state
         Then we have an alert rule 'NodeFilesystemSpaceFillingUp' in group 'node-exporter' with severity 'warning' and 'annotations.summary' equal to 'Filesystem is predicted to run out of space within the next 48 hours.'
+
+    Scenario: Dex pods spreading
+        Given the Kubernetes API is available
+        And we are on a multi node cluster
+        And we have a 'metalk8s-dex-config' CSC in namespace 'metalk8s-auth'
+        When we update the CSC 'spec.deployment.affinity' to '{"podAntiAffinity": {"hard": [{"topologyKey": "kubernetes.io/hostname"}]}}'
+        And we apply the 'metalk8s.addons.dex.deployed' state
+        And we wait for the rollout of 'deploy/dex' in namespace 'metalk8s-auth' to complete
+        Then pods with label 'app.kubernetes.io/name=dex' are 'Ready'
+        And each pods with label 'app.kubernetes.io/name=dex' are on a different node

tests/post/steps/test_service_configuration.py

@@ -33,6 +33,14 @@ def test_prometheus_rules_customization(host):
     pass
 
 
+@scenario(
+    "../features/service_configuration.feature",
+    "Dex pods spreading",
+)
+def test_dex_spread(host):
+    pass
+
+
 # }}}
 # Given {{{
 
@@ -65,6 +73,8 @@ def csc(host, ssh_config, version, k8s_client, name, namespace):
 
 @when(parsers.parse("we update the CSC '{path}' to '{value}'"))
 def update_csc(csc, path, value):
+    value = yaml.safe_load(value)
+
     csc_content = csc.get()
     utils.set_dict_element(csc_content, path, value)
     csc.update(csc_content, apply_config=False)