salt: Allow to provide featureGates for apiserver in bootstrap config

Add ability to provide some feature gates for kube apiserver in
bootstrap config that will be persisted after upgrade/salt highstate ...

Fixes: #3294

CHANGELOG.md

@@ -6,6 +6,10 @@
   Alertmanager are now stored in Loki database for persistence
   (PR[#3191](https://github.com/scality/metalk8s/pull/))
 
+- [#3294](https://github.com/scality/metalk8s/issues/3294) - Allow to manage
+  Kube Api Server feature gates from Bootstrap Configuration file
+  (PR[#3318](https://github.com/scality/metalk8s/pull/3318))
+
 ### Enhancements
 - Bump Kubernetes version to 1.20.6
   (PR[#3311](https://github.com/scality/metalk8s/pull/3311))

salt/_pillar/metalk8s.py

@@ -170,6 +170,7 @@ def ext_pillar(minion_id, pillar, bootstrap_config):  # pylint: disable=unused-a
             "networks": _load_networks(config),
             "metalk8s": metal_data,
             "proxies": config.get("proxies", {}),
+            "kubeApiServer": config.get("kubeApiServer", {}),
         }
 
         if not isinstance(metal_data["archives"], list):

salt/metalk8s/kubernetes/apiserver/installed.sls

@@ -26,6 +26,11 @@ include:
 {%- endif %}
 {%- set etcd_servers = etcd_servers | unique %}
 
+{%- set feature_gates = [] %}
+{%- for feature, value in pillar.kubeApiServer.get("featureGates", {}).items() %}
+{%-   do feature_gates.append(feature ~ "=" ~ value) %}
+{%- endfor %}
+
 Create kube-apiserver Pod manifest:
   metalk8s.static_pod_managed:
     - name: /etc/kubernetes/manifests/kube-apiserver.yaml
@@ -96,6 +101,9 @@ Create kube-apiserver Pod manifest:
           - --oidc-groups-claim=groups
           - '"--oidc-groups-prefix=oidc:"'
           - --v={{ 2 if metalk8s.debug else 0 }}
+          {% if feature_gates %}
+          - --feature-gates={{ feature_gates | join(",") }}
+          {%- endif %}
         requested_cpu: 250m
         volumes:
           - path: {{ encryption_k8s_path }}

salt/tests/unit/formulas/data/base_pillar.yaml

@@ -172,3 +172,4 @@ certificates:
         watched: true
       workload-plane-ingress:
         watched: true
+kubeApiServer: {}