doc: Add a design doc for user customization & persistence

Initially, user defined configurations such as changing the number of
replicas for a given deployment in a MetalK8s cluster are lost during
an upgrade/downgrade scenario.

This document explains some of the design choices considered while
designing a simplistic tool for MetalK8s that guarantees that user defined
configurations are persisted throughout.

Closes: #2233

docs/developer/architecture/index.rst

@@ -7,3 +7,4 @@ Architecture Documents
    deployment
    monitoring
    requirements
+   user_customization_and_persistence

docs/developer/architecture/user_customization_and_persistence.rst

@@ -0,0 +1,214 @@
+User Customization & Persistence
+================================
+
+Context
+-------
+
+.. todo::
+
+   This section will be handled by the requirements PR.
+
+
+Design Choices
+--------------
+
+:term:`ConfigMap` KV(Key-Value) store is chosen as a unified data access and
+storage media for user editable configurations in a MetalK8s cluster based on
+the above requirements for the following reasons:
+
+* The limited number of read/write operations even for multi-node setups means
+  that concurrency issues on CRUD operations will not lag even if we do not
+  implement cross-app locking.
+* Ability to support CRUD operations on ConfigMap's with CLI and UI easily
+  using our already existing python kubernetes module.
+* Guarantee of adaptability and ease of changing the design and implementation
+  in cases were customer needs evolve rapidly(modular design nature)
+* Ease of operation and a guarantee of no downtime in a MetalK8s cluster
+  especially when configuration changes are made on the fly.
+
+Rejected design choices
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Consul KV store
+~~~~~~~~~~~~~~~
+
+This approach offers a full fledge KV store with a /kv endpoint which allows
+CRUD operations on all KV data stored in it.
+Consul KV also allows access to past versions of objects and has an optimistic
+concurrency(CRUD operations will not lag with load).
+
+Note that, Consul KV store was rejected because managing operations such as
+performing full backups, system restores for a full fledged KV system
+requires time and much more efforts than the ConfigMap KV store which is
+simplistic and matches the requirements stated.
+
+
+Implementation Details
+----------------------
+
+A sample ConfigMap KV store can be defined with the following fields. These
+objects will be namespace bounded ensuring that sensitive information is
+restricted from a cluster-wide access.
+
+An example of such a ConfigMap KV store:
+
+   .. code-block:: yaml
+
+       apiVersion: v1
+       kind: ConfigMap
+       metadata:
+         namespace: <metalk8s-namespace>
+         name: configmap-kv
+       data:
+         <key>: <value>
+         <key>: <value>
+         <key>: <value>
+
+**Use case 1:**
+
+Configure and store the number of replicas for all Deployments found in the
+`metalk8s-monitoring` namespace using the ConfigMap KV store format.
+
+   .. code-block:: yaml
+
+       apiVersion: v1
+       kind: ConfigMap
+       metadata:
+         namespace: metalk8s-monitoring
+         name: <example-name>
+       data:
+         replicas.grafana: 2
+         replicas.prometheus: 2
+         replicas.alertmanager: 1
+
+**Use case 2:**
+
+Configure and store the Dex configuration parameters for Active Directory,
+OpenLDAP and Static user store.
+
+   .. code-block:: yaml
+
+       apiVersion: v1
+       kind: ConfigMap
+       metadata:
+         namespace: metalk8s-monitoring
+         name: <example-name>
+       data:
+         connector.ldap.host: 'ldap.example.com:636'
+         connector.ldap.startTLS: true
+         connector.ldap.bindDN: ''
+         connector.ldap.bindPW: ''
+         connector.ldap.rootCA: ''
+
+         connector.ldap.userSearch.baseDN: ''
+         connector.ldap.userSearch.filter: '(objectClass=inetOrgPerson)'
+         connector.ldap.userSearch.username: ''
+         connector.ldap.userSearch.idAttr: 'DN'
+         connector.ldap.userSearch.emailAttr: ''
+         connector.ldap.userSearch.nameAttr: 'cn'
+
+         connector.ldap.userSearch.groupSearch.baseDN: ''
+         connector.ldap.userSearch.groupSearch.filter: '(objectClass=groupOfUniqueNames)'
+         connector.ldap.userSearch.groupSearch.userAttr: 'DN'
+         connector.ldap.userSearch.groupSearch.groupAttr: 'uniqueMember'
+         connector.ldap.userSearch.groupSearch.nameAttr: 'cn'
+
+         # setting this to false should disable the use of static users
+         # for users who own no LDAP or AD, add configs allowing provisioning
+         # more static users or changing the password for the default user.
+         # TBD
+         connector.staticUser.enabled: true
+
+**Use case 3:**
+
+Configure and store Alertmanager notification receivers so that alerts can
+be dispatched to the customers choice of notifier(Slack, email, hipchat).
+
+   .. code-block:: yaml
+
+       apiVersion: v1
+       kind: ConfigMap
+       metadata:
+         namespace: metalk8s-monitoring
+         name: <example-name>
+       data:
+
+        channel.smtp.smtp_smarthost: 'smtp.gmail.com:587'
+        channel.smtp.smtp_from: '[email protected]'
+        channel.smtp.smtp_auth_username: '[email protected]'
+        channel.smtp.smtp_auth_password: 'barfoo'
+
+        channel.slack.slack_api_url: 'https://hooks.slack.com/services/abc123'
+
+        channel.hipchat.hipchat_auth_token: '1234556789'
+        channel.hipchat.hipchat_url: 'https://hipchat.foobar.org/'
+
+        # route.group_wait: '30s'
+        # route.repeat_interval: '15m'
+        # route.group_interval: '5m'
+        route.receiver: 'default'
+
+        routes.match.severity: 'email'
+        routes.receiver: 'default'
+        routes.match.severity: 'slack'
+        routes.match.receiver: 'slack_alert'
+
+        receivers.name: 'default'
+        receivers.email_configs.to: '[email protected]'
+
+        receivers.name: 'slack_alert'
+        receivers.slack_configs.channel: '#devops'
+        receivers.slack_configs.send_resolved: 'true'
+
+Iteration 1
+~~~~~~~~~~~
+
+- Define and deploy new namespaced bounded ConfigMap KV stores that will hold
+  user configurations as listed in the requirements
+- Template and render Deployment and Pod manifests that will make use of
+  this user customizable configurations using default pillar values
+- Provide a CLI tool(Bash script) for changing any of the user configurations:
+
+    - Count of replicas for chosen Deployments(Prometheus)
+    - Adding/modifying/removing a Dex authentication connector(OpenLDAP, AD and
+      staticUser store)
+    - Adding/modifying/removing Alertmanager notification configuration
+
+Iteration 2
+~~~~~~~~~~~
+
+- Provide a UI interface that allows CRUD operations on all user customizable
+  settings based on the above requirements
+- Provide a UI interface for adding, updating and deleting service specific
+  configurations for example Dex-LDAP  connector integration.
+- Provide a UI interface for listing MetalK8s available/supported Dex
+  authentication Connectors
+- Provide a UI interface for enabling or disabling Dex authentication
+  connectors(LDAP, Active Directory, StaticUser store)
+- Provide a UI interface for changing the number of replicas for a chosen set
+  of MetalK8s deployments(Prometheus, ....)
+- Add a UI interface for listing Alertmanager notification systems MetalK8s
+  will support(Slack, email, hipchat)
+- Provide a UI interface for adding, modifying and deleting Alertmanager
+  configurations from the listing above
+
+Documentation
+-------------
+
+In the Operational Guide:
+
+* Document how to customize or change any given setting using the CLI tool
+* Document how to customize or change any given setting using the UI interface
+
+Test Plan
+---------
+
+Dex Static User authentication is currently covered in our test-suite and it
+will make sense to cover atleast one other authentication connector with the
+easiest being LDAP since we readily have access to OpenLDAP Docker images and
+automating this process is possible on Scality Cloud.
+
+CRUD Operations on the ConfigMap KV store should be tested both from the UI
+test libraries and the CLI tool provided(An initial idea here is to have a
+simplistic configurator.sh bash script(CLI tool) performing sets of CRUD
+operations on a sample ConfigMap KV store).

docs/glossary.rst

@@ -31,6 +31,18 @@ Glossary
      and from where the cluster will be deployed to other machines. It also
      serves as the entrypoint for upgrades of the cluster.
 
+
+   ConfigMap
+     A ConfigMap is a Kubernetes object that allows one to store general
+     configuration information such as environment variables in a key-value
+     pair format.
+     ConfigMaps can only be applied to namespaces and once created, they can
+     be updated automatically without the need of restarting containers that
+     depend on it.
+
+     |see K8s docs|
+     `ConfigMap <https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#understanding-configmaps-and-pods/>`_.
+
    Controller Manager
    ``kube-controller-manager``
      The Kubernetes controller manager embeds the core control loops shipped