chore: improve the design docs

Signed-off-by: Suleyman Akbas <[email protected]>

README.md

@@ -107,12 +107,12 @@ After the CR is deployed, the following actions are executed:
 - A Logical Volume Manager (LVM) volume group named `vg1` is created, utilizing all available disks on the cluster.
 - A thin pool named `thin-pool-1` is created within `vg1`, with a size equivalent to 90% of `vg1`.
 - The TopoLVM Container Storage Interface (CSI) plugin is deployed, resulting in the launch of the `topolvm-controller` and `topolvm-node` pods.
-- A Storage Class and a Volume Snapshot Class are created, both named `lvms-vg1`. This facilitates storage provisioning for OpenShift workloads. The Storage Class is configured with the `WaitForFirstConsumer` volume binding mode that is utilized in a multi-node configuration to optimize the scheduling of pod placement. This strategy prioritizes the allocation of pods to nodes with the greatest amount of available storage capacity.
+- A storage class and a volume snapshot class are created, both named `lvms-vg1`. This facilitates storage provisioning for OpenShift workloads. The storage class is configured with the `WaitForFirstConsumer` volume binding mode that is utilized in a multi-node configuration to optimize the scheduling of pod placement. This strategy prioritizes the allocation of pods to nodes with the greatest amount of available storage capacity.
 - The LVMS system also creates two additional internal CRs to support its functionality:
-  * `LVMVolumeGroup` is generated and managed by LVMS to monitor the individual Volume Groups across multiple nodes in the cluster.
-  * `LVMVolumeGroupNodeStatus` is created by the VG Manager. This CR is used to monitor the status of volume groups on individual nodes in the cluster.
+  * `LVMVolumeGroup` is generated and managed by LVMS to monitor the individual volume groups across multiple nodes in the cluster.
+  * `LVMVolumeGroupNodeStatus` is created by the [Volume Group Manager](docs/design/vg-manager.md). This CR is used to monitor the status of volume groups on individual nodes in the cluster.
 
-Wait until the LVMCluster reaches the `Ready` status:
+Wait until the `LVMCluster` reaches the `Ready` status:
 
 ```bash
 $ oc get lvmclusters.lvm.topolvm.io my-lvmcluster
@@ -128,11 +128,11 @@ $ oc get pods -w
 
 The `topolvm-node` pod remains in the initialization phase until the `vg-manager` completes all the necessary preparations.
 
-Once all the pods have been launched, the LVMS is ready to manage your Logical Volumes and make them available for use in your applications.
+Once all the pods have been launched, the LVMS is ready to manage your logical volumes and make them available for use in your applications.
 
 ### Inspecting the storage objects on the node
 
-Prior to the deployment of the Logical Volume Manager Storage (LVMS), there are no pre-existing LVM Physical Volumes (PVs), Volume Groups (VGs), or Logical Volumes (LVs) associated with the disks.
+Prior to the deployment of the Logical Volume Manager Storage (LVMS), there are no pre-existing LVM physical volumes, volume groups, or logical volumes associated with the disks.
 
 ```bash
 sh-4.4# lsblk
@@ -228,7 +228,7 @@ spec:
 EOF
 ```
 
-Once the pod has been created and associated with the corresponding PVC, the PVC will be bound, and the pod will transition to the Running state in due course.
+Once the pod has been created and associated with the corresponding PVC, the PVC will be bound, and the pod will transition to the `Running` state in due course.
 
 ```bash
 $ oc get pvc,pods
@@ -246,21 +246,21 @@ To perform a full cleanup, follow these steps:
 
 1. Remove all the application pods which are using PVCs created with LVMS, and then remove all these PVCs.
 
-2. Ensure that there are no remaining LogicalVolume custom resources that were created by LVMS.
+2. Ensure that there are no remaining `LogicalVolume` custom resources that were created by LVMS.
 
     ```bash
     $ oc get logicalvolumes.topolvm.io 
     No resources found
     ```
 
-3. Remove the LVMCluster CR.
+3. Remove the `LVMCluster` CR.
     
     ```bash
     $ oc delete lvmclusters.lvm.topolvm.io my-lvmcluster
     lvmcluster.lvm.topolvm.io "my-lvmcluster" deleted
     ```
 
-4. Verify that the only remaining resource in the `openshift-storage` namespace is the operator.
+4. Verify that the only remaining resource in the `openshift-storage` namespace is the Operator.
 
     ```bash
     oc get pods -n openshift-storage

controllers/lvmcluster_controller.go

@@ -335,7 +335,7 @@ func (r *LVMClusterReconciler) getExpectedVgCount(ctx context.Context, instance
 	return vgCount, nil
 }
 
-// NOTE: when updating this, please also update docs/design/operator.md
+// NOTE: when updating this, please also update docs/design/reconciler.md
 type resourceManager interface {
 
 	// getName should return a camelCase name of this unit of reconciliation

docs/README.md

@@ -1,15 +1,6 @@
 # Contents
 
-1. [Reconcile][reconciler]
-2. [VG Manager][vg-manager]
-3. [CSI Units][topolvm-csi]
-4. [LVMO Units][lvmo-units]
-5. [Thin Pools][thin_pool]
-6. [Operator][operator]
-
-[reconciler]: design/reconciler.md
-[vg-manager]: design/vg-manager.md
-[topolvm-csi]: design/topolvm-csi.md
-[lvmo-units]: design/lvmo-units.md
-[thin_pool]: design/thin-pool.md
-[operator]: design/operator.md
+1. [Reconciler Design](design/reconciler.md)
+2. [The LVM Operator Manager](design/lvm-operator-manager.md)
+2. [The Volume Group Manager](design/vg-manager.md)
+5. [Thin Provisioning](design/thin-provisioning.md)

docs/design/lvm-operator-manager.md

@@ -0,0 +1,60 @@
+# The LVM Operator Manager
+
+The LVM Operator Manager runs the LVM Cluster controller/reconciler that manages the following reconcile units: 
+
+- [LVMCluster Custom Resource (CR)](#lvmcluster-custom-resource--cr-)
+- [TopoLVM CSI](#topolvm-csi)
+  * [CSI Driver](#csi-driver)
+  * [TopoLVM Controller](#topolvm-controller)
+  * [Topolvm Node and lvmd](#topolvm-node-and-lvmd)
+  * [TopoLVM Scheduler](#topolvm-scheduler)
+- [Storage Classes](#storage-classes)
+- [Volume Group Manager](#volume-group-manager)
+- [LVM Volume Groups](#lvm-volume-groups)
+- [Openshift Security Context Constraints (SCCs)](#openshift-security-context-constraints--sccs-)
+
+Upon receiving a valid [LVMCluster custom resource](#lvmcluster-custom-resource--cr-), the LVM Cluster Controller initiates the reconciliation process to set up the TopoLVM Container Storage Interface (CSI) along with all the required resources for using locally available storage through Logical Volume Manager (LVM).
+
+## LVMCluster Custom Resource (CR)
+
+The LVMCluster CR is a crucial component of the LVM Operator, as it represents the volume groups that should be created and managed across nodes with custom node selector, toleration, and device selectors. This CR must be created and edited by the user in the namespace where the Operator is also installed. However, it is important to note that only a single CR instance is supported. The user can choose to specify the devices in `deviceSelector.paths` field to be used for the volume group, or if no paths are specified, all available disks will be used. The `status` field is updated based on the status of volume group creation across nodes. It is through the LVMCluster CR that the LVM Operator can create and manage the required volume groups, ensuring that they are available for use by the applications running on the OpenShift cluster.
+
+The LVM Cluster Controller generates an LVMVolumeGroup CR for each `deviceClass` present in the LVMCluster CR. The Volume Group Manager controller manages the reconciliation of the LVMVolumeGroups. The LVM Cluster Controller also collates the device class status across nodes from LVMVolumeGroupNodeStatus and updates the status of LVMCluster CR.
+
+> Note: Each device class corresponds to a single volume group.
+
+## TopoLVM CSI
+
+The LVM Operator deploys the TopoLVM CSI plugin, which enables dynamic provisioning of local storage. For more detailed information about TopoLVM, consult the [TopoLVM documentation](https://github.com/topolvm/topolvm/tree/main/docs).
+
+### CSI Driver
+
+The *csiDriver* reconcile unit creates the TopoLVM CSIDriver resource.
+
+### TopoLVM Controller
+
+The *topolvmController* reconcile unit is responsible for deploying a single instance of the TopoLVM Controller plugin Deployment and ensuring that any necessary updates are made to the Deployment. As part of this process, an init container is used to generate openssl certificates that are utilized by the TopoLVM Controller. However, it should be noted that this method will be replaced with the use of cert-manager in the near future.
+
+### Topolvm Node and lvmd
+
+The *topolvmNode* reconcile unit is responsible for deploying and managing the TopoLVM Node plugin and lvmd daemon set. It scales the DaemonSet based on the node selector specified in the devicesClasses field in the LVMCluster CR. During initialization, an init container polls for the availability of the lvmd configuration file before starting the `lvmd` and `topolvm-node` containers.
+
+### TopoLVM Scheduler
+
+The TopoLVM Scheduler is **not** used in LVMS for scheduling Pods. Instead, the CSI StorageCapacity tracking feature is utilized by the Kubernetes scheduler to determine the Node on which to provision storage. This feature provides the necessary information to the scheduler regarding the available storage on each Node, allowing it to make an informed decision about where to place the Pod.
+
+## Storage Classes
+
+The *topolvmStorageClass* reconcile unit is responsible for creating and managing all storage classes associated with the device classes specified in the LVMCluster CR. Each storage class is named with a prefix of 'lvms-' followed by the name of the corresponding device class in the LVMCluster CR.
+
+## Volume Group Manager
+
+The *vgManager* reconcile unit is responsible for deploying and managing the [Volume Group Manager](./vg-manager.md).
+
+## LVM Volume Groups
+
+The *lvmVG* reconcile unit is responsible for deploying and managing the LVMVolumeGroup CRs. It creates individual LVMVolumeGroup CRs for each deviceClass specified in the LVMCluster CR. These CRs are then used by the [Volume Group Manager](./vg-manager.md) to create volume groups and generate the lvmd config file for TopoLVM.
+
+## Openshift Security Context Constraints (SCCs)
+
+The Operator requires elevated permissions to interact with the host's LVM commands, which are executed through `nsenter`. When deployed on an OpenShift cluster, all the necessary Security Context Constraints (SCCs) are created by the *openshiftSccs* reconcile unit. This ensures that the `vg-manager`, `topolvm-node`, and `lvmd` containers have the required permissions to function properly.

docs/design/reconciler.md

@@ -1,59 +1,30 @@
-# Operator design
-
-## Controllers and their managed resources
-
-### lvmcluster-controller
-
-- On receiving a valid LVMCluster CR, lvmcluster-controller reconciles the
-  following resource units for setting up [Topolvm](topolvm-repo) CSI and all
-  the supporting resources to use storage local to the node via Logical Volume
-  Manager (lvm)
-- *csiDriver*: Reconciles TopoLVM CSI Driver
-- *topolvmController*: Reconciles TopoLVM controller plugin
-- *lvmVG*: Reconciles volume groups from LVMCluster CR
-- *openshiftSccs*: Manages SCCs when the operator is run in Openshift
-  environment
-- *topolvmNode*: Reconciles TopoLVM nodeplugin along with lvmd
-- *vgManager*: Responsible for creation of Volume Groups
-- *topolvmStorageClass*: Manages storage class life cycle based on
-  devicesClasses in LVMCluster CR
-- The LVMO creates an LVMVolumeGroup CR for each deviceClass in the
-  LVMCluster CR. The LVMVolumeGroups are reconciled by the vgmanager controllers.
-- In addition to managing the above resource units, lvmcluster-controller collates
-  the status of deviceClasses across nodes from LVMVolumeGroupNodeStatus and
-  updates status of LVMCluster CR
-- `resourceManager` interface is defined in
-  [lvmcluster_controller.go][contorller]
-- Depending on the resource unit some of the methods can be no-op
-
-Note:
-- Above names refers to the struct which satisfies `resourceManager` interface
-- Please refer to the topolvm [design][topolvm-design] doc to know more about TopoLVM
-  CSI
-- Any new resource units should also implement `resourceManager` interface
-
-### Lifecycle of Custom Resources
-
-- [LVMCluster CR][lvmcluster] represents the volume groups that should be
-  created and managed across nodes with custom node selector, toleration and
-  device selectors
-- Should be created and edited by user in operator installed namespace
-- Only a single CR instance with a single volume group is supported.
-- The user can choose to specify the devices to be used for the volumegroup. 
-- All available disks will be used if no devicePaths are specified,.
-- All fields in `status` are updated based on the status of volume groups
-  creation across nodes
-
-Note:
-- Device Class and Volume Group can be read interchangeably
-- Multiple CRs exist to separate concerns of which component deployed by LVMO
-  updates which CR there by reducing multiple reconcile loops and colliding
-  requests/updates to Kubernetes API Server
-- Feel free to raise a github [issue][issue] for open discussions about API
-  changes if required
-
-[topolvm-repo]: https://github.com/topolvm/topolvm
-[topolvm-design]: https://github.com/topolvm/topolvm/blob/main/docs/design.md
-[controller]: ../../controllers/lvmcluster_controller.go
-[lvmcluster]: ../../api/v1alpha1/lvmcluster_types.go
-[issue]: https://github.com/openshift/lvm-operator/issues
+# Operator Design
+
+The LVM Operator consists of two managers: 
+
+- The [LVM Operator Manager](lvm-operator-manager.md) runs in a deployment called `lvms-operator` and manages multiple reconciliation units.
+- The [Volume Group Manager](vg-manager.md) runs in a daemon set called `vg-manager` and manages a single reconciliation unit.
+
+### Implementation Notes
+
+Each unit of reconciliation should implement the reconcileUnit interface. This will be run by the controller, and errors and success will be propagated to the status and events. This interface is defined in [lvmcluster_controller.go](../../controllers/lvmcluster_controller.go)
+
+```go
+type resourceManager interface {
+
+	// getName should return a camelCase name of this unit of reconciliation
+	getName() string
+
+	// ensureCreated should check the resources managed by this unit
+	ensureCreated(*LVMClusterReconciler, context.Context, lvmv1alpha1.LVMCluster) error
+
+	// ensureDeleted should wait for the resources to be cleaned up
+	ensureDeleted(*LVMClusterReconciler, context.Context, lvmv1alpha1.LVMCluster) error
+
+	// updateStatus should optionally update the CR's status about the health of the managed resource
+	// each unit will have updateStatus called induvidually so
+	// avoid status fields like lastHeartbeatTime and have a
+	// status that changes only when the operands change.
+	updateStatus(*LVMClusterReconciler, context.Context, lvmv1alpha1.LVMCluster) error
+}
+```